CN115730003A

CN115730003A - Method and system for automatic data expansion and relation mining

Info

Publication number: CN115730003A
Application number: CN202211485678.4A
Authority: CN
Inventors: 郑汉军; 陈学远; 黄惠海; 王磊; 黄键; 陈建章
Original assignee: Xiamen Anscen Network Technology Co ltd
Current assignee: Xiamen Anscen Network Technology Co ltd
Priority date: 2022-11-24
Filing date: 2022-11-24
Publication date: 2023-03-03

Abstract

The invention provides a method for data automatic expansion and relation mining, which comprises the following steps: acquiring various information of nodes connected with resources; dynamically acquiring resource table information based on the database of the resource node, and further importing a field information list corresponding to the resource table; further creating and setting a resource table mapping model; calling a scene analysis module to obtain and store specific object data information; rendering the resource nodes according to the obtained object data information and displaying the resource nodes on an analysis billboard. By combining the analysis scene and the resource table mapping model, the technology has stronger practicability, wider applicable scenes and more flexible analysis; according to the node graph operation function provided by the analysis billboard, the relation graph is more visual and concise; the invention can flexibly analyze different resource tables and package the acquired data into corresponding entity object information and relation information of the entity object.

Description

Method and system for data automatic expansion and relation mining

Technical Field

The invention belongs to the technical field of big data mining, and particularly relates to a method and a system for data automatic expansion and relation mining.

Background

In the background of the information age, the value of data lies in the process of generation, mining and collection, wherein the most important data lies in the mining of the data.

The method aims to meet new requirements on data analysis under the condition of big data, meet the requirements of real-time analysis and application of the big data, solve the problem that the current decentralized, simplified and offline analysis modes are not suitable for study and judgment and analysis work under the big data environment, and meet the working requirements of visualization study and judgment of the big data of each unit by uniformly constructing a set of complete, flexible and practical data visualization analysis.

Based on a brand-new visual data analysis concept, a large amount of various dispersed data are displayed in a graph mode, association, aggregation, characteristics and the like between the data and the data are described and displayed in the graph mode, and furthermore, public elements and association hidden in the data are discovered and revealed by using a plurality of graph analysis methods (association analysis, network analysis, path analysis and the like). The user is helped to convert a large amount of unknown-quality, low-relevance and low-value information into a small amount of easily-understood, high-relevance, high-value and operable information, so that the analysis work is helped.

In view of this, it is very significant to provide a method and system for data automation extension and relationship mining.

Disclosure of Invention

The invention provides a method and a system for automatically expanding data and mining relationships, aiming at solving the problems that in the existing data relationship analysis system, resource data are required to be arranged in a fixed information table defined by scenes when different scenes are analyzed, and the traditional analysis can lead the fixed information table to be too bloated and the relationship data is difficult to trace to the source when the related data is obtained by analyzing the fixed information table.

In a first aspect, the present invention provides a method for data automation extension and relationship mining, including the following steps:

acquiring various information of nodes connected with resources;

dynamically acquiring resource table information based on the database of the resource node, and further importing a field information list corresponding to the resource table;

further creating and setting a resource table mapping model;

calling a scene analysis module to acquire and store specific object data information; and

rendering the resource nodes according to the obtained object data information and displaying the resource nodes on an analysis billboard.

Preferably, the database of the resource node comprises mysql, elasticsearch, mongodb, hbase; acquiring various information of nodes connected by resources, wherein the information comprises names, types, IP addresses, ports, database names, user names and passwords; the acquired resource table information comprises a resource table name, a resource table Chinese name, a resource node name and a database name; and importing a field information list corresponding to the resource table, wherein the field information list comprises field names, types and field names.

Further preferably, the creating and setting of the resource table mapping model specifically includes:

firstly, creating information of the resource table mapping model, wherein the information comprises a model name, model creation time and model creation personnel;

an execution step of creating a model, namely pulling out a set input parameter on a model design billboard, pulling out a resource table, and setting analysis conditions and fields needing analysis on the resource table, wherein the analysis conditions comprise conditions of being equal to, not equal to, less than or equal to, greater than or equal to, containing, empty and not empty;

selecting a certain field or a plurality of field combinations as the input parameter according to the acquired data, taking the selected field or the plurality of field combinations as the input parameter of the current step, and operating the steps according to different configured analysis conditions and analysis fields;

if the next step exists, repeating the previous step for configuration;

configuring mapping information and object relation information of an output object, an output data field and a field for mapping a corresponding object;

storing resource table mapping model configuration information to a model resource table;

and configuring the resource table mapping model associated information to which the scene belongs.

Further preferably, the acquiring of the specific object data information by the scene analysis module specifically includes:

the analysis billboard calls a scene analysis operation program of the system, and the transmitted conditions comprise values of input parameter addresses, scene information and resource table mapping model groups needing to be analyzed;

if the analysis condition does not particularly select the resource table mapping model group, calculating all resource table mapping models related to the scene information, and if the condition exists, calculating by using the transmitted resource table mapping model group;

the system program puts the acquired resource table mapping model information into a model data analysis pool through redis, and a data analysis thread pool of the system program allocates corresponding threads according to data in the model data analysis pool to perform data analysis;

the data analysis thread analyzes and obtains corresponding data according to the input data, the input types, the execution step sequence of the models and the related resource table of the resource table mapping model, and extracts corresponding data output entity object data and entity relation data according to the output entity object mapping information and the output entity relation information;

and finally returning the entity object data and the entity relation data.

Further preferably, rendering and displaying the resource nodes on the analysis billboard according to the obtained object data information specifically includes:

acquiring entity object data and entity relation data returned by scene analysis;

rendering icons of corresponding entity object nodes on the analysis billboard according to the entity object data, and assigning corresponding data of the entity object to the corresponding node icons;

according to the entity relationship data, two corresponding entity nodes are found on the analysis billboard, a connecting line of the two nodes is drawn, and the corresponding entity relationship data is assigned to the connecting line;

according to a screening tool provided by the analysis billboard, inputting corresponding conditions to screen nodes and lines, and removing the nodes and lines which do not meet the conditions;

according to the layout tool of the analysis billboard, selecting corresponding layout to typeset the nodes, and typesetting the nodes and the connecting lines according to a certain calculation rule, so that the node relation map is more orderly and visual;

saving the node relationship graph to the Janusgraph graph database.

In a second aspect, the present invention further provides a system for data automation extension and relationship mining, including:

an acquisition module: the system comprises a database, a scene analysis module and a resource connection module, wherein the database is used for acquiring various information of nodes connected with resources, dynamically acquiring resource table information based on the database of the resource nodes and calling the scene analysis module to acquire specific object data information;

a creation module: used for creating and setting up the resource table mapping model;

a scene analysis module: the system is used for calling a scene analysis module to acquire and store specific object data information;

an analysis billboard module: the resource node is used for rendering according to the acquired object data information;

a display module: and the resource nodes are rendered according to the acquired object data information and displayed on the analysis billboard.

In a third aspect, an embodiment of the present invention provides an electronic device, including: one or more processors; storage means for storing one or more programs which, when executed by one or more processors, cause the one or more processors to carry out a method as described in any one of the implementations of the first aspect.

In a fourth aspect, the present invention provides a computer-readable storage medium, on which a computer program is stored, which, when executed by a processor, implements the method described in any implementation manner of the first aspect.

Compared with the prior art, the beneficial results of the invention are as follows:

(1) By combining the analysis scene and the resource table mapping model, the technology has stronger practicability, wider applicable scene and more flexible analysis; according to the node graph operation function provided by the analysis billboard, the relation graph is more visual and concise; by the method and the device, different resource tables can be flexibly analyzed, and the acquired data is packaged into corresponding entity object information and relationship information of the entity object.

(2) The data relation mining visualization technology analyzes information from different angles through rich graphical display modes; the machine can acquire business knowledge, automatically complete a large amount of mental work, liberate manpower and reduce time consumption; meanwhile, visual analysis means such as correlation analysis and flow direction analysis based on the map are provided, so that difficult analysis and judgment work is simplified.

Drawings

The accompanying drawings are included to provide a further understanding of the embodiments and are incorporated in and constitute a part of this specification. The drawings illustrate embodiments and together with the description serve to explain the principles of the invention. Other embodiments and many of the intended advantages of embodiments will be readily appreciated as they become better understood by reference to the following detailed description. The elements of the drawings are not necessarily to scale relative to each other. Like reference numerals designate corresponding similar parts.

FIG. 1 is an exemplary device architecture diagram in which one embodiment of the present invention may be applied;

FIG. 2 is a schematic flow chart diagram of a method of data automation extension and relationship mining in accordance with an embodiment of the present invention;

FIG. 3 is a flowchart illustrating step S13 of the method for data automation extension and relationship mining according to the embodiment of the present invention;

FIG. 4 is a flowchart illustrating step S14 of the method for data automation extension and relationship mining according to the embodiment of the present invention;

FIG. 5 is a flowchart illustrating step S15 of the method for data automation extension and relationship mining according to the embodiment of the present invention;

FIG. 6 is a schematic flow diagram of a system for automated data expansion and relationship mining, in accordance with an embodiment of the present invention;

FIG. 7 is a schematic diagram of a computer apparatus suitable for use with an electronic device to implement an embodiment of the invention.

Detailed Description

In the following detailed description, reference is made to the accompanying drawings, which form a part hereof, and in which is shown by way of illustration specific embodiments in which the invention may be practiced. In this regard, directional terminology, such as "top," "bottom," "left," "right," "up," "down," etc., is used with reference to the orientation of the figures being described. Because components of embodiments can be positioned in a number of different orientations, the directional terminology is used for purposes of illustration and is in no way limiting. It is to be understood that other embodiments may be utilized and logical changes may be made without departing from the scope of the present invention. The following detailed description is, therefore, not to be taken in a limiting sense, and the scope of the present invention is defined by the appended claims.

It should be understood that the number of terminal devices, networks, and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.

Fig. 1 illustrates an exemplary system architecture 100 for a method for processing information or an apparatus for processing information to which embodiments of the present invention may be applied.

As shown in fig. 1, the system architecture 100 may include

terminal devices

101, 102, 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the

terminal devices

101, 102, 103 and the server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.

A user may use

terminal devices

101, 102, 103 to interact with a server 105 over a network 104 to receive or send messages or the like. Various communication client applications, such as a web browser application, a shopping application, a search application, an instant messaging tool, a mailbox client, social platform software, etc., may be installed on the

terminal devices

101, 102, 103.

The

terminal devices

101, 102, 103 may be various electronic devices having communication functions, including but not limited to smart phones, tablet computers, laptop portable computers, desktop computers, and the like.

The server 105 may be a server that provides various services, such as a background information processing server that processes check request information transmitted by the

terminal apparatuses

101, 102, 103. The background information processing server may analyze and perform other processing on the received verification request information, and obtain a processing result (e.g., verification success information used to represent that the verification request is a legal request).

It should be noted that the method for processing information provided by the embodiment of the present invention is generally executed by the server 105, and accordingly, the apparatus for processing information is generally disposed in the server 105. In addition, the method for sending information provided by the embodiment of the present invention is generally executed by the

terminal equipment

101, 102, 103, and accordingly, the apparatus for sending information is generally disposed in the

terminal equipment

101, 102, 103.

The server may be hardware or software. When the server is hardware, it may be implemented as a distributed server cluster formed by multiple servers, or may be implemented as a single server. When the server is software, it may be implemented as a plurality of software or software modules (for example, to provide distributed services), or may be implemented as a single software or a plurality of software modules, and is not limited in particular herein.

Different resource tables have these different table fields, with different table fields having different practical meanings. If the resource data are required to be treated in a fixed information table defined by scenes in the traditional analysis of different scenes, the fixed information table is too bloated in the traditional analysis and the tracing of the relationship data is difficult when the related data are obtained by analyzing the fixed information table.

The invention provides a data automatic expansion and relation mining mode, which is used for mining and displaying data relations. The method mainly comprises scene configuration of data analysis, data mining and data relation display. The scene configuration of data analysis makes the technology more adaptive; the data mining is information of scene configuration with data analysis, and corresponding resource data is calculated and analyzed; and the relation display of the data is realized by generating entity object nodes and entity relation lines for graphical display according to the relevant data mined from the data.

FIG. 2 shows an embodiment of the present invention discloses a method for data automation extension and relationship mining, and as shown in FIG. 2, the method comprises the following steps:

s11, acquiring various information of nodes connected by resources;

s12, dynamically acquiring resource table information based on the database of the resource node, and further importing a field information list corresponding to the resource table;

specifically, the database of the resource node type includes mysql, elasticsearch, mongodb, hbase, and the like, the node content for acquiring resource connection includes information such as name, type, IP address, port, database name, user name, password, and the like, and the resource table information is dynamically acquired through the database of the resource node, including: the resource table name, the Chinese name in the resource table, the resource node name and the database name; and a field information list corresponding to the resource table is required to be imported, wherein the field information list comprises field names, types and field names.

S13, further establishing and setting a resource table mapping model;

in this embodiment, as shown in fig. 3, creating and setting the resource table mapping model specifically includes:

s131, firstly, information of the resource table mapping model is created, wherein the information comprises a model name, model creation time and model creation personnel. If a transaction fund resource table mapping model is created, the name is a transaction model.

S132, establishing a model, namely pulling out a code address node (entry parameter value) on a model design billboard, pulling out a resource table, and setting analysis conditions and fields needing to be analyzed on the resource table, wherein the analysis conditions comprise conditions of being equal to, not equal to, less than or equal to, more than or equal to, containing, empty, not empty and the like. If the trade model, the code address is the card number, the trade resource table is drawn up, and the analysis condition is that the comparison between the card number and the trade account number field is equal.

And S133, selecting a field or a combination of a plurality of fields as the entry parameter of the data acquired in the second step, and performing the operation of the step according to different configured analysis conditions and analysis fields.

S134, if any, repeating step 133 to carry out configuration.

S135, configuring mapping information and object relationship information of the output object, the output data field, and mapping the field of the corresponding object, for example, mapping the transaction model as an object, and mapping the card number object: including card number value, card number information, card holder information, etc.; extracting object relation of the transaction model, wherein the object relation is as follows: including information such as transfer card number, receiving card number, transfer amount, transfer time, transfer location, etc.

S136, the resource table is stored to map the model configuration information to the model resource table.

And S137, configuring the resource table mapping model associated information to which the scene belongs.

S14, calling a scene analysis module to acquire and store specific object data information;

in this embodiment, as shown in fig. 4, the scene analysis obtains specific object data, and the specific steps are as follows:

and S141, calling a scene analysis operation program of the system by the analysis billboard, wherein the transmitted conditions comprise the values of the input parameter addresses, scene information and resource table mapping model groups to be analyzed.

And S142, if the analyzed conditions do not particularly select the resource table mapping model group, calculating all resource table mapping models related to the scene information, and if the analyzed conditions do not particularly select the resource table mapping model group, calculating all resource table mapping models related to the scene information by using the incoming resource table mapping model group.

And S143, the system program puts the acquired resource table mapping model information into a model data analysis pool through redis.

And S144, the data analysis thread pool of the system program distributes corresponding threads according to the data in the model data analysis pool to perform data analysis.

S145, the data analysis thread analyzes and obtains corresponding data according to the parameter data, parameter types, execution step sequences of the models and the related resource tables of the resource table mapping models, and extracts corresponding data output entity object data and entity relation data according to the output entity object mapping information and the output entity relation information.

And S146, returning the entity object data and the entity relation data.

And S15, rendering the resource nodes according to the acquired object data information and displaying the resource nodes on an analysis billboard.

The analysis billboard performs operations such as node rendering and relation drawing according to the return value, and the flow is shown in fig. 5, and the specific steps are as follows:

and S151, acquiring entity object data and entity relation data returned by scene analysis.

S152, rendering the icons of the corresponding entity object nodes on the analysis board according to the entity object data, and assigning the data of the corresponding entity objects to the corresponding node icons.

S153, according to the entity relation data, two corresponding opposite entity nodes are found on the analysis billboard, a connection line of the two nodes is drawn, and corresponding entity relation data is assigned to the connection line.

And S154, inputting corresponding conditions to screen nodes and lines according to the screening tool provided by the analysis billboard, and removing the nodes and lines which do not meet the conditions.

S155, according to the analysis billboard layout tool, selecting corresponding layout to typeset the nodes, and typesetting the nodes and the connecting lines according to a certain calculation rule, so that the node relation map is more orderly and visually.

And S156, storing the node relation graph to the Janusgraph graph database.

Further, in this embodiment, a specific scheme of a specific embodiment of the present invention is as follows:

s1, using information of an entity and information of an entity relation, and adding the information into a resource table mapping configuration set;

s2, using the resource table information and adding the resource table information into a resource data set of a resource table billboard;

s3, setting corresponding resource conditions corresponding to the input parameter values for the resource tables in the resource data set, and combining entity information and entity relation information in the mapping configuration set to combine complete resource table mapping model information;

s4, using the scene information and storing the scene information into a scene mapping configuration set;

s5, mapping model information by using a resource table, and inputting the model information into a scene mapping configuration set;

s6, carrying out corresponding association on data in the scene mapping configuration set, and combining the data into a group scene mapping information according to the data association resource table mapping model information of the scene information;

s7, using the scene mapping information and storing the scene mapping information into an analysis billboard data analysis set;

and S8, acquiring input parameters to be analyzed, analyzing a resource table according to scene mapping information in a data analysis set of the analysis billboard, and acquiring entity object information data and entity relation information data. And storing the data into the data set of the analysis billboard.

And S9, rendering nodes and connecting lines in the analysis billboard according to the entity object data and the entity relation data in the data set to generate a relation graph.

S10, analyzing the relationship information source of the relationship of the entity object by using the data in the data set of the billboard, and acquiring the node data of the entity object and the data of the entity relationship;

and S11, analyzing the kangsraph use relation graph and storing the kanusgraph database.

Further, step S3 specifically includes:

s31, using code address information and storing the code address information into a resource table mapping configuration set;

s32, using the entity object information and the entity relation information and storing the entity object information and the entity relation information into a resource table mapping configuration set;

s33, using the resource table information and storing the resource table information in a resource table mapping configuration set;

s34, using the scene information and storing the scene information into the mapping configuration combination of the resource table;

s35, acquiring code address information and resource table information from the configuration set, taking the code address information as a parameter entry condition, associating corresponding fields of the resource table, selecting a parameter entry and field comparison condition, screening out corresponding fields after the resource table is analyzed as an output value of the next step, and storing the configuration in the resource table mapping configuration step set;

s36, acquiring input field values from the configuration step set as access conditions, acquiring resource table information of the configuration set, associating the access conditions with corresponding fields of the resource table, selecting comparison conditions, screening out corresponding fields after resource table analysis as input values, and storing configuration in the resource table configuration step set;

s37, if the multi-step operation exists, repeating the step S33;

s38, acquiring the analyzed data field from the configuration step set, acquiring entity object information from the configuration set, mapping the data field and the field of the entity object information, and storing the mapping information into a resource table mapping set;

s39, acquiring the analyzed data field from the configuration step set, acquiring entity relationship information from the configuration set, mapping the data field and the field of the entity relationship information, and storing the mapped information into the resource table configuration step set;

s310, acquiring resource table mapping model information from the configuration step set, acquiring scene information from the configuration set, associating the resource table mapping model information with the scene information, and storing the associated information into a scene information table.

Further, step S8 specifically includes:

s81, the system firstly acquires a code address value, analyzed scene information and resource table mapping model group information which are transmitted by an analysis billboard;

s82, acquiring configured resource table mapping model information according to the scene information and the resource table mapping model group information;

s83, executing the resource table mapping model through different threads according to the obtained resource table mapping model information group, analyzing data of the resource table, and obtaining entity object information and entity object relation information data;

s84, acquiring entity object information and entity object relation information data and returning the information and the data to the analysis billboard;

further, step S9 specifically includes:

s91, acquiring entity object information and entity relation information data calculated by a system through an analysis billboard;

s92, rendering the entity object nodes to an analysis billboard according to the entity object information, and assigning corresponding attribute values to the entity object nodes;

s93, rendering lines of the relation connecting lines among the nodes according to the entity relation information, and assigning corresponding relation data and the relation data of the original resource table to the lines of the relation connecting lines;

and S94, screening the nodes of the billboard according to the screening conditions provided by the analysis billboard, for example, screening the nodes with the transaction amount less than one hundred thousand in a financial transaction scene, deleting the nodes, and the like.

By combining the analysis scene and the resource table mapping model, the method has stronger technical practicability, wider applicable scene and more flexible analysis; according to the node graph operation function provided by the analysis billboard, the relation graph is more visual and concise.

In a second aspect, an embodiment of the present invention further provides a system for data automation extension and relationship mining, as shown in fig. 6, including an obtaining module 61, a creating module 62, a scene analysis module 63, an analysis kanban module 64, and a presentation module 65.

Wherein the obtaining module 61: the system comprises a database, a scene analysis module and a resource connection module, wherein the database is used for acquiring various information of nodes connected with resources, dynamically acquiring resource table information based on the database of the resource nodes and calling the scene analysis module to acquire specific object data information; the creation module 62: the resource table mapping model is used for establishing and setting a resource table mapping model; the scene analysis module 63: the system is used for calling a scene analysis module to acquire and store specific object data information; the analysis billboard module 64: the resource node is used for rendering according to the acquired object data information; the display module 65: and the resource nodes are rendered according to the acquired object data information and displayed on the analysis billboard.

Referring now to fig. 7, a schematic diagram of a computer apparatus 600 of an electronic device (e.g., the server or terminal device shown in fig. 1) suitable for implementing embodiments of the present invention is shown. The electronic device shown in fig. 7 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present invention.

As shown in fig. 7, the computer apparatus 600 includes a Central Processing Unit (CPU) 601 and a Graphics Processing Unit (GPU) 602, which can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 603 or a program loaded from a storage section 609 into a Random Access Memory (RAM) 606. In the RAM 604, various programs and data necessary for the operation of the apparatus 600 are also stored. The CPU 601, GPU602, ROM 603, and RAM 604 are connected to each other via a bus 605. An input/output (I/O) interface 606 is also connected to bus 605.

The following components are connected to the I/O interface 606: an input portion 607 including a keyboard, a mouse, and the like; an output section 608 including a display such as a Liquid Crystal Display (LCD) and a speaker; a storage section 609 including a hard disk and the like; and a communication section 610 including a network interface card such as a LAN card, a modem, or the like. The communication section 610 performs communication processing via a network such as the internet. The driver 611 may also be connected to the I/O interface 606 as needed. A removable medium 612 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 611 as necessary, so that a computer program read out therefrom is mounted into the storage section 609 as necessary.

In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 610 and/or installed from the removable media 612. The computer programs, when executed by a Central Processing Unit (CPU) 601 and a Graphics Processor (GPU) 602, perform the above-described functions defined in the method of the present invention.

It should be noted that the computer readable medium of the present invention can be a computer readable signal medium or a computer readable medium or any combination of the two. The computer readable medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor device, apparatus, or any combination of the foregoing. More specific examples of the computer readable medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present invention, a computer readable medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution apparatus, device, or apparatus. In the present invention, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution apparatus, device, or apparatus. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.

Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based devices that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The modules described in the embodiments of the present invention may be implemented by software or hardware. The modules described may also be provided in a processor.

As another aspect, the present invention also provides a computer-readable medium, which may be contained in the electronic device described in the above embodiments; or may exist separately without being assembled into the electronic device. The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to perform the method steps as described in the first aspect of the invention.

The foregoing description is only exemplary of the preferred embodiments of the invention and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the invention according to the present invention is not limited to the specific combination of the above-mentioned features, but also encompasses other embodiments in which any combination of the above-mentioned features or their equivalents is possible without departing from the scope of the invention as defined by the appended claims. For example, the above features and the technical features (but not limited to) having similar functions disclosed in the present invention are mutually replaced to form the technical solution.

Claims

1. A method for data automation extension and relationship mining is characterized by comprising the following steps:

acquiring various information of nodes connected by resources;

further creating and setting a resource table mapping model;

2. The method of automated data expansion and relationship mining according to claim 1, wherein the database of resource nodes comprises mysql, elasticsearch, mongodb, hbase; acquiring various information of nodes connected by resources, wherein the information comprises names, types, IP addresses, ports, database names, user names and passwords; the acquired resource table information comprises a resource table name, a resource table Chinese name, a resource node name and a database name; and importing a field information list corresponding to the resource table, wherein the field information list comprises field names, types and field names.

3. The method for automated data extension and relationship mining of claim 2, wherein creating and setting a resource table mapping model specifically comprises:

an execution step of creating a model, namely pulling out a set input parameter value on a model design billboard, pulling out a resource table, and setting analysis conditions and fields needing to be analyzed for the resource table, wherein the analysis conditions comprise conditions of being equal to, not equal to, less than or equal to, greater than or equal to, including, being empty and not being empty;

selecting a field or a combination of a plurality of fields as a parameter according to the acquired data, taking the selected field or combination of a plurality of fields as the parameter of the current step, and operating the steps according to different configured analysis conditions and analysis fields;

if the next step exists, repeating the previous step for configuration;

configuring mapping information and object relation information of an output object, an output data field and a field of a mapping corresponding object;

saving resource table mapping model configuration information to a model resource table;

4. The method for data automation extension and relationship mining as claimed in claim 3, wherein the scene analysis module obtaining specific object data information specifically includes:

if the analyzed conditions do not particularly select the resource table mapping model group, calculating all the resource table mapping models related to the scene information, and if the resource table mapping models are related to the scene information, calculating all the resource table mapping models by using the introduced resource table mapping model group;

the data analysis thread acquires corresponding data according to the parameter input data, the parameter input type, the execution step sequence of the model and the related resource table analysis of the resource table mapping model, and extracts corresponding data output entity object data and entity relation data according to the output entity object mapping information and the output entity relation information;

and finally, returning the entity object data and the entity relation data.

5. The method for automated data expansion and relationship mining according to claim 4, wherein rendering and displaying the resource nodes on an analysis billboard according to the obtained object data information specifically comprises:

according to the entity relationship data, two corresponding entity nodes are found on the analysis billboard, a connecting line of the two nodes is drawn, and the connecting line is assigned with corresponding entity relationship data;

according to an analysis billboard layout tool, selecting corresponding layout to typeset the nodes, and typesetting the nodes and the connecting lines according to a certain calculation rule, so that the node relation graph is more orderly and visual;

saving the node relationship graph to the Janusgraph graph database.

6. A system for automated data expansion and relationship mining, comprising:

an acquisition module: the system comprises a database, a scene analysis module, a resource connection module and a resource management module, wherein the database is used for storing resource table information of nodes connected with resources;

a scene analysis module: the scene analysis module is used for calling to acquire and store specific object data information;

7. An electronic device, comprising:

one or more processors;

storage means for storing one or more programs;

when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-5.

8. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1 to 5.