CN106897446A - A kind of data flow method for visualizing and device - Google Patents
A kind of data flow method for visualizing and device Download PDFInfo
- Publication number
- CN106897446A CN106897446A CN201710120952.0A CN201710120952A CN106897446A CN 106897446 A CN106897446 A CN 106897446A CN 201710120952 A CN201710120952 A CN 201710120952A CN 106897446 A CN106897446 A CN 106897446A
- Authority
- CN
- China
- Prior art keywords
- list
- node
- target data
- back end
- relation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24564—Applying rules; Deductive queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Abstract
The present invention provides a kind of data flow method for visualizing and device, after all unidirectional relationship lists are got, unidirectional relationship in opposite direction can be obtained from unidirectional relationship list, bidirectional relationship is merged into by unidirectional relationship in opposite direction, and delete the unidirectional relationship repeated in all unidirectional relationship lists, according to the bidirectional relationship that remaining unidirectional relationship and merging are obtained, the digraph of all target data relationships between nodes in generation distributed processing system(DPS), realize by the relation between visual digraph display data node, realize to data source and the automatic tracing of whereabouts, relation between so user passes through the back end that digraph is shown can just be visually known data source and whereabouts.
Description
Technical field
The invention belongs to mass data processing technical field, in particular, more particularly to a kind of data flow visualization side
Method and device.
Background technology
In the big data epoch of Information technology technology high-speed development, the big data platform construction of industry-by-industry generally require from
Data are obtained in original each system, to be effectively utilized these data, it is necessary to carry out multistep processing to data,
For example：Data cleansing, setup time slide fastener and carry out all kinds of dimensionality analysis etc..The processing of each step can all produce new number
According to for example：A tables generate C by generating A1 tables, B1 tables after data cleansing respectively with B tables after A1 tables and the joint processing of B1 tables
Table, C tables are the need for meeting different bores and have derived D tables and E tables so that industry-by-industry have accumulated mass data.
When these mass datas are processed, a problem can be usually faced：How to clear by mass data constitute it is crisscross
The data flow of complicated cobweb formula, to clear data source and whereabouts from the tracking of data flow, so as to follow the trail of problematic
The change of data and discovery data.Such as, how from back end track up or down all of father node and
Child node, therefore mass data is analyzed excavates and find that the value of data has turned into trend of the times.For this present invention is carried
For a kind of data flow method for visualizing and device, so that by the relation between visual digraph display data node, it is right to realize
The automatic tracing of data source and whereabouts, is easy to user intuitively to check data source and whereabouts.
The content of the invention
In view of this, it is an object of the invention to provide a kind of data flow method for visualizing and device, with by visualization
Digraph display data node between relation, realize to data source and the automatic tracing of whereabouts, be easy to user intuitively to check
Data source and whereabouts.Technical scheme is as follows：
The present invention provides a kind of data flow method for visualizing, is applied in distributed processing system(DPS), the distributed treatment
System collects server and at least one including one is used for the storage server of data storage, and methods described includes：
It is described to collect the unidirectional relationship list that server obtains the transmission of each storage server, wherein the storage server
The unidirectional relationship between all target data nodes of itself storage is obtained, the unidirectional relationship list of itself is obtained；
The server that collects is obtained while having the mesh of first direction and second direction from all unidirectional relationship lists
Two unidirectional relationships between mark back end, the target data node that will simultaneously have first direction and second direction merge in pairs
To relation, wherein first direction and second direction is both direction in opposite direction, and the first direction and the second party
To being direction that unidirectional relationship between the target data node is indicated；
The server that collects deletes the unidirectional relationship repeated in all unidirectional relationship lists；
It is described to collect server according to remaining unidirectional relationship and merge the bidirectional relationship for obtaining, generate at the distribution
The digraph of all target data relationships between nodes in reason system.
Preferably, it is described collect server obtain itself storage all target data nodes between unidirectional relationship, obtain
The unidirectional relationship list of itself.
Preferably, the server that collects obtains lookup information, and is obtained from the digraph according to the lookup information
Get the target data nodal information for meeting the lookup information；
According to the target data nodal information, the node relationships figure of generation and lookup information matches.
Preferably, it is described to collect server and between the storage server obtains all target data nodes of itself storage
Unidirectional relationship, obtain the unidirectional relationship list of itself, including：
According to the node rule and relation rule that pre-set, back end list and data relationships between nodes row are obtained
Table；
The all target datas being present between the back end in relation list are obtained from the back end list
Node；
According to the relation between each back end that relation list between the back end is indicated, all target datas are determined
Unidirectional relationship between node, to obtain the unidirectional relationship list.
Preferably, the basis pre-sets node rule and relation rule, obtain back end list and data section
Relation list between point, including：
According to node regular expression and relation regular expression, the back end list and the back end are obtained
Between relation list, wherein the node regular expression is used to indicate the node rule that pre-sets, the relation regular expressions
Formula is used for the relation rule for indicating to pre-set.
The present invention also provides a kind of data flow visualization device, is applied in distributed processing system(DPS), at the distribution
Reason system collects server and at least one including one is used for the storage server of data storage, and described device is located at the remittance
In director server, described device includes：
Acquiring unit, the unidirectional relationship list for obtaining the transmission of each storage server, wherein the storage server
The unidirectional relationship between all target data nodes of itself storage is obtained, the unidirectional relationship list of itself is obtained；
Combining unit, for obtaining the target with first direction and second direction from all unidirectional relationship lists
Two unidirectional relationships between back end, the target data node that will simultaneously have first direction and second direction are merged into two-way
Relation, wherein first direction and second direction are both direction in opposite direction, and the first direction and the second direction
It is the direction of the unidirectional relationship instruction between the target data node；
Unit is deleted, for deleting the unidirectional relationship repeated in all unidirectional relationship lists；
Generation unit, for the bidirectional relationship obtained according to remaining unidirectional relationship and merging, generates at the distribution
The digraph of all target data relationships between nodes in reason system.
Preferably, the acquiring unit, between being additionally operable to obtain all target data nodes for collecting server itself storage
Unidirectional relationship, obtain the unidirectional relationship list of itself.
Preferably, described device also includes：Information acquisition unit, for obtaining lookup information, and searches letter according to described
Breath gets the target data nodal information for meeting the lookup information from the digraph；
The generation unit, for according to the target data nodal information, generation to be closed with the node for searching information matches
System's figure.
Preferably, the acquiring unit, including：
First obtains subelement, for according to the node rule and relation rule for pre-setting, obtaining back end list
With data relationships between nodes list；
Second obtains subelement, relation row between being present in the back end for the acquisition from the back end list
All target data nodes in table；
Determination subelement, for according between the back end relation list indicate each back end between relation,
The unidirectional relationship between all target data nodes is determined, to obtain the unidirectional relationship list.
Preferably, described first subelement is obtained, for according to node regular expression and relation regular expression, obtaining
Relation list between the back end list and the back end, wherein the node regular expression is used to indicate to set in advance
The node rule put, the relation regular expression is used for the relation rule for indicating to pre-set.
Compared with prior art, the above-mentioned technical proposal that the present invention is provided has the following advantages that：
By above-mentioned technical proposal, after all unidirectional relationship lists are got, can be obtained from unidirectional relationship list
Unidirectional relationship in opposite direction, is merged into bidirectional relationship, and delete all unidirectional relationship row by unidirectional relationship in opposite direction
The unidirectional relationship repeated in table, according to the bidirectional relationship that remaining unidirectional relationship and merging are obtained, generates distributed processing system(DPS)
In all target data relationships between nodes digraph, realize by the relation between visual digraph display data node,
Realize that the relation that such user passes through between the back end that digraph is shown can to data source and the automatic tracing of whereabouts
To be visually known data source and whereabouts.
Brief description of the drawings
In order to illustrate more clearly about the embodiment of the present invention or technical scheme of the prior art, below will be to embodiment or existing
The accompanying drawing to be used needed for having technology description is briefly described, it should be apparent that, drawings in the following description are the present invention
Some embodiments, for those of ordinary skill in the art, on the premise of not paying creative work, can also basis
These accompanying drawings obtain other accompanying drawings.
Fig. 1 is the Organization Chart of distributed processing system(DPS) provided in an embodiment of the present invention；
Fig. 2 is the signaling diagram of data flow method for visualizing provided in an embodiment of the present invention；
Fig. 3 is the schematic diagram of digraph provided in an embodiment of the present invention；
Fig. 4 is the schematic diagram with node relationships figure that is searching information matches provided in an embodiment of the present invention；
Fig. 5 is the structural representation of data flow visualization device provided in an embodiment of the present invention.
Specific embodiment
To make the purpose, technical scheme and advantage of the embodiment of the present invention clearer, below in conjunction with the embodiment of the present invention
In accompanying drawing, the technical scheme in the embodiment of the present invention is clearly and completely described, it is clear that described embodiment is
A part of embodiment of the present invention, rather than whole embodiments.Based on the embodiment in the present invention, those of ordinary skill in the art
The every other embodiment obtained under the premise of creative work is not made, belongs to the scope of protection of the invention.
Fig. 1 provides the Organization Chart of the distributed processing system(DPS) of application data stream method for visualizing for the embodiment of the present invention, ginseng
According to Fig. 1, the distributed processing system(DPS) includes：One is collected the storage server 20 of server 10 and at least one (Fig. 1 is deposited with two
Illustrated as a example by storage server).
Wherein each storage server 20, for data storage, specific storage server can be deposited with back end
Storage data, back end can be a tables of data or a type of database, in actual applications, distributed treatment system
The data that system is used carry out multistep processing, for example：Data cleansing, setup time slide fastener and all kinds of dimensionality analysis etc. are carried out, and
The processing of each step can all produce new data, for example：A tables and B tables by generating A1 tables after data cleansing respectively, B1 tables,
C tables are generated after A1 tables and the joint processing of B1 tables, C tables are the need for meeting different bores and have derived D tables and E tables, these
Table can be respectively seen as a back end, store in one or more storage server 20.
There is certain relation knowable to the example above, between back end, such as data source in B1 tables in B tables, then B1 tables
It is the child node (representing the whereabouts node of data flow) of B tables, B tables are the father nodes (representing the source Nodes of data flow) of B1 tables,
All there is certain relation between certain back end not all in actual applications, be this to follow the trail of distributed processing system(DPS)
The source of middle mass data and whereabouts, each storage server 20 are analyzed to the back end that itself is stored, and are had
The target data node of unidirectional relationship, and will itself storage all target data nodes unidirectional relationship store in the list of itself
To in relation list.That is to have related back end in storage server, each storage takes target data node
Business device can get all target data nodes from all back end of itself storage, and by between all target data nodes
Unidirectional relationship store in respective unidirectional relationship list.
These unidirectional relationship lists are sent in collecting server 10, are arranged by collecting all unidirectional relationships of server 10 pairs
Unidirectional relationship in table is processed, to finally give the oriented of all target data relationships between nodes in distributed processing system(DPS)
Figure.That is, being integrated the scattered unidirectional relationship that storage server 20 is obtained by collecting server 10, obtain everywhere
In distributed processing system(DPS) in all storage servers all target data relationships between nodes digraph, realize by visual
Relation between the digraph display data node of change, realizes that such user is by having to data source and the automatic tracing of whereabouts
Data source and whereabouts can be just visually known to the relation between shown back end is schemed.
Collecting after server 10 obtains digraph, collecting server 10 can include the visitor used in user by digraph
On family end 30, certain user can also be input into lookup information by client 30, related to certain target data node to search
Node relationships figure.
Organization Chart based on system shown in Figure 1, the signaling diagram of data flow method for visualizing provided in an embodiment of the present invention is such as
Shown in Fig. 2, may comprise steps of：
201：Storage server obtains back end list and data according to the node rule and relation rule for pre-setting
Relationships between nodes list.
The node rule for wherein pre-setting is defined to the back end for being used for data storage in storage server, refers to
Show which kind of node can be back end, and the relation rule for pre-setting is by each number in each storage server
The data stored according to node are analyzed, and the data of storage have relation between instructions which back end for obtaining, and these have
The back end of relation can be back end in same storage server, or positioned at different storage servers
Back end, specifically depending on the data stored in each storage server.
In embodiments of the present invention, in order to rapidly and effectively obtain back end list and data node relationships list,
The above-mentioned node rule for pre-setting and relation rule can be shown in regular expression mode, pre-set as described above
Node rule can be represented using node regular expression, and node regular expression is sent to each storage server, by
Whether the back end that storage server matches itself data storage according to node regular expression meets node regular expressions
Definition of the formula to back end, if it is, node regular expression will be met being deposited to the back end of the definition of back end
In storage to back end list.
For the acquisition of back end relation list, it is also possible to which the above-mentioned relation rule for pre-setting is used into relation
Regular expression is represented, and relation regular expression is sent to each storage server, by storage server according to relation just
Then whether expression formula meets relation regular expression to back end come the relation between the back end for matching itself data storage
Between relation definition, if it is, relation regular expression will be met between the back end of the definition of data relationships between nodes
Relation is stored into relation list between back end.
For example：Node regular expression is：It is node regular expression in regex_node='Def (b.* b) ' bracket
The back end of definition, such as defines the nodename of back end；Relation regular expression be regex_relate1=(
BFrom b) and/or between regex_relate2=(bTo b), wherein from represent back end relation is from relations, i.e.,
The data of certain back end get from which back end, to represent back end between relation for relations, i.e. data are
From which back end to which back end, the two relationships between nodes are unidirectional relationships, and so-called unidirectional relationship refers to one
There is vertical linkage to another target data node in individual target data node.
Herein it should be noted is that：Above-mentioned node regular expression and relation regular expression can be with data knots
Structure mode is stored in the internal memory of each storage server, such as can be using in interception regular expression bracket by the way of content pair
Above-mentioned node regular expression and relation regular expression are simplified, and the content for obtaining is with following but be not limited to following data knot
Structure mode is stored：
＜ nodename, from, to ＞, wherein nodename are the back end of node regular expression definition, and
From and to are relations between the back end that above-mentioned relation regular expression is defined, and EMS memory occupation can be reduced in this way
Rate, and after node regular expression or the change of relation regular expression, the content stored in change data structure.
The original document of the following content that is stored with such as each storage server：
In the matching result obtained by above-mentioned node regular expression and relation regular expression：
Storage server 1：
The back end at least included in the back end list for obtaining is：B、B1、A1、A、C……；
Relation is between the back end that relation list includes between the back end for obtaining：
B to B1
A1 from A
B1 to C；
Storage server 2：
The back end at least included in the back end list for obtaining is：A1、D、F、C……；
Relation is between the back end that relation list includes between the back end for obtaining：
A1 to C
D from F
D from C；
Storage server 3：
The back end at least included in the back end list for obtaining is：E、C……；
Relation is between the back end that relation list includes between the back end for obtaining：
E from C。
202：Storage server obtains all mesh being present between back end in relation list from back end list
Mark back end.Because the back end stored in storage server is numerous, and these numerous back end are not
Each back end is present between above-mentioned back end in relation list, thus after back end list is obtained, it is necessary to from
Node between acquisition is present in back end in back end list in relation list, that is, get the related data section of tool
Point, these related back end of tool are considered as target data node.
203：Storage server determines institute according to the relation between each back end of relation list instruction between back end
There is the unidirectional relationship between target data node, to obtain unidirectional relationship list.
That is, in embodiments of the present invention, stored in unidirectional relationship list be all target data nodes between
Unidirectional relationship between unidirectional relationship, target data node is obtained according to relation list between back end, is referred to by unidirectional relationship
The direction for showing can know source and the whereabouts of data flow, such as relation list between back end in a relation be：
A1to C, illustrate that data flow is that then the two target data nodes is unidirectional from target data node A1 to target data node C
The direction that relation is indicated is to point to target data node C from target data node A1, then the source of data flow is target data section
Point A1, the whereabouts of data flow is target data node C.
204：The unidirectional relationship list that each storage server will each get is sent into collecting server.
205：Collect server to be obtained while having the mesh of first direction and second direction from all unidirectional relationship lists
Two unidirectional relationships between mark back end, the target data node that will simultaneously have first direction and second direction merge in pairs
To relation.Wherein first direction and second direction is both direction in opposite direction, and the first direction and the second party
To being direction that unidirectional relationship between the target data node is indicated.
Such as：There are two target datas node A and B in unidirectional relationship list, and recorded in unidirectional relationship list and have
Two unidirectional relationships of the two target data nodes A and B, be respectively：A → B and A ← B, then " → " first party can be considered as
To " ← " can be considered as second direction, have two unidirectional relationships, and the mesh in opposite direction that unidirectional relationship is indicated for this
, it is necessary to two unidirectional relationships between the two target data nodes are merged into bidirectional relationship for mark back end, such asWherein there is the relation of equity in bidirectional relationship between indicating target data node.
206：Collect server and delete the unidirectional relationship repeated in all unidirectional relationship lists.Still with above-mentioned two number of targets
According to node A and B, if record has two unidirectional relationships of the two target data nodes A and B in unidirectional relationship list, respectively
It is：A → B and A → B, i.e., record has the identical of two same target back end unidirectional in all unidirectional relationship lists
Relation, for this target data node, only retains a unidirectional relationship in unidirectional relationship list, therefore for
, it is necessary to the unidirectional relationship repeated in deleting unidirectional relationship list, arranges with all unidirectional relationships for this target data node
Retain a unidirectional relationship in table in a plurality of unidirectional relationship of identical.
207：Collect server according to remaining unidirectional relationship and merge the bidirectional relationship for obtaining, generation distributed treatment system
The digraph of all target data relationships between nodes in system, i.e., show distributed treatment system by a visual digraph
The source of all target data internodal data streams and whereabouts in system, by taking the list that above three storage server is obtained as an example,
In the digraph obtained after collecting server process as shown in figure 3, from digraph shown in Fig. 3 can be visually known to
Data source and whereabouts.
208：Collect server and obtain lookup information, and got from digraph according to lookup information and meet lookup information
Target data nodal information.In embodiments of the present invention, collecting server can externally provide Retrieval Interface, and such user is led to
Crossing the Retrieval Interface can be input into various lookup information, be intercepted out and the node relationships for searching information matches with from digraph
Figure.
Wherein searching information at least includes：The keyword (such as title of target data node) and relationship depth of node, its
Middle relationship depth is 1, represents that searching the target data node indicated with the keyword of input has other targets of direct relation
Back end, relationship depth is 2, to represent and search the target data node indicated with the keyword of input by an intermediate node
Other target data nodes with indirect relation, the like, relationship depth is N, represents and is searched by N-1 intermediate node
The target data node indicated with the keyword of input has other target data nodes of indirect relation.
When related other target data nodes of tool are searched, can be looked into according to relationship depth by recursive fashion
Look for, such as can according to the target data node of relationship depth recursive lookup upstream, the target data node in recursive lookup downstream and
The two-way target data node of recursive lookup, the target data node of its middle and upper reaches can represent the source of data flow, and downstream
Target data node can represent the whereabouts of data flow.
209：Collect server according to target data nodal information, generation and the node relationships figure for searching information matches.Its
Middle target data nodal information at least includes：Relation and number of targets between title, the target data node of target data node
According to the relation between the target data node that node and the keyword of input are indicated, graph model can be then carried out by these and is rendered,
With the node relationships figure for obtaining with search information matches.
The keyword of such as node is the target data node C in digraph shown in above-mentioned Fig. 3, and relationship depth is 1, then converge
The target data nodal information that director server gets from digraph includes：Target data node A1, B1, E and D, number of targets
According to the relation of the relation between node and target data node and C as shown in figure 3, then collecting server according to these target datas
Nodal information, the node relationships figure of generation is as shown in Figure 4.
By above-mentioned technical proposal, after all unidirectional relationship lists are got, can be obtained from unidirectional relationship list
Unidirectional relationship in opposite direction, is merged into bidirectional relationship, and delete all unidirectional relationship row by unidirectional relationship in opposite direction
The unidirectional relationship repeated in table, according to the bidirectional relationship that remaining unidirectional relationship and merging are obtained, generates distributed processing system(DPS)
In all target data relationships between nodes digraph, realize by the relation between visual digraph display data node,
Realize that the relation that such user passes through between the back end that digraph is shown can to data source and the automatic tracing of whereabouts
To be visually known data source and whereabouts.
And in embodiments of the present invention, can also be got from digraph and meet lookup information according to the information of lookup
Target data nodal information, and according to target data nodal information generate with search information matches node relationships figure, realize
The node relationships figure that user wants is obtained from digraph, the interception on demand to digraph is realized, is easy to originate partial data
With the analysis and tracking of whereabouts.
Additionally, in embodiments of the present invention, the above-mentioned server that collects can be used for data storage, that is, collecting server is
A special storage server in distributed processing system(DPS), on the basis of data storage this function, also with collecting
Function, so collects all target data sections that server can obtain itself storage according to above-mentioned steps 201 to step 203
Unidirectional relationship between point, obtains the unidirectional relationship list of itself, is then receiving the unidirectional pass of other storage servers transmission
After series of tables, it is possible to unidirectional relationship list is processed according to step 204 to step 209.
Corresponding with above method embodiment, the embodiment of the present invention also provides a kind of data flow visualization device, the number
According to stream visualization device in server is collected, its structural representation is as shown in figure 5, can include：Acquiring unit 11, merging
Unit 12, deletion unit 13 and generation unit 14.
Acquiring unit 11, the unidirectional relationship list for obtaining the transmission of each storage server, wherein storage server is obtained
The unidirectional relationship between all target data nodes of body storage is taken from, the unidirectional relationship list of itself is obtained.
That is, in embodiments of the present invention, stored in unidirectional relationship list be all target data nodes between
Unidirectional relationship between unidirectional relationship, target data node is obtained according to relation list between back end, is referred to by unidirectional relationship
The direction for showing can know source and the whereabouts of data flow, such as relation list between back end in a relation be：
A1to C, illustrate that data flow is that then the two target data nodes is unidirectional from target data node A1 to target data node C
The direction that relation is indicated is to point to target data node C from target data node A1, then the source of data flow is target data section
Point A1, the whereabouts of data flow is target data node C.
Certainly it is unidirectional between acquiring unit 11 can also obtain all target data nodes for collecting server itself storage
Relation, obtains the unidirectional relationship list of itself, with to collecting the unidirectional relationship list that server and other storage servers are obtained
Processed.
Wherein acquiring unit 11 can include：First obtains subelement, the second acquisition subelement and determination subelement.
First obtains subelement, for according to the node rule and relation rule for pre-setting, obtaining back end list
With data relationships between nodes list, in embodiments of the present invention, the node rule and relation rule for pre-setting are with regular expressions
Formula mode shows that such as first obtains subelement is used to, according to node regular expression and relation regular expression, obtain data
Node listing and data relationships between nodes list, its interior joint regular expression are used for the node rule for indicating to pre-set, and close
It is that regular expression is used for the relation rule that instruction pre-sets, the related description in embodiment of the method is specifically referred to, to this
The embodiment of the present invention is no longer illustrated.
Second obtains subelement, the institute between being present in back end for the acquisition from back end list in relation list
There is target data node.Because the back end stored in storage server is numerous, and these numerous back end are simultaneously
It is not that each back end is present between above-mentioned back end in relation list, so after back end list is obtained, needing
To be obtained from back end list and be present in node between back end in relation list, that is, get the related data of tool
Node, these related back end of tool are considered as target data node.
Determination subelement, for according between back end relation list indicate each back end between relation, it is determined that
Unidirectional relationship between all target data nodes, to obtain unidirectional relationship list.
Combining unit 12, for obtaining the mesh with first direction and second direction from all unidirectional relationship lists
Two unidirectional relationships between mark back end, the target data node that will simultaneously have first direction and second direction merge in pairs
To relation, wherein first direction and second direction are both direction in opposite direction, and first direction and second direction are targets
The direction that unidirectional relationship between back end is indicated.
Such as：There are two target datas node A and B in unidirectional relationship list, and recorded in unidirectional relationship list and have
Two unidirectional relationships of the two target data nodes A and B, be respectively：A → B and A ← B, then " → " first party can be considered as
To " ← " can be considered as second direction, have two unidirectional relationships, and the mesh in opposite direction that unidirectional relationship is indicated for this
, it is necessary to two unidirectional relationships between the two target data nodes are merged into bidirectional relationship for mark back end, such asWherein there is the relation of equity in bidirectional relationship between indicating target data node.
Unit 13 is deleted, for deleting the unidirectional relationship repeated in all unidirectional relationship lists.Still with above-mentioned two target
Back end A and B, if record has two unidirectional relationships of the two target data nodes A and B in unidirectional relationship list, point
It is not：A → B and A → B, i.e., record has two identical lists of same target back end in all unidirectional relationship lists
To relation, for this target data node, a unidirectional relationship is only retained in unidirectional relationship list, thus it is right
For this target data node, it is necessary in deleting unidirectional relationship list repeat unidirectional relationship, with all unidirectional relationships
Retain a unidirectional relationship in list in a plurality of unidirectional relationship of identical.
Generation unit 14, for the bidirectional relationship obtained according to remaining unidirectional relationship and merging, generates distributed treatment
The digraph of all target data relationships between nodes in system, i.e., show distributed treatment by a visual digraph
The source of all target data internodal data streams and whereabouts in system.
By above-mentioned technical proposal, after all unidirectional relationship lists are got, can be obtained from unidirectional relationship list
Unidirectional relationship in opposite direction, is merged into bidirectional relationship, and delete all unidirectional relationship row by unidirectional relationship in opposite direction
The unidirectional relationship repeated in table, according to the bidirectional relationship that remaining unidirectional relationship and merging are obtained, generates distributed processing system(DPS)
In all target data relationships between nodes digraph, realize by the relation between visual digraph display data node,
Realize that the relation that such user passes through between the back end that digraph is shown can to data source and the automatic tracing of whereabouts
To be visually known data source and whereabouts.
Additionally, data flow visualization device provided in an embodiment of the present invention can also include：Information acquisition unit, for obtaining
Lookup information is taken, and gets the target data nodal information for meeting lookup information from digraph according to lookup information.At this
In inventive embodiments, collecting server can externally provide Retrieval Interface, and such user can be input into respectively by the Retrieval Interface
Kind of lookup information, to intercept out from digraph and search the node relationships figure of information matches.
Wherein searching information at least includes：The keyword (such as title of target data node) and relationship depth of node, its
Middle relationship depth is 1, represents that searching the target data node indicated with the keyword of input has other targets of direct relation
Back end, relationship depth is 2, to represent and search the target data node indicated with the keyword of input by an intermediate node
Other target data nodes with indirect relation, the like, relationship depth is N, represents and is searched by N-1 intermediate node
The target data node indicated with the keyword of input has other target data nodes of indirect relation.
When related other target data nodes of tool are searched, can be looked into according to relationship depth by recursive fashion
Look for, such as can according to the target data node of relationship depth recursive lookup upstream, the target data node in recursive lookup downstream and
The two-way target data node of recursive lookup, the target data node of its middle and upper reaches can represent the source of data flow, and downstream
Target data node can represent the whereabouts of data flow.
Generation unit 14, for according to target data nodal information, generation and the node relationships figure for searching information matches.Its
Middle target data nodal information at least includes：Relation and number of targets between title, the target data node of target data node
According to the relation between the target data node that node and the keyword of input are indicated, graph model can be then carried out by these and is rendered,
With the node relationships figure for obtaining with search information matches.
By above-mentioned technical proposal, realize obtaining the node relationships figure that user wants from digraph, realize to digraph
Interception on demand, be easy to partial data source and whereabouts analysis and tracking.
It should be noted that each embodiment in this specification is described by the way of progressive, each embodiment weight
Point explanation is all difference with other embodiment, between each embodiment identical similar part mutually referring to.
For device class embodiment, because it is substantially similar to embodiment of the method, so description is fairly simple, related part ginseng
See the part explanation of embodiment of the method.
Finally, in addition it is also necessary to explanation, herein, such as first and second or the like relational terms be used merely to by
One entity or operation make a distinction with another entity or operation, and not necessarily require or imply these entities or operation
Between there is any this actual relation or order.And, term " including ", "comprising" or its any other variant meaning
Covering including for nonexcludability, so that process, method, article or equipment including a series of key elements not only include that
A little key elements, but also other key elements including being not expressly set out, or also include for this process, method, article or
The intrinsic key element of equipment.In the absence of more restrictions, the key element limited by sentence "including a ...", does not arrange
Except also there is other identical element in the process including the key element, method, article or equipment.
The foregoing description of the disclosed embodiments, enables those skilled in the art to realize or uses the present invention.To this
Various modifications of a little embodiments will be apparent for a person skilled in the art, and generic principles defined herein can
Without departing from the spirit or scope of the present invention, to realize in other embodiments.Therefore, the present invention will not be limited
It is formed on the embodiments shown herein, and is to fit to consistent with principles disclosed herein and features of novelty most wide
Scope.
The above is only the preferred embodiment of the present invention, it is noted that for the ordinary skill people of the art
For member, under the premise without departing from the principles of the invention, some improvements and modifications can also be made, these improvements and modifications also should
It is considered as protection scope of the present invention.
Claims (10)
1. a kind of data flow method for visualizing, it is characterised in that be applied in distributed processing system(DPS), the distributed treatment system
System include one collect server and at least one for data storage storage server, methods described includes：
It is described to collect the unidirectional relationship list that server obtains the transmission of each storage server, wherein the storage server is obtained
Unidirectional relationship between all target data nodes of itself storage, obtains the unidirectional relationship list of itself；
The server that collects is obtained while having the number of targets of first direction and second direction from all unidirectional relationship lists
According to node, two unidirectional relationships between the target data node that will simultaneously have first direction and second direction are merged into two-way pass
System, wherein first direction and second direction are both direction in opposite direction, and the first direction and the second direction are
The direction that unidirectional relationship between the target data node is indicated；
The server that collects deletes the unidirectional relationship repeated in all unidirectional relationship lists；
It is described to collect server according to remaining unidirectional relationship and merge the bidirectional relationship for obtaining, generate the distributed treatment system
The digraph of all target data relationships between nodes in system.
2. method according to claim 1, it is characterised in that described to collect all targets that server obtains itself storage
Unidirectional relationship between back end, obtains the unidirectional relationship list of itself.
3. method according to claim 1 and 2, it is characterised in that the server that collects obtains lookup information, and according to
The lookup information gets the target data nodal information for meeting the lookup information from the digraph；
According to the target data nodal information, the node relationships figure of generation and lookup information matches.
4. method according to claim 2, it is characterised in that described to collect server and the storage server is obtained from
Unidirectional relationship between all target data nodes of body storage, obtains the unidirectional relationship list of itself, including：
According to the node rule and relation rule that pre-set, back end list and data relationships between nodes list are obtained；
The all target data nodes being present between the back end in relation list are obtained from the back end list；
According to the relation between each back end that relation list between the back end is indicated, all target data nodes are determined
Between unidirectional relationship, to obtain the unidirectional relationship list.
5. method according to claim 4, it is characterised in that node rule and relation rule that the basis pre-sets
Then, back end list and data relationships between nodes list are obtained, including：
According to node regular expression and relation regular expression, obtain and closed between the back end list and the back end
Series of tables, wherein the node regular expression is used for the node rule for indicating to pre-set, the relation regular expression is used
In the relation rule that instruction pre-sets.
6. a kind of data flow visualization device, it is characterised in that be applied in distributed processing system(DPS), the distributed treatment system
System include one collect server and at least one be used for data storage storage server, described device be located at described in collect clothes
In business device, described device includes：
Acquiring unit, the unidirectional relationship list for obtaining the transmission of each storage server, wherein the storage server is obtained
Unidirectional relationship between all target data nodes of itself storage, obtains the unidirectional relationship list of itself；
Combining unit, for obtaining the target data with first direction and second direction from all unidirectional relationship lists
Two unidirectional relationships between node, the target data node that will simultaneously have first direction and second direction are merged into two-way pass
System, wherein first direction and second direction are both direction in opposite direction, and the first direction and the second direction are
The direction that unidirectional relationship between the target data node is indicated；
Unit is deleted, for deleting the unidirectional relationship repeated in all unidirectional relationship lists；
Generation unit, for the bidirectional relationship obtained according to remaining unidirectional relationship and merging, generates the distributed treatment system
The digraph of all target data relationships between nodes in system.
7. device according to claim 6, it is characterised in that the acquiring unit, be additionally operable to acquisition collect server from
Unidirectional relationship between all target data nodes of body storage, obtains the unidirectional relationship list of itself.
8. the device according to claim 6 or 7, it is characterised in that described device also includes：Information acquisition unit, is used for
Lookup information is obtained, and gets the target data for meeting the lookup information from the digraph according to the lookup information
Nodal information；
The generation unit, for according to the target data nodal information, generation and the node relationships figure for searching information matches.
9. device according to claim 7, it is characterised in that the acquiring unit, including：
First obtains subelement, for according to the node rule and relation rule for pre-setting, obtaining back end list sum
According to relationships between nodes list；
Second obtains subelement, is present between the back end in relation list for being obtained from the back end list
All target data nodes；
Determination subelement, for according between the back end relation list indicate each back end between relation, it is determined that
Unidirectional relationship between all target data nodes, to obtain the unidirectional relationship list.
10. device according to claim 9, it is characterised in that described first obtains subelement, for according to node canonical
Expression formula and relation regular expression, obtain relation list between the back end list and the back end, wherein described
Node regular expression is used for the node rule for indicating to pre-set, and the relation regular expression is used to indicate what is pre-set
Relation rule.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710120952.0A CN106897446A (en) | 2017-03-02 | 2017-03-02 | A kind of data flow method for visualizing and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710120952.0A CN106897446A (en) | 2017-03-02 | 2017-03-02 | A kind of data flow method for visualizing and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106897446A true CN106897446A (en) | 2017-06-27 |
Family
ID=59184922
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710120952.0A Pending CN106897446A (en) | 2017-03-02 | 2017-03-02 | A kind of data flow method for visualizing and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106897446A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021184691A1 (en) * | 2020-03-19 | 2021-09-23 | 北京三快在线科技有限公司 | Data presentation method and apparatus, electronic device and computer-readable medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102158531A (en) * | 2010-02-01 | 2011-08-17 | 北京大学 | Distributed transmission method for query data stream |
| US20150026217A1 (en) * | 2013-07-16 | 2015-01-22 | Fujitsu Limited | Data output method, computer-readable recording medium storing data output program and data output system |
| CN104462084A (en) * | 2013-09-13 | 2015-03-25 | Sap欧洲公司 | Search refinement advice based on multiple queries |
| CN104504082A (en) * | 2014-12-24 | 2015-04-08 | 北京德塔普博软件有限公司 | Path showing method and system for target knowledge node sets of multiple knowledge networks |
| CN106161106A (en) * | 2016-08-24 | 2016-11-23 | 北京奇艺世纪科技有限公司 | A kind of node aggregation method and device being applied to oriented topological diagram |
-
2017
- 2017-03-02 CN CN201710120952.0A patent/CN106897446A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102158531A (en) * | 2010-02-01 | 2011-08-17 | 北京大学 | Distributed transmission method for query data stream |
| US20150026217A1 (en) * | 2013-07-16 | 2015-01-22 | Fujitsu Limited | Data output method, computer-readable recording medium storing data output program and data output system |
| CN104462084A (en) * | 2013-09-13 | 2015-03-25 | Sap欧洲公司 | Search refinement advice based on multiple queries |
| CN104504082A (en) * | 2014-12-24 | 2015-04-08 | 北京德塔普博软件有限公司 | Path showing method and system for target knowledge node sets of multiple knowledge networks |
| CN106161106A (en) * | 2016-08-24 | 2016-11-23 | 北京奇艺世纪科技有限公司 | A kind of node aggregation method and device being applied to oriented topological diagram |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021184691A1 (en) * | 2020-03-19 | 2021-09-23 | 北京三快在线科技有限公司 | Data presentation method and apparatus, electronic device and computer-readable medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Chen | Structuring and visualising the WWW by generalised similarity analysis | |
| WO2009030247A1 (en) | Detecting correlations between data representing information | |
| CN106294358A (en) | The search method of a kind of information and system | |
| Kang et al. | Trendspedia: An internet observatory for analyzing and visualizing the evolving web | |
| CN106897446A (en) | A kind of data flow method for visualizing and device | |
| Ohmukai et al. | Metadata-driven personal knowledge publishing | |
| Mishra et al. | An effective algorithm for web mining based on topic sensitive link analysis | |
| Alp et al. | Influential user detection on Twitter: Analyzing effect of focus rate | |
| WO2009030248A1 (en) | Detecting correlations between data representing information | |
| Muzumdar et al. | Neural Network Approach for Web Usage Mining | |
| Kaur et al. | Metamorphosis of data (small to big) and the comparative study of techniques (HADOOP, HIVE and PIG) to handle big data | |
| Bodra | Processing queries over partitioned graph databases: An approach and it’s evaluation | |
| Cherichi et al. | New metric measure for the improvement of search results in microblogs | |
| Anwar et al. | Modeling a web forum ecosystem into an enriched social graph | |
| Alahmari et al. | Linked Data and Entity Search: A Brief History and Some Ways Ahead. | |
| du Toit | Network visualisation as a citator user interface | |
| JP2008541296A (en) | Personalizable information network | |
| Bama et al. | Improved pagerank algorithm for web structure mining | |
| CN108874819A (en) | A kind of data digging method of database | |
| Wang et al. | Notice of Retraction: Mindmap-NG: A novel framework for modeling effective thinking | |
| Manivel et al. | Indication on Web Mining and Different Performance used for Web Personalization | |
| Chandsarkar et al. | Information retrieval system: For skill set improvement in software projects | |
| Olivieri et al. | What is a meme, technically speaking | |
| Lee et al. | A recommender system for library based on hadoop ecosystem | |
| Li et al. | Research and Implementation of a Skyline Query Method for Hidden Web Database |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20170627 |