CN108984592B - Data processing system, method and device based on graphic database - Google Patents
Data processing system, method and device based on graphic database Download PDFInfo
- Publication number
- CN108984592B CN108984592B CN201810548344.4A CN201810548344A CN108984592B CN 108984592 B CN108984592 B CN 108984592B CN 201810548344 A CN201810548344 A CN 201810548344A CN 108984592 B CN108984592 B CN 108984592B
- Authority
- CN
- China
- Prior art keywords
- data
- graph structure
- structure data
- graph
- processed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Abstract
The embodiment of the invention provides a data processing system, a method and a device based on a graphic database, wherein the system comprises: the system comprises a central control subsystem, a graphic database subsystem, a graphic data calculation subsystem and a graphic structure data storage subsystem; the system provided by the embodiment of the invention can directly acquire the processing result stored in the graph structure data storage subsystem under the condition that the to-be-processed relational data is not determined to be processed for the first time; and under the condition of processing the to-be-processed relational data for the first time, graph structuring processing is carried out on the to-be-processed relational data through the graph database subsystem to obtain first graph structure data, second graph structure data which is in incidence relation with the first graph structure data is obtained through the graph data computing subsystem, the first graph structure data and the second graph structure data are combined to obtain third graph structure data, and the third graph structure data is used as a processing result.
Description
Technical Field
The invention relates to the technical field of computer application, in particular to a data processing system, a method and a device based on a graphic database.
Background
With the advent of the big data era, the graphic database is gradually and widely applied to processing relational data with an association relationship due to the characteristics of rapid relational query, stable single-machine environment and the like. For example, the data of the staff member of a movie includes: multiple actors, directors, special effects personnel, etc., while a director itself may also be an actor of another movie or television show; a particular actor may also be the director of other movies, and it can be seen that the cast data for that movie has a complex association and, therefore, may be referred to as relational data.
As shown in fig. 1, a system for processing relational data by using a graph database is a common system in the prior art, and the system includes: the application subsystem and the graphic database, the concrete processing process comprises: after receiving the relational data to be processed, the application subsystem sends the received relational data to a graphic database; the graph database carries out graph structural processing on the received relational data to obtain graph structural data of the relational data, and the obtained graph structural data is used as a processing result and fed back to the application subsystem. Although the system can process the relational data to obtain the graph structure data, the processing speed is slow when the association relation of the relational data to be processed is too complex due to the limited data processing capacity of the graph database.
Disclosure of Invention
Embodiments of the present invention provide a data processing system, method and device based on a graph database, so as to improve the processing speed of processing relational data. The specific technical scheme is as follows:
one aspect of the present invention provides a data processing system based on a graphic database, the system comprising: the system comprises a central control subsystem, a graphic database subsystem, a graphic data calculation subsystem and a graphic structure data storage subsystem; wherein the content of the first and second substances,
the central control subsystem acquires the relational data to be processed and judges whether the relational data to be processed is processed for the first time, and if so, the relational data to be processed is sent to the graphic database subsystem; if not, requesting a processing result of the to-be-processed relational data from the graph structure data storage subsystem, wherein the processing result comprises: the graph structure data of the relational data to be processed and the graph structure data which has an incidence relation with the graph structure data of the relational data to be processed;
after the graphic database subsystem receives the relational data to be processed, carrying out graph structural processing on the relational data to be processed to obtain first graph structure data, and feeding back the first graph structure data to the central control subsystem; after receiving the first graph structure data, the central control subsystem sends the first graph structure data to the graph data calculation subsystem; after receiving the first graph structure data, the graph data calculation subsystem acquires second graph structure data which is associated with the first graph structure data, combines the first graph structure data and the second graph structure data to obtain third graph structure data, and feeds the third graph structure data back to the central control subsystem; after receiving the third graph structure data, the central control subsystem takes the third graph structure data as a processing result of the to-be-processed relational data and sends the third graph structure data to the graph structure data storage subsystem; the graph structure data storage subsystem receives and stores the third graph structure data;
and after receiving the third graph structure data, the central control subsystem takes the third graph structure data as the processing result.
Optionally, the graphic database subsystem includes: at least one graph database, and one graph database handles relational data for one business type.
Optionally, the step of sending the to-be-processed relational data to the graphic database subsystem includes:
determining the content type of the relational data to be processed;
acquiring a service type corresponding to the content type;
and sending the relational data to be processed to a graphic database for processing the relational data of the acquired service type.
Optionally, the step of performing graph structural processing on the to-be-processed relational data to obtain first graph structural data includes:
determining an object contained in the relational data to be processed;
acquiring the determined association relation among the objects;
and obtaining the first graph structure data according to the determined objects and the obtained association relation.
Optionally, the step of obtaining second graph structure data associated with the first graph structure data, and merging the first graph structure data and the second graph structure data to obtain third graph structure data includes:
acquiring second graph structure data, wherein the second graph structure data and the first graph structure data comprise at least one same object;
and performing deduplication processing on the same object contained in the first graph structure data and the second graph structure data to obtain third graph structure data.
Optionally, the graph structure data storage subsystem is: a storage system for storing graph structure data based on a distributed architecture,
the step of sending the third graph structure data to the graph structure data storage subsystem includes:
sending the acquired service type and the third graph structure data to the graph structure data storage subsystem;
accordingly, the step of the graph structure data storage subsystem receiving and storing the third graph structure data comprises:
determining a storage partition corresponding to the received service type;
storing the third graph structure data to the determined storage partition.
The present invention also provides a data processing method based on a graphic database, which is applied to a central control subsystem in a data processing system based on a graphic database, wherein the data processing system includes: the method comprises a central control subsystem, a graphic database subsystem, a graphic data calculation subsystem and a graph structure data storage subsystem, and comprises the following steps:
obtaining relation type data to be processed, and judging whether the relation type data to be processed is processed for the first time;
if yes, sending the to-be-processed relational data to the graphic database subsystem, and receiving first graph structure data fed back by the graphic database subsystem, wherein the first graph structure data is as follows: carrying out graph structural processing on the relational data to be processed to obtain graph structural data; sending the first graph structure data to the graph data computing subsystem, and receiving third graph structure data fed back by the graph computing subsystem, wherein the third graph structure data is: and graph structure data obtained by merging the first graph structure data and the second graph structure data, wherein the second graph structure data is as follows: the graph structure data which has an incidence relation with the first graph structure data take the third graph structure data as the processing result of the relational data to be processed, and the third graph structure data are sent to the graph structure data storage subsystem;
if not, requesting a processing result of the to-be-processed relational data from the graph structure data storage subsystem, wherein the processing result comprises: the graph structure data of the relational data to be processed and the graph structure data which has an incidence relation with the graph structure data of the relational data to be processed; and receiving the third graph structure data fed back by the graph structure data storage subsystem, and taking the third graph structure data as the processing result.
Optionally, the graphic database subsystem includes: at least one graphic database, and one graphic database processes relational data of one service type, and accordingly, the step of sending the relational data to be processed to the graphic database subsystem includes:
determining the content type of the relational data to be processed;
acquiring a service type corresponding to the content type;
and sending the relational data to be processed to a graphic database for processing the relational data of the acquired service type.
Optionally, the graph structure data storage subsystem is: a storage system for storing graph structure data based on a distributed architecture,
the step of sending the third graph structure data to the storage subsystem includes:
and sending the acquired service type and the third graph structure data to the graph structure data storage subsystem.
The present invention also provides a data processing apparatus based on a graphic database, which is applied to a central control subsystem in a data processing system based on a graphic database, wherein the data processing system includes: the device comprises a central control subsystem, a graphic database subsystem, a graphic data calculation subsystem and a graphic structure data storage subsystem, and the device comprises:
the obtaining module is used for obtaining the relational data to be processed and judging whether the relational data to be processed is processed for the first time, the sending module is triggered under the condition that the judging result is yes, and the requesting module is triggered under the condition that the judging result is no;
a sending module, configured to send the to-be-processed relational data to the graph database subsystem, and receive first graph structure data fed back by the graph database subsystem, where the first graph structure data is: carrying out graph structural processing on the relational data to be processed to obtain graph structural data; sending the first graph structure data to the graph data computing subsystem, and receiving third graph structure data fed back by the graph computing subsystem, wherein the third graph structure data is: and graph structure data obtained by merging the first graph structure data and the second graph structure data, wherein the second graph structure data is as follows: the graph structure data which has an incidence relation with the first graph structure data take the third graph structure data as the processing result of the relational data to be processed, and the third graph structure data are sent to the graph structure data storage subsystem;
a request module, configured to request a processing result of the to-be-processed relational data from the graph structure data storage subsystem, where the processing result includes: the graph structure data of the relational data to be processed and the graph structure data which has an incidence relation with the graph structure data of the relational data to be processed; and receiving the third graph structure data fed back by the graph structure data storage subsystem, and taking the third graph structure data as the processing result.
The embodiment of the invention also provides a central control subsystem, which comprises the following components: a central control subsystem in a graphic database based data processing system, wherein the data processing system comprises: the system comprises a central control subsystem, a graphic database subsystem, a graphic data calculation subsystem and a graphic structure data storage subsystem; the central control subsystem comprises: the system comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete mutual communication through the communication bus;
a memory for storing a computer program;
and the processor is used for realizing any one of the data processing methods based on the graphic database when executing the program stored in the memory.
Yet another aspect of the present invention also provides a computer-readable storage medium, where the computer-readable storage medium is: a storage medium of a central control subsystem in a data processing system based on a graphic database, wherein the data system comprises: the system comprises a central control subsystem, a graphic database subsystem, a graphic data calculation subsystem and a graphic structure data storage subsystem; the computer-readable storage medium has instructions stored therein, which when run on a computer, cause the computer to implement any one of the above-described graphic database-based data processing methods.
The data processing system, the method and the device based on the graph database can directly acquire the processing result stored in the graph structure data storage subsystem under the condition that the relation type data to be processed is not determined to be processed for the first time; and under the condition of processing the relational data to be processed for the first time, the graphic database subsystem carries out graphic structuring processing on the relational data to be processed to obtain first graphic structure data, the graphic data calculation subsystem obtains second graphic structure data which is in incidence relation with the first graphic structure data, the first graphic structure data and the second graphic structure data are merged to obtain third graphic structure data, and the third graphic structure data are used as a processing result of the relational data to be processed.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a schematic diagram of a prior art relational data processing system;
FIG. 2 is a schematic structural diagram of a data processing system based on a graph database according to an embodiment of the present invention;
fig. 3 is a schematic flowchart of a data processing method based on a graph database according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of a data processing apparatus based on a graphic database according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of a central control subsystem according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 2, a schematic structural diagram of a data processing system based on a graph database according to an embodiment of the present invention is shown, where the system includes: a central control subsystem 100, a graphic database subsystem 200, a graphic data calculation subsystem 300 and a graph structure data storage subsystem 400; wherein the content of the first and second substances,
the central control subsystem 100 is used for obtaining the relational data to be processed, judging whether the relational data to be processed is processed for the first time or not, and if so, sending the relational data to be processed to the graphic database subsystem 200; if not, requesting a processing result of the to-be-processed relational data to the graph structure data storage subsystem 400, wherein the processing result comprises: graph structure data of the relational data to be processed and graph structure data having an association relationship with the graph structure data of the relational data to be processed;
after the graphic database subsystem 200 receives the relational data to be processed, carrying out graph structural processing on the relational data to be processed to obtain first graph structural data, and feeding back the first graph structural data to the central control subsystem 100; after receiving the first graph structure data, the central control subsystem 100 sends the first graph structure data to the graphic data calculation subsystem 300; after receiving the first graph structure data, the graph data calculation subsystem 300 acquires graph structure data associated with the first graph structure data, merges the first graph structure data and the second graph structure data to obtain third graph structure data, and feeds the third graph structure data back to the central control subsystem 100; after receiving the third graph structure data, the central control subsystem 100 takes the third graph structure data as a processing result of the to-be-processed relational data, and sends the third graph structure data to the graph structure data storage subsystem 400; the graph structure data storage subsystem 400 receives and stores third graph structure data;
after receiving the request sent by the central control subsystem 100, the graph structure data storage subsystem 400 feeds back the third graph structure data to the central control subsystem 100, and after receiving the third graph structure data, the central control subsystem 100 takes the third graph structure data as a processing result.
In practical applications, in the process of obtaining the to-be-processed relational data, the central control subsystem 100 may repeatedly obtain the same relational data, in order to prevent the same relational data from being repeatedly processed. In an implementation manner of the embodiment of the present invention, the central control subsystem 100 may record, for each piece of relational data that has obtained a processing result, the piece of relational data that has obtained the processing result. Based on this, when the central control subsystem 100 determines that a certain obtained to-be-processed relational data is not the first-processed relational data, it indicates that the processing result of the to-be-processed relational data has already been obtained, and therefore, the processing result of the obtained to-be-processed relational data can be directly requested to the graph structure data storage subsystem 400; and under the condition that the obtained to-be-processed relational data is judged to be firstly processed relational data, the obtained to-be-processed relational data is sent to the graphic database subsystem 200 for processing.
The to-be-processed relational data obtained by the central control subsystem 100 may be relational data sent by a client of the application system.
Accordingly, after the central control subsystem 100 obtains the processing result of the to-be-processed relational data from the graph structure data storage subsystem 400 or the graph data calculation subsystem 300, the processing result may be fed back to the client of the application system requesting to process the to-be-processed relational data.
In order to increase the processing speed of processing the relational data, in one implementation manner of the embodiment of the present invention, the graphic database subsystem 200 includes: at least one graph database, and one graph database handles relational data for one business type.
The service type may be a service type to which relational data to be processed in actual application belongs, and may include: a consumption traffic type, a call log traffic type, etc.
In an implementation manner, which business type relational data each graph database is to process may be preset, and a data processing manner of processing one business type relational data by one graph database can accelerate data processing speed and improve data security.
In one implementation, the graphics database subsystem 200 may be: a system for processing relational data based on a distributed architecture. The relational data are calculated and processed in a distributed data processing mode, so that the data processing speed can be increased, and the data security can be improved.
Accordingly, in one implementation, the step of sending the to-be-processed relational data to the graphic database subsystem 200 includes:
determining the content type of the relational data to be processed;
step two, acquiring a service type corresponding to the content type;
and step three, sending the relational data to be processed to a graphic database for processing the relational data of the acquired service types.
The content type may be determined according to the specific content of the relational data to be processed, and accordingly, the relational data of different content types having similar or same content in the specific content of the relational data may belong to one service type. For example, if the specific content of a certain to-be-processed relational data is a shopping record, the content type of the to-be-processed relational data is a shopping type; the specific content of a certain to-be-processed relational data is a credit card use record, and the content type of the to-be-processed relational data is a credit type; and the record of shopping or the record of credit card use can be used as a measure of the consumption level of one person, so that the shopping type and the credit type can be both of the consumption business type.
In an implementation manner of the embodiment of the present invention, the step of performing graph structure processing on the to-be-processed relational data by the graph database subsystem 200 to obtain first graph structure data may include:
determining an object contained in relational data to be processed;
acquiring the association relation between the determined objects;
and step three, obtaining first graph structure data according to the determined objects and the obtained incidence relation.
Objects may be understood as elements required to form graph structure data, and may include people and/or things. For example, for the call log of zhang san, zhang san and each person included in the call log are one object. For the third shopping record, each object in the third shopping record and the shopping record is an object.
The association relationship can be understood as; for example, in the call record of zhang san, if a call or an answer occurs between zhang san and lie si, the relationship between zhang san and lie si is: a relationship resulting from a call or pickup between zhang san and lie san.
In one implementation, the objects may be connected by an arrow, and an association relationship between two objects is indicated above the arrow, so that the graph structure data is obtained according to the association relationship between the objects and the objects included in the to-be-processed relational data. For example, the resulting graph structure data may be represented as:
in one implementation manner, in order to make the association relationship between two objects more explicit, the association relationship may further include: time, place, etc. of the actual action occurring between the two objects. For example, several minutes may be indicated for an object in the correspondence record to make a call. The location at which the purchase is made can be indicated for the object in the shopping record.
In an implementation manner of the embodiment of the present invention, the step of acquiring, by the graphic data computing subsystem 300, second graph structure data having an association relationship with the first graph structure data, and merging the first graph structure data and the second graph structure data to obtain third graph structure data may include:
step one, acquiring second graph structure data, wherein the second graph structure data and the first graph structure data comprise at least one same object;
and step two, carrying out duplicate removal processing on the same objects contained in the first graph structure data and the second graph structure data to obtain third graph structure data.
In one implementation, the graphics data computation subsystem 300 may be: a system for computing graph structure data based on a distributed architecture. Through a distributed data processing mode, the received graph structure data is calculated, and the data processing speed and the data security can be improved.
The de-duplication treatment canTo be understood as a merging process of the same objects contained in the two graph structure data. For example, the first graph structure data is:
the second graph structure data is:
the same object contained in the first graph structure data and the second graph structure data is subjected to deduplication processing, and the obtained third graph structure data is:
compared with the mode of singly taking a plurality of graph structure data as a processing result, the graph structure data obtained after the duplication removal can ensure that the processing result of the relational data to be processed is more comprehensive, and simultaneously, the incidence relation among objects in the processing result is clearer.
In order to accelerate the read/write speed of the graph structure data storage subsystem 400, in an implementation manner of the embodiment of the present invention, the graph structure data storage subsystem 400 may be: a storage system for storing graph structure data based on a distributed architecture. Specifically, the graph structure data storage subsystem 400 includes different storage partitions, where one storage partition is used to store a processing result of the to-be-processed relational data of one service type.
Based on this, the step of sending the third graph structure data to the graph structure data storage subsystem 400 may include:
sending the acquired service type of the to-be-processed relational data and the third graph structure data to the graph structure data storage subsystem 400;
accordingly, the step of the graph structure data storage subsystem 400 receiving and storing the third graph structure data may include:
determining a storage partition corresponding to the received service type;
and storing the third graph structure data to the determined storage partition.
The embodiment of the invention can improve the query efficiency of the processing result by storing the processing result of the to-be-processed relational data with the same service type into the same storage partition.
In each scheme provided by the embodiment of the invention, the data processing system based on the graph database can directly acquire the processing result stored in the graph structure data storage subsystem under the condition that the relation type data to be processed is not determined to be processed for the first time; and under the condition of processing the relational data to be processed for the first time, the graphic database subsystem carries out graphic structuring processing on the relational data to be processed to obtain first graphic structure data, the graphic data calculation subsystem obtains second graphic structure data which is in incidence relation with the first graphic structure data, the first graphic structure data and the second graphic structure data are merged to obtain third graphic structure data, and the third graphic structure data are used as a processing result of the relational data to be processed.
Referring to fig. 3, a schematic flowchart of a data processing method based on a graph database according to an embodiment of the present invention is shown, and is applied to a central control subsystem in a data processing system based on a graph database, where the data processing system includes: the method comprises the following steps of:
s500, obtaining relational data to be processed, judging whether the relational data to be processed is processed for the first time, if so, executing S600, and if not, executing S700;
s600, sending the relational data to be processed to the graphic database subsystem, and receiving first graph structure data fed back by the graphic database subsystem, wherein the first graph structure data is as follows: carrying out graph structural processing on the relational data to be processed to obtain graph structural data; sending the first graph structure data to the graph data computing subsystem, and receiving third graph structure data fed back by the graph computing subsystem, wherein the third graph structure data is: and graph structure data obtained by merging the first graph structure data and the second graph structure data, wherein the second graph structure data is as follows: the graph structure data which has an incidence relation with the first graph structure data take the third graph structure data as the processing result of the relational data to be processed, and the third graph structure data are sent to the graph structure data storage subsystem;
s700, requesting a processing result of the to-be-processed relational data from the graph structure data storage subsystem, wherein the processing result comprises: the graph structure data of the relational data to be processed and the graph structure data which has an incidence relation with the graph structure data of the relational data to be processed; and receiving the third graph structure data fed back by the graph structure data storage subsystem, and taking the third graph structure data as the processing result.
In an implementation manner of the embodiment of the present invention, the graphic database subsystem includes: at least one graphic database, and one graphic database processes relational data of one service type, and accordingly, the step of sending the relational data to be processed to the graphic database subsystem includes:
determining the content type of the relational data to be processed;
acquiring a service type corresponding to the content type;
and sending the relational data to be processed to a graphic database for processing the relational data of the acquired service type.
In an implementation manner of the embodiment of the present invention, the graph structure data storage subsystem is: a storage system for storing graph structure data based on a distributed architecture,
the step of sending the third graph structure data to the storage subsystem includes:
and sending the acquired service type and the third graph structure data to the graph structure data storage subsystem.
In each scheme provided by the embodiment of the invention, the data processing method based on the graph database can directly acquire the processing result stored in the graph structure data storage subsystem under the condition that the relation type data to be processed is not determined to be processed for the first time; and under the condition of processing the relational data to be processed for the first time, the graphic database subsystem carries out graphic structuring processing on the relational data to be processed to obtain first graphic structure data, the graphic data calculation subsystem obtains second graphic structure data which is in incidence relation with the first graphic structure data, the first graphic structure data and the second graphic structure data are merged to obtain third graphic structure data, and the third graphic structure data are used as a processing result of the relational data to be processed.
Referring to fig. 4, a schematic structural diagram of a data processing apparatus based on a graphic database according to an embodiment of the present invention is shown, and the data processing apparatus is applied to a central control subsystem in a data processing system based on a graphic database, where the data processing system includes: the device comprises a central control subsystem, a graphic database subsystem, a graphic data calculation subsystem and a graphic structure data storage subsystem, and comprises:
an obtaining module 800, configured to obtain to-be-processed relational data, determine whether to process the to-be-processed relational data for the first time, trigger the sending module if the determination result is yes, and trigger the requesting module if the determination result is no;
a sending module 900, configured to send the to-be-processed relational data to the graph database subsystem, and receive first graph structure data fed back by the graph database subsystem, where the first graph structure data is: carrying out graph structural processing on the relational data to be processed to obtain graph structural data; sending the first graph structure data to the graph data computing subsystem, and receiving third graph structure data fed back by the graph computing subsystem, wherein the third graph structure data is: and graph structure data obtained by merging the first graph structure data and the second graph structure data, wherein the second graph structure data is as follows: the graph structure data which has an incidence relation with the first graph structure data take the third graph structure data as the processing result of the relational data to be processed, and the third graph structure data are sent to the graph structure data storage subsystem;
a request module 1000, configured to request a processing result of the to-be-processed relational data from the graph structure data storage subsystem, where the processing result includes: the graph structure data of the relational data to be processed and the graph structure data which has an incidence relation with the graph structure data of the relational data to be processed; and receiving the third graph structure data fed back by the graph structure data storage subsystem, and taking the third graph structure data as the processing result.
In an implementation manner of the embodiment of the present invention, the graphic database subsystem includes: at least one graph database, and one graph database processes relational data of one service type, accordingly, the sending module 900 is specifically configured to:
determining the content type of the relational data to be processed;
acquiring a service type corresponding to the content type;
and sending the relational data to be processed to a graphic database for processing the relational data of the acquired service type.
In an implementation manner of the embodiment of the present invention, the graph structure data storage subsystem is: the storage system for storing graph structure data based on a distributed architecture, accordingly, the sending module 900 is further specifically configured to:
and sending the acquired service type and the third graph structure data to the graph structure data storage subsystem.
In each scheme provided by the embodiment of the invention, the data processing device based on the graph database can directly acquire the processing result stored in the graph structure data storage subsystem under the condition that the to-be-processed relational data is not determined to be processed for the first time; and under the condition of processing the relational data to be processed for the first time, the graphic database subsystem carries out graphic structuring processing on the relational data to be processed to obtain first graphic structure data, the graphic data calculation subsystem obtains second graphic structure data which is in incidence relation with the first graphic structure data, the first graphic structure data and the second graphic structure data are merged to obtain third graphic structure data, and the third graphic structure data are used as a processing result of the relational data to be processed.
As shown in fig. 5, an embodiment of the present invention further provides a central control subsystem, where the central control subsystem is: the central control subsystem in the data processing system based on the graphic database comprises: the system comprises a central control subsystem, a graphic database subsystem, a graphic data calculation subsystem and a graphic structure data storage subsystem; the above-mentioned central control subsystem includes: processor 001, communication interface 002, memory 003 and communication bus 004, wherein processor 001, communication interface 002, memory 003 communicate with each other via communication bus 004,
a memory 003 for storing a computer program;
the processor 001 is configured to implement the data processing method based on the graphic database according to the embodiment of the present invention when executing the program stored in the memory 003.
Specifically, the data processing method based on the graphic database includes:
obtaining relation type data to be processed, and judging whether the relation type data to be processed is processed for the first time;
if yes, sending the to-be-processed relational data to the graphic database subsystem, and receiving first graph structure data fed back by the graphic database subsystem, wherein the first graph structure data is as follows: carrying out graph structural processing on the relational data to be processed to obtain graph structural data; sending the first graph structure data to the graph data computing subsystem, and receiving third graph structure data fed back by the graph computing subsystem, wherein the third graph structure data is: and graph structure data obtained by merging the first graph structure data and the second graph structure data, wherein the second graph structure data is as follows: the graph structure data which has an incidence relation with the first graph structure data take the third graph structure data as the processing result of the relational data to be processed, and the third graph structure data are sent to the graph structure data storage subsystem;
if not, requesting a processing result of the to-be-processed relational data from the graph structure data storage subsystem, wherein the processing result comprises: the graph structure data of the relational data to be processed and the graph structure data which has an incidence relation with the graph structure data of the relational data to be processed; and receiving the third graph structure data fed back by the graph structure data storage subsystem, and taking the third graph structure data as the processing result.
It should be noted that, the other embodiments of the method for implementing data processing based on a graph database by the processor 001 executing the program stored in the memory 003 are the same as the embodiments provided in the foregoing method embodiments, and are not described again here.
In each scheme provided by the embodiment of the invention, the central control subsystem can directly acquire the processing result stored in the graph structure data storage subsystem under the condition that the to-be-processed relational data is not determined to be processed for the first time; and under the condition of processing the relational data to be processed for the first time, the graphic database subsystem carries out graphic structuring processing on the relational data to be processed to obtain first graphic structure data, the graphic data calculation subsystem obtains second graphic structure data which is in incidence relation with the first graphic structure data, the first graphic structure data and the second graphic structure data are merged to obtain third graphic structure data, and the third graphic structure data are used as a processing result of the relational data to be processed.
The communication bus mentioned in the above-mentioned central control subsystem may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.
The communication interface is used for communication between the central control subsystem and other equipment.
The Memory may include a Random Access Memory (RAM) or a Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.
The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components.
Yet another aspect of the present invention also provides a computer-readable storage medium, where the computer-readable storage medium is: a storage medium of a central control subsystem in a data processing system based on a graphic database, wherein the data processing system comprises: the system comprises a central control subsystem, a graphic database subsystem, a graphic data calculation subsystem and a graphic structure data storage subsystem; the computer-readable storage medium has instructions stored therein, which when run on a computer, cause the computer to implement any one of the above-described graphic database-based data processing methods.
Specifically, the data processing method based on the graphic database includes:
obtaining relation type data to be processed, and judging whether the relation type data to be processed is processed for the first time;
if yes, sending the to-be-processed relational data to the graphic database subsystem, and receiving first graph structure data fed back by the graphic database subsystem, wherein the first graph structure data is as follows: carrying out graph structural processing on the relational data to be processed to obtain graph structural data; sending the first graph structure data to the graph data computing subsystem, and receiving third graph structure data fed back by the graph computing subsystem, wherein the third graph structure data is: and graph structure data obtained by merging the first graph structure data and the second graph structure data, wherein the second graph structure data is as follows: the graph structure data which has an incidence relation with the first graph structure data take the third graph structure data as the processing result of the relational data to be processed, and the third graph structure data are sent to the graph structure data storage subsystem;
if not, requesting a processing result of the to-be-processed relational data from the graph structure data storage subsystem, wherein the processing result comprises: the graph structure data of the relational data to be processed and the graph structure data which has an incidence relation with the graph structure data of the relational data to be processed; and receiving the third graph structure data fed back by the graph structure data storage subsystem, and taking the third graph structure data as the processing result.
It should be noted that other embodiments of the method for implementing data processing based on a graph database by using the computer-readable storage medium are the same as the embodiments provided in the foregoing method embodiments, and are not described herein again.
In each scheme provided by the embodiment of the invention, by operating the instruction stored in the computer-readable storage medium, the processing result stored in the graph structure data storage subsystem can be directly obtained under the condition that the to-be-processed relational data is determined not to be processed for the first time; and under the condition of processing the relational data to be processed for the first time, the graphic database subsystem carries out graphic structuring processing on the relational data to be processed to obtain first graphic structure data, the graphic data calculation subsystem obtains second graphic structure data which is in incidence relation with the first graphic structure data, the first graphic structure data and the second graphic structure data are merged to obtain third graphic structure data, and the third graphic structure data are used as a processing result of the relational data to be processed.
Yet another aspect of the present invention also provides a computer program product comprising instructions for: program product for a central control subsystem in a data processing system based on a graphic database, wherein the data processing system comprises: the system comprises a central control subsystem, a graphic database subsystem, a graphic data calculation subsystem and a graphic structure data storage subsystem; when the computer program product runs on a computer, the computer is enabled to realize the data processing method based on the graphic database.
Specifically, the data processing method based on the graphic database includes:
obtaining relation type data to be processed, and judging whether the relation type data to be processed is processed for the first time;
if yes, sending the to-be-processed relational data to the graphic database subsystem, and receiving first graph structure data fed back by the graphic database subsystem, wherein the first graph structure data is as follows: carrying out graph structural processing on the relational data to be processed to obtain graph structural data; sending the first graph structure data to the graph data computing subsystem, and receiving third graph structure data fed back by the graph computing subsystem, wherein the third graph structure data is: and graph structure data obtained by merging the first graph structure data and the second graph structure data, wherein the second graph structure data is as follows: the graph structure data which has an incidence relation with the first graph structure data take the third graph structure data as the processing result of the relational data to be processed, and the third graph structure data are sent to the graph structure data storage subsystem;
if not, requesting a processing result of the to-be-processed relational data from the graph structure data storage subsystem, wherein the processing result comprises: the graph structure data of the relational data to be processed and the graph structure data which has an incidence relation with the graph structure data of the relational data to be processed; and receiving the third graph structure data fed back by the graph structure data storage subsystem, and taking the third graph structure data as the processing result.
It should be noted that other embodiments of the method for implementing data processing based on a graph database by using the computer program product are the same as the embodiments provided in the foregoing method embodiments, and are not described herein again.
In each scheme provided by the embodiment of the invention, by operating the computer program product containing the instruction, the processing result stored in the graph structure data storage subsystem can be directly obtained under the condition that the to-be-processed relational data is determined not to be processed for the first time; and under the condition of processing the relational data to be processed for the first time, the graphic database subsystem carries out graphic structuring processing on the relational data to be processed to obtain first graphic structure data, the graphic data calculation subsystem obtains second graphic structure data which is in incidence relation with the first graphic structure data, the first graphic structure data and the second graphic structure data are merged to obtain third graphic structure data, and the third graphic structure data are used as a processing result of the relational data to be processed.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, as for the method, apparatus, computer-readable storage medium, and computer program product embodiments, since they are substantially similar to the system embodiments, the description is relatively simple, and reference may be made to some descriptions of the system embodiments for relevant points.
The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.
Claims (10)
1. A data processing system based on a graphic database, the system comprising: the system comprises a central control subsystem, a graphic database subsystem, a graphic data calculation subsystem and a graphic structure data storage subsystem; wherein the content of the first and second substances,
the central control subsystem acquires the relational data to be processed and judges whether the relational data to be processed is processed for the first time, and if so, the relational data to be processed is sent to the graphic database subsystem; if not, requesting a processing result of the to-be-processed relational data from the graph structure data storage subsystem, wherein the processing result comprises: the graph structure data of the relational data to be processed and the graph structure data which has an incidence relation with the graph structure data of the relational data to be processed;
after the graphic database subsystem receives the relational data to be processed, carrying out graph structural processing on the relational data to be processed to obtain first graph structure data, and feeding back the first graph structure data to the central control subsystem; after receiving the first graph structure data, the central control subsystem sends the first graph structure data to the graph data calculation subsystem; after receiving the first graph structure data, the graph data calculation subsystem acquires second graph structure data which is associated with the first graph structure data, combines the first graph structure data and the second graph structure data to obtain third graph structure data, and feeds the third graph structure data back to the central control subsystem; after receiving the third graph structure data, the central control subsystem takes the third graph structure data as a processing result of the to-be-processed relational data and sends the third graph structure data to the graph structure data storage subsystem; the graph structure data storage subsystem receives and stores the third graph structure data;
and after receiving the third graph structure data, the central control subsystem takes the third graph structure data as the processing result.
2. The system of claim 1, wherein the graphics database subsystem comprises: at least one graph database, and one graph database handles relational data for one business type.
3. The system of claim 2, wherein the step of sending the pending relational data to the graphic database subsystem comprises:
determining the content type of the relational data to be processed;
acquiring a service type corresponding to the content type;
and sending the relational data to be processed to a graphic database for processing the relational data of the acquired service type.
4. The system of claim 1, wherein the step of graph-structuring the relational data to be processed to obtain the first graph-structured data comprises:
determining an object contained in the relational data to be processed;
acquiring the determined association relation among the objects;
and obtaining the first graph structure data according to the determined objects and the obtained association relation.
5. The system according to claim 4, wherein the step of obtaining second graph structure data having an association relationship with the first graph structure data and merging the first graph structure data and the second graph structure data to obtain third graph structure data comprises:
acquiring second graph structure data, wherein the second graph structure data and the first graph structure data comprise at least one same object;
and performing deduplication processing on the same object contained in the first graph structure data and the second graph structure data to obtain third graph structure data.
6. The system of claim 3, wherein the graph structure data storage subsystem is: a storage system for storing graph structure data based on a distributed architecture,
the step of sending the third graph structure data to the graph structure data storage subsystem includes:
sending the acquired service type and the third graph structure data to the graph structure data storage subsystem;
accordingly, the step of the graph structure data storage subsystem receiving and storing the third graph structure data comprises:
determining a storage partition corresponding to the received service type;
storing the third graph structure data to the determined storage partition.
7. The data processing method based on the graphic database is applied to a central control subsystem in a data processing system based on the graphic database, wherein the data processing system comprises: the method comprises a central control subsystem, a graphic database subsystem, a graphic data calculation subsystem and a graph structure data storage subsystem, and comprises the following steps:
obtaining relation type data to be processed, and judging whether the relation type data to be processed is processed for the first time;
if yes, sending the to-be-processed relational data to the graphic database subsystem, and receiving first graph structure data fed back by the graphic database subsystem, wherein the first graph structure data is as follows: carrying out graph structural processing on the relational data to be processed to obtain graph structural data; sending the first graph structure data to the graph data calculation subsystem, and receiving third graph structure data fed back by the graph data calculation subsystem, wherein the third graph structure data is: and graph structure data obtained by merging the first graph structure data and the second graph structure data, wherein the second graph structure data is as follows: the graph structure data which has an incidence relation with the first graph structure data take the third graph structure data as the processing result of the relational data to be processed, and the third graph structure data are sent to the graph structure data storage subsystem;
if not, requesting a processing result of the to-be-processed relational data from the graph structure data storage subsystem, wherein the processing result comprises: the graph structure data of the relational data to be processed and the graph structure data which has an incidence relation with the graph structure data of the relational data to be processed; and receiving the third graph structure data fed back by the graph structure data storage subsystem, and taking the third graph structure data as the processing result.
8. The method of claim 7, wherein the graphics database subsystem comprises: at least one graphic database, and one graphic database processes relational data of one service type, and accordingly, the step of sending the relational data to be processed to the graphic database subsystem includes:
determining the content type of the relational data to be processed;
acquiring a service type corresponding to the content type;
and sending the relational data to be processed to a graphic database for processing the relational data of the acquired service type.
9. The method of claim 8, wherein the graph structure data storage subsystem is: a storage system for storing graph structure data based on a distributed architecture,
the step of sending the third graph structure data to the storage subsystem includes:
and sending the acquired service type and the third graph structure data to the graph structure data storage subsystem.
10. A data processing device based on a graphic database is applied to a central control subsystem in a data processing system based on the graphic database, wherein the data processing system comprises: the device comprises a central control subsystem, a graphic database subsystem, a graphic data calculation subsystem and a graphic structure data storage subsystem, and the device comprises:
the obtaining module is used for obtaining the relational data to be processed and judging whether the relational data to be processed is processed for the first time, the sending module is triggered under the condition that the judging result is yes, and the requesting module is triggered under the condition that the judging result is no;
a sending module, configured to send the to-be-processed relational data to the graph database subsystem, and receive first graph structure data fed back by the graph database subsystem, where the first graph structure data is: carrying out graph structural processing on the relational data to be processed to obtain graph structural data; sending the first graph structure data to the graph data calculation subsystem, and receiving third graph structure data fed back by the graph data calculation subsystem, wherein the third graph structure data is: and graph structure data obtained by merging the first graph structure data and the second graph structure data, wherein the second graph structure data is as follows: the graph structure data which has an incidence relation with the first graph structure data take the third graph structure data as the processing result of the relational data to be processed, and the third graph structure data are sent to the graph structure data storage subsystem;
a request module, configured to request a processing result of the to-be-processed relational data from the graph structure data storage subsystem, where the processing result includes: the graph structure data of the relational data to be processed and the graph structure data which has an incidence relation with the graph structure data of the relational data to be processed; and receiving the third graph structure data fed back by the graph structure data storage subsystem, and taking the third graph structure data as the processing result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810548344.4A CN108984592B (en) | 2018-05-31 | 2018-05-31 | Data processing system, method and device based on graphic database |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810548344.4A CN108984592B (en) | 2018-05-31 | 2018-05-31 | Data processing system, method and device based on graphic database |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108984592A CN108984592A (en) | 2018-12-11 |
| CN108984592B true CN108984592B (en) | 2020-11-20 |
Family
ID=64542851
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810548344.4A Active CN108984592B (en) | 2018-05-31 | 2018-05-31 | Data processing system, method and device based on graphic database |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108984592B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102193990A (en) * | 2011-03-25 | 2011-09-21 | 北京世纪互联工程技术服务有限公司 | Pattern database and realization method thereof |
| CN104123369A (en) * | 2014-07-24 | 2014-10-29 | 中国移动通信集团广东有限公司 | CMDB system based on graphic data base and implementation method |
| CN106874422A (en) * | 2017-01-25 | 2017-06-20 | 东南大学 | A kind of figure querying method of facing relation type database |
| CN107016072A (en) * | 2017-03-23 | 2017-08-04 | 成都市公安科学技术研究所 | Knowledge-based inference system and method based on social networks knowledge mapping |
| CN107679160A (en) * | 2017-09-28 | 2018-02-09 | 深圳市华傲数据技术有限公司 | Data processing method and device based on chart database |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US4506326A (en) * | 1983-02-28 | 1985-03-19 | International Business Machines Corporation | Apparatus and method for synthesizing a query for accessing a relational data base |
| US5201046A (en) * | 1990-06-22 | 1993-04-06 | Xidak, Inc. | Relational database management system and method for storing, retrieving and modifying directed graph data structures |
| JPH07248948A (en) * | 1994-03-10 | 1995-09-26 | Fujitsu Ltd | Dynamic constitution system for data base |
| US6675170B1 (en) * | 1999-08-11 | 2004-01-06 | Nec Laboratories America, Inc. | Method to efficiently partition large hyperlinked databases by hyperlink structure |
| US6665863B1 (en) * | 2000-05-31 | 2003-12-16 | Microsoft Corporation | Data referencing within a database graph |
| EP1367503A1 (en) * | 2002-05-24 | 2003-12-03 | Sun Microsystems, Inc. | Method for displaying and modifying a relational database schema |
| US20180144061A1 (en) * | 2016-11-23 | 2018-05-24 | Linkedin Corporation | Edge store designs for graph databases |
-
2018
- 2018-05-31 CN CN201810548344.4A patent/CN108984592B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102193990A (en) * | 2011-03-25 | 2011-09-21 | 北京世纪互联工程技术服务有限公司 | Pattern database and realization method thereof |
| CN104123369A (en) * | 2014-07-24 | 2014-10-29 | 中国移动通信集团广东有限公司 | CMDB system based on graphic data base and implementation method |
| CN106874422A (en) * | 2017-01-25 | 2017-06-20 | 东南大学 | A kind of figure querying method of facing relation type database |
| CN107016072A (en) * | 2017-03-23 | 2017-08-04 | 成都市公安科学技术研究所 | Knowledge-based inference system and method based on social networks knowledge mapping |
| CN107679160A (en) * | 2017-09-28 | 2018-02-09 | 深圳市华傲数据技术有限公司 | Data processing method and device based on chart database |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108984592A (en) | 2018-12-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11605087B2 (en) | Method and apparatus for identifying identity information | |
| CN110874440B (en) | Information pushing method and device, model training method and device, and electronic equipment | |
| WO2018214835A1 (en) | Block chain service acceptance and consensus method and device | |
| US10091230B1 (en) | Aggregating identity data from multiple sources for user controlled distribution to trusted risk engines | |
| TWI706663B (en) | Data storage method and system based on multiple blockchain networks | |
| CN111756644B (en) | Hot spot current limiting method, system, equipment and storage medium | |
| WO2020108152A1 (en) | Method, device and electronic equipment for preventing misuse of identity data | |
| CN110245684B (en) | Data processing method, electronic device, and medium | |
| CN107566499B (en) | Data synchronization method, device and system | |
| US11290602B2 (en) | Information processing device, information processing system, and information processing method | |
| CN113297409A (en) | Image searching method and device, electronic equipment and storage medium | |
| CN108984592B (en) | Data processing system, method and device based on graphic database | |
| CN106611118B (en) | Method and device for applying login credentials | |
| CN108491160B (en) | Data writing method and device | |
| CN112988604B (en) | Object testing method, testing system, electronic device and readable storage medium | |
| US10114973B2 (en) | Protecting user privacy from intrusive mobile applications | |
| CN114218283A (en) | Abnormality detection method, apparatus, device, and medium | |
| AU2017371238A1 (en) | Method of inputting document information, device, server, and storage medium | |
| CN111104528B (en) | Picture acquisition method and device and client | |
| WO2020108055A1 (en) | Multiple blockchain network-based data reading method and system | |
| CN109561123B (en) | Token caching method and device | |
| CN106845202B (en) | File access method, device and system for face recognition system | |
| CN112215527A (en) | Logistics management method and device | |
| US9600508B1 (en) | Data layer service availability | |
| CN116501976B (en) | Data recommendation, model training, similar user analysis methods, apparatus and media |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |