CN111737252A - Data fusion method and system based on data center - Google Patents
Data fusion method and system based on data center Download PDFInfo
- Publication number
- CN111737252A CN111737252A CN202010442115.1A CN202010442115A CN111737252A CN 111737252 A CN111737252 A CN 111737252A CN 202010442115 A CN202010442115 A CN 202010442115A CN 111737252 A CN111737252 A CN 111737252A
- Authority
- CN
- China
- Prior art keywords
- data
- database
- mirror
- mirror image
- write
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2358—Change logging, detection, and notification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a data fusion method and a system based on a data center, wherein the method comprises the following steps: acquiring a data table in a service system used by a networking hall, constructing a mirror database in the data center, and generating a corresponding mirror data table; the data center receives a write-back data request sent by the office hall, performs write-back operation on the corresponding mirror image data table according to the write-back data to generate a corresponding write-back operation record, starts a trigger to implement data exchange based on a preset rule, and synchronizes the data of the mirror image database to a service system; after the data synchronization of the mirror database is completed, the data center cleans and converts corresponding data and stores the data into a main database of the data center; and the networking hall acquires display data from the master database. The invention realizes data fusion by converting the data written back in the mirror database, keeps the consistency of the data and really realizes one-stop work, service and data.
Description
Technical Field
The invention relates to the technical field of data storage, in particular to a data fusion method and system based on a data center.
Background
At present, a data exchange technology is generally adopted by an online office hall to realize the exchange between a data center and a service system, but the online office hall only allows data to be obtained from the data center, cannot write back the data to the data center, and cannot directly obtain or write back the data from the service system to a service system database, as shown in fig. 1, the data is mainly considered that the data written back to the data center or the service system is difficult to achieve the unification of rules and the real-time performance of data synchronization, so that data errors or inconsistency of data tables of the data center and the service system are caused. Therefore, the online office hall adopts the traditional mode to exchange data, data fusion is difficult to achieve, information, data, office, service and the like are split, the one-stop type of office, service and data cannot be really realized, and the user experience is greatly influenced.
Disclosure of Invention
The present invention is directed to solving at least one of the problems of the prior art. Therefore, the invention provides a data fusion method based on a data center, which can write back data to the data center or the data center to realize data fusion.
The invention also provides a data fusion system based on the data center, which uses the data fusion method based on the data center.
The data center-based data fusion method according to the embodiment of the first aspect of the invention comprises the following steps: acquiring a data table in a service system used by a networking hall, constructing a mirror database in the data center, and generating a corresponding mirror data table; the data center receives a write-back data request sent by the office hall, performs write-back operation on the corresponding mirror image data table according to the write-back data to generate a corresponding write-back operation record, starts a trigger to implement data exchange based on a preset rule, and synchronizes the data of the mirror image database to a service system; after the data synchronization of the mirror database is completed, the data center cleans and converts corresponding data and stores the data into a main database of the data center; the networking hall is connected with the master database, and display data are obtained from the master database.
The data fusion method based on the data center provided by the embodiment of the invention at least has the following beneficial effects: by constructing the mirror database, the mirror database transfers the data and stores the written-back data into the data center and the service system, data fusion is realized, the consistency of the data is kept, and the one-stop type of work handling, service and data is realized in a real sense.
According to some embodiments of the invention, the method of data exchange comprises: updating a mirror database data table, namely comparing the mirror database data table with the structure rule information of the corresponding data table in the service system, and updating the mirror database data table according to the comparison result; and a service system data table synchronization step, namely synchronizing the data of the mirror image data table to the service system according to the write-back operation record. And keeping the data table of the mirror image library consistent with the data table structure and data of the service system.
According to some embodiments of the invention, the mirror database table updating step comprises: a table structure detection step, namely acquiring a first table structure of a corresponding data table in the service system according to the mirror image data table, and judging whether a second table structure of the mirror image data table is consistent with the first table structure to obtain a table structure detection result; a rule detection step, namely acquiring a first referential integrity rule of a corresponding data table in the service system according to the mirror image data table, and judging whether a second referential integrity rule of the mirror image data table is consistent with the first referential integrity rule or not to obtain a rule detection result; and updating the table structure of the mirror image data table if at least one of the table structure detection result and the rule detection result is inconsistent. And maintaining the structure of the mirror image base table consistent with the corresponding data table structure in the service system according to the table structure definition and the reference integrity rule.
According to some embodiments of the invention, the method of writing the writeback data into the mirror database and generating the corresponding writeback operation record comprises: and according to the type of the write-back data request, adding a corresponding operation mark in a corresponding record of the mirror image data table, and recording an operation starting position and an operation number. The data of the mirror image library is operated and recorded, but is not actually deleted, so that the data can be conveniently and integrally synchronized to a service system according to a preset rule instead of frequently requesting for operation.
According to some embodiments of the present invention, the method for synchronizing the data of the mirror database to the business system according to the write-back operation record comprises: and positioning to the operation starting position in a corresponding data table in the service system according to the operation mark of the mirror image data table, and triggering corresponding updating operation on the data of the operation number. And performing corresponding operation on the service system data table according to the operation record of the mirror database, finishing the write-back operation on the service system data table, and ensuring that the data of the service system data table is consistent with the data in the mirror database.
According to some embodiments of the invention, the method of constructing a mirror database comprises: extracting a data table used by the networking hall from the service system, and acquiring the first table structure and the first reference integrity rule of the data table; and generating the corresponding mirror image data table in the mirror image database according to the first table structure and the first reference integrity rule, and storing the second table structure and the second reference integrity rule of the mirror image data table. The mirror image data table is generated in advance according to the table structure definition of the business data table and the reference integrity rule, so that the data table of the mirror image database is consistent with the corresponding data table structure in the business system, errors are avoided during subsequent write-back of data, the synchronization quantity of the mirror image table structure caused by the change of the business data table structure rule can be reduced, and the subsequent write-back processing speed is improved.
According to some embodiments of the invention, the method for obtaining the display data by the website hall comprises the following steps: the website hall acquires the display data by sending a data request to the master database, or the master database pushes the display data to the website hall in real time; the preset rules include: and starting the trigger according to a preset time interval. The flexibility of the network office hall is increased by acquiring display data; the write-back triggers the timing operation, so that the phenomenon that the mirror database and the service system work normally due to frequent synchronization is avoided while the data of the mirror database is inconsistent with the data of the service system for a long time is avoided.
The data fusion system based on the data center comprises a service system, a networking hall and the data center, wherein the data center comprises: the mirror database is used for storing mirror image data tables corresponding to the data tables in the business system processed by the networked hall; the main database is used for storing the mirror image data table which keeps synchronization with the service system; and the data management module is used for carrying out corresponding operation on the mirror database according to the write-back data request of the office hall and synchronizing the data of the mirror database to the service system according to a preset rule.
The data fusion system based on the data center provided by the embodiment of the invention at least has the following beneficial effects: the mirror database is used for write-back transfer, the business system data can be changed in the network handling hall, the data consistency of the data center and the business system is guaranteed, data fusion is achieved, the data consistency is kept, and the one-stop type of handling, service and data is achieved in the true sense.
According to some embodiments of the invention, the data management module comprises: the write-back data processing unit is used for adding corresponding operation marks in corresponding records of the mirror image data table according to the type of the write-back data request and recording an operation starting position and operation number; the mirror image data table structure synchronization unit is used for updating the corresponding mirror image data table according to a first table structure and a first reference integrity definition of the data table of the business system based on a preset rule; and the service system data synchronization unit is used for positioning to the operation starting position in a corresponding data table in the service system according to the operation mark based on a preset rule and triggering corresponding updating operation on the operation destination data. The mirror database processes the write-back data and adds an operation mark in a corresponding record, so that the write-back data is synchronized to a service system conveniently; the mirror image data table structure synchronization unit enables the data table of the mirror image base and the data table of the service system to be consistent in structure, and write-back data synchronization to the service system is facilitated.
According to some embodiments of the invention, the website lobby comprises: the data acquisition module is used for generating write-back data and sending a write-back request to the data center; and the data application module is used for acquiring and displaying data from the main database. The data acquisition module collects the write-back data and is connected with the mirror database; the data application module is used for displaying data; the handling hall processes data in a classified and orderly manner, so that the efficiency is improved, and the error occurrence rate is reduced.
Drawings
The above and/or additional aspects and advantages of the present invention will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
FIG. 1 is a schematic diagram of data exchange in a conventional online office hall;
FIG. 2 is a schematic illustration of the steps of a method according to an embodiment of the present invention;
FIG. 3 is a diagram illustrating a process of writing back data according to the method of the present invention;
FIG. 4 is a block schematic diagram of modules of a system of an embodiment of the present invention;
FIG. 5 is a schematic block diagram of a switch management module in the system of an embodiment of the present invention;
fig. 6 is a schematic diagram of data interaction of the system according to the embodiment of the present invention.
Reference numerals:
the data center 100, the service system 200 and the networking hall 300;
write-back data processing unit 131, mirror data table structure synchronization unit 132, service system data synchronization unit 133;
a data collection module 310, a data application module 320, a lobby database 330.
Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the accompanying drawings are illustrative only for the purpose of explaining the present invention, and are not to be construed as limiting the present invention.
In the description of the present invention, the meaning of a plurality of means is one or more, the meaning of a plurality of means is two or more, and larger, smaller, larger, etc. are understood as excluding the number, and larger, smaller, inner, etc. are understood as including the number. If the first and second are described for the purpose of distinguishing technical features, they are not to be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated or implicitly indicating the precedence of the technical features indicated.
Referring to fig. 2, a method of an embodiment of the present invention includes: acquiring a data table in a service system used by a networking hall, constructing a mirror database (the mirror database is also called as a mirror database in the invention) in the data center, and generating a corresponding mirror image data table; the data center receives a write-back data request sent by a networking office hall, writes the write-back data into a mirror database, performs write-back operation on a corresponding mirror image data table to generate a corresponding write-back operation record, starts a trigger to implement data exchange based on a preset rule, and synchronizes the data of the mirror database to a service system; after the data in the mirror database is synchronized with the service system, the data center cleans and converts corresponding data and stores the data into a main database of the data center; the networking hall is connected with the master database, and the display data are obtained from the master database. Wherein the write back operation records include, but are not limited to: operation time, operation type, operation start position, operation number and the like.
The data exchange process of the method of the embodiment of the invention comprises the following steps: updating a data table of a mirror database and synchronizing a data table of a service system. In the step of updating the data table of the mirror image library, comparing the structure rule information of the data table corresponding to the mirror image data table and the service system, and updating the mirror image data table according to the comparison result, the method comprises the following steps: table structure detection, rule detection and updating. In the table structure detection step, a first table structure of a corresponding data table in the service system is obtained according to the mirror image data table, and whether a second table structure of the mirror image data table is consistent with the first table structure is judged to obtain a table structure detection result. For example: data updating is carried out in a data table A ' (called a mirror table A ' for short) in the mirror image library, and the mirror table A ' is a mirror image of the data table A in the service system; when data in the mirror image table A 'are synchronized to the data table A, firstly, judging whether the table structure of the data table A is consistent with that of the mirror image table A' or not, and obtaining a table structure detection result; if not, the mirror table A' needs to be updated. In the rule detection step, a first referential integrity rule of a corresponding data table in the service system is obtained according to the mirror image data table, and whether a second referential integrity rule of the mirror image data table is consistent with the first referential integrity rule or not is judged to obtain a rule detection result. For example: the mirror image table A 'is a mirror image of the data table A in the service system, and whether the reference integrity of the mirror image table A' is consistent with that of the data table A or not is checked and judged; if not, the table structure of the mirror table A' and its referential rules are updated. And the updating step judges whether the table structure detection result and the rule detection result are consistent, and if at least one item is inconsistent, the table structure of the corresponding mirror image data table in the mirror image library is updated. And in the step of synchronizing the data table of the service system, synchronizing the data of the mirror image library to the service system according to the write-back operation record.
In some embodiments of the present invention, the table structure detecting step finds that the second table structure with the mirror table is inconsistent with its corresponding first table structure, i.e. triggers an update of the mirror table. And then, continuously checking the reference integrity rules of the mirror image table, including the updating rules, the deleting rules and the inserting rules, whether the reference integrity rules are consistent with the reference integrity rules of the corresponding data tables in the database of the service system, and if the reference integrity rules are not consistent with the reference integrity rules, triggering the updating of all table structures related to the mirror image table and the mirror image table in the mirror image database.
The write-back data processing process according to an embodiment of the present invention, referring to fig. 3, includes: and the data center detects that the office hall has a data write-back request, triggers a data exchange strategy, writes the data write-back into the mirror database, starts a trigger according to a preset time interval to implement the data exchange strategy, and synchronously updates the mirror database table data to the service system database. The preset time interval can be set to be real-time and specific minutes according to application requirements. When the write-back data is written into the mirror image database, the types of the write-back data requests, such as addition, insertion and deletion, are detected, operation marks of addition, insertion, deletion and the like are added behind corresponding records of the mirror image database table, and the operation initial position and the operation number are recorded. In the data exchange, firstly, whether the table structure of the mirror image database table is consistent with the table structure of the business system database is detected. If not, triggering the mirror image library to update the table structure; if the integrity rules are consistent, detecting whether the integrity rules (including updating rules, deleting rules and inserting rules) of the mirror database table are consistent with the integrity rules of the service system database table, and if not, triggering the mirror database to update the table structure. And if the table structure is consistent with the reference integrity rule, updating the data of the new service system data table according to the write-back operation record. Reading the write-back operation record (including, but not limited to, operation time, operation type, operation start position and operation number), positioning to the operation start position in the corresponding data table in the business system according to the operation mark recorded in the mirror data table, triggering corresponding update operation (such as addition, insertion, deletion, etc.) on the data of the operation number, and synchronizing the data in the mirror data table to the business system. In other embodiments of the present invention, the mirror data table is directly read, and the corresponding data table in the service system is updated according to the operation flag.
The system of an embodiment of the invention comprises: a data center 100, a business system 200, and a networking lobby 300. Referring to fig. 4, among others, the data center 100 includes: a mirror database 110 for storing mirror data tables corresponding to the data tables in the business system for processing the working hall; a main database 120 for storing a mirror data table synchronized with the service system; and the data management module 130 is configured to perform corresponding operations on the mirror database according to the request for writing back data of the office hall, and synchronize data of the mirror database to the service system according to a preset rule. In some embodiments of the present invention, the predetermined rule is to synchronize data to the service system according to a predetermined time interval; in some embodiments of the present invention, the preset rule is to synchronize data to the service system according to the amount of the write-back data; in other embodiments of the present invention, the predetermined rule is to synchronize the data to the business system according to the operation record amount of the mirror data table.
Referring to fig. 5, the data management module 130 includes: a write-back data processing unit 131, a mirror data table structure synchronization unit 132, and a service system data synchronization unit 133. And the write-back data processing unit 131 is configured to receive the write-back data request, perform corresponding processing on the mirrored data table according to the type of the write-back data request, add a relevant operation flag to a corresponding record in the mirrored data table, and record an operation start position and an operation number. For example, adding new data, after adding a new record in the mirror image data table, adding a new mark at the record, recording the operation position and the operation number, and inserting the updated data similarly; and deleting the data, searching the corresponding record, adding a deletion mark, recording the operation position and the operation number, and not performing record deletion processing on the mirror image library. It should be understood that the mirror database records are not deleted, and only are deleted, so that the records in the mirror database can be asynchronously synchronized to the service system according to the preset rule. The mirror image data table structure synchronization unit 132 is configured to update the corresponding mirror image data table according to the first table structure and the first referential integrity definition of the data table of the service system based on a preset rule. The service system data synchronization unit 133 is configured to read an operation record (including, but not limited to, operation time, operation type, operation start position, and operation number) of the mirror image table based on a preset rule, locate the operation record to the operation start position in a corresponding data table in the service system according to an operation flag recorded in the mirror image, and trigger a corresponding update operation on data of the same operation number in the record. In other embodiments of the present invention, the service system data synchronization unit 133 is configured to directly read the mirror data table based on a preset rule, and update the corresponding data table in the service system according to the operation flag.
The networking hall 300 in the system according to the embodiment of the present invention, referring to fig. 4, includes: the data acquisition module 310 is configured to generate write-back data and send a write-back request to the data center; and the data application module 320 is used for acquiring data from the main database and displaying the data to the user. In some systems of the present invention, referring to FIG. 6, the networking lobby 300 further includes a lobby database 330 for storing data received and processed during operation of the networking lobby 300.
Fig. 6 is a schematic diagram of data interaction after implementation of system deployment according to an embodiment of the present invention. When the mirror database 110 is deployed in the data center, the data tables used by the office hall are extracted from the business system, the first table structure and the first integrity reference rule of the data tables are obtained, then the corresponding mirror data tables are generated in the mirror database 110 according to the first table structure and the first integrity reference rule, and the second table structure and the second integrity reference of the mirror data tables are stored. Obviously, when the mirror database 110 is initially created, the second table structure of the mirror table is the same as the first table structure of the corresponding data table in the business system, and the second integrity reference rule of the mirror table is the same as the first integrity reference rule of the corresponding data table in the business system. The website hall 300 acquires data from the master database 120 through the data application module 320, wherein the method for acquiring data includes: the data request is directly sent to the master database 120 to obtain the display data, or the master database 120 is pushed to the networked hall 300 in real time, the hall database 330 of the networked hall 300 receives the pushed data, and the data application module 320 reads and displays the pushed data from the hall database 330. Data related to the online hall 300 in the operation process is stored in the hall database 330, and the data collection module 310 collects write-back data from the hall database 330 and sends a write-back request to the mirror database 110. The mirror database 110 of the data center 100 receives the writeback request, writes the writeback data into the mirror database, starts data exchange according to a preset rule, and synchronizes the mirror data to the service database in the service system 200. And after the mirror image data are synchronized, cleaning and converting the data in the mirror image database and storing the data in the main database. In other embodiments of the present invention, the data collection module 310 obtains the write back data directly from the data application module 320. It should be understood that, in the embodiment of the present invention, if the lobby 300 uses the data tables in the service databases of more than one service system, the mirror database of the corresponding data may be generated according to the service systems.
Although specific embodiments have been described herein, those of ordinary skill in the art will recognize that many other modifications or alternative embodiments are equally within the scope of this disclosure. For example, any of the functions and/or processing capabilities described in connection with a particular device or component may be performed by any other device or component. In addition, while various illustrative implementations and architectures have been described in accordance with embodiments of the present disclosure, those of ordinary skill in the art will recognize that many other modifications of the illustrative implementations and architectures described herein are also within the scope of the present disclosure.
Certain aspects of the present disclosure are described above with reference to block diagrams and flowchart illustrations of systems, methods, apparatus and/or computer program products according to example embodiments. It will be understood that one or more blocks of the block diagrams and flowchart illustrations, and combinations of blocks in the block diagrams and flowchart illustrations, respectively, can be implemented by executing computer-executable program instructions. Also, according to some embodiments, some blocks of the block diagrams and flow diagrams may not necessarily be performed in the order shown, or may not necessarily be performed in their entirety. In addition, additional components and/or operations beyond those shown in the block diagrams and flow diagrams may be present in certain embodiments.
Accordingly, blocks of the block diagrams and flowchart illustrations support combinations of means for performing the specified functions, combinations of elements or steps for performing the specified functions and program instruction means for performing the specified functions. It will also be understood that each block of the block diagrams and flowchart illustrations, and combinations of blocks in the block diagrams and flowchart illustrations, can be implemented by special purpose hardware-based computer systems that perform the specified functions, elements or steps, or combinations of special purpose hardware and computer instructions.
Program modules, applications, etc. described herein may include one or more software components, including, for example, software objects, methods, data structures, etc. Each such software component may include computer-executable instructions that, in response to execution, cause at least a portion of the functionality described herein (e.g., one or more operations of the illustrative methods described herein) to be performed. The software components may be encoded in any of a variety of programming languages.
The embodiments of the present invention have been described in detail with reference to the accompanying drawings, but the present invention is not limited to the above embodiments, and various changes can be made within the knowledge of those skilled in the art without departing from the gist of the present invention.
Claims (10)
1. A data fusion method based on a data center is characterized by comprising the following steps:
acquiring a data table in a service system used by a networking hall, constructing a mirror database in the data center, and generating a corresponding mirror data table;
the data center receives a write-back data request sent by the office hall, performs write-back operation on the corresponding mirror image data table according to the write-back data to generate a corresponding write-back operation record, starts a trigger to implement data exchange based on a preset rule, and synchronizes the data of the mirror image database to a service system;
after the data synchronization of the mirror database is completed, the data center cleans and converts corresponding data and stores the data into a main database of the data center;
the networking hall is connected with the master database, and display data are obtained from the master database.
2. The data center-based data fusion method of claim 1, wherein the data exchange method comprises:
updating a mirror database data table, namely comparing the mirror database data table with the structure rule information of the corresponding data table in the service system, and updating the mirror database data table according to the comparison result;
and a service system data table synchronization step, namely synchronizing the data of the mirror image data table to the service system according to the write-back operation record.
3. The data center-based data fusion method of claim 2, wherein the mirrored data table updating step comprises:
a table structure detection step, namely acquiring a first table structure of a corresponding data table in the service system according to the mirror image data table, and judging whether a second table structure of the mirror image data table is consistent with the first table structure to obtain a table structure detection result;
a rule detection step, namely acquiring a first referential integrity rule of a corresponding data table in the service system according to the mirror image data table, and judging whether a second referential integrity rule of the mirror image data table is consistent with the first referential integrity rule or not to obtain a rule detection result;
and updating the table structure of the corresponding mirror image data table in the mirror image database if at least one of the table structure detection result and the rule detection result is inconsistent.
4. The data center-based data fusion method of claim 2, wherein writing the writeback data to the mirror database, the method of generating a corresponding writeback record comprises:
and according to the type of the write-back data request, adding a corresponding operation mark in a corresponding record of the mirror image data table, and recording an operation starting position and an operation number.
5. The data center-based data fusion method of claim 4, wherein the method for synchronizing the data of the mirror database to the business system according to the write-back operation record comprises:
and positioning to the operation starting position in a corresponding data table in the service system according to the operation mark of the mirror image data table, and triggering corresponding updating operation on the data of the operation number.
6. The data center-based data fusion method of claim 1, wherein the method of building a mirror database comprises:
extracting a data table used by the networking hall from the service system, and acquiring the first table structure and the first reference integrity rule of the data table;
and generating the corresponding mirror image data table in the mirror image database according to the first table structure and the first reference integrity rule, and storing the second table structure and the second reference integrity rule of the mirror image data table.
7. The data center-based data fusion method of claim 1, wherein the method for the website hall to obtain the presentation data comprises: the website hall acquires the display data by sending a data request to the master database, or the master database pushes the display data to the website hall in real time; the preset rules include: and starting the trigger according to a preset time interval.
8. A data center based data fusion system using the method of any one of claims 1 to 7, comprising a business system, a web hall and a data center, wherein the data center comprises:
the mirror database is used for storing mirror image data tables corresponding to the data tables in the business system processed by the networked hall;
the main database is used for storing the mirror image data table which keeps synchronization with the service system;
and the data management module is used for carrying out corresponding operation on the mirror database according to the write-back data request of the office hall and synchronizing the data of the mirror database to the service system according to a preset rule.
9. The data center-based data fusion system of claim 8, wherein the data management module comprises:
the write-back data processing unit is used for adding corresponding operation marks in corresponding records of the mirror image data table according to the type of the write-back data request and recording an operation starting position and operation number;
the mirror image data table structure synchronization unit is used for updating the corresponding mirror image data table according to a first table structure and a first reference integrity definition of the data table of the business system based on a preset rule;
and the service system data synchronization unit is used for positioning to the operation starting position in a corresponding data table in the service system according to the operation mark based on a preset rule and triggering corresponding updating operation on the operation destination data.
10. The data center-based data fusion system of claim 9, wherein the website lobby includes:
the data acquisition module is used for generating write-back data and sending a write-back request to the data center;
and the data application module is used for acquiring and displaying data from the main database.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010442115.1A CN111737252B (en) | 2020-05-22 | 2020-05-22 | Data fusion method and system based on data center |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010442115.1A CN111737252B (en) | 2020-05-22 | 2020-05-22 | Data fusion method and system based on data center |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111737252A true CN111737252A (en) | 2020-10-02 |
| CN111737252B CN111737252B (en) | 2023-10-03 |
Family
ID=72647627
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010442115.1A Active CN111737252B (en) | 2020-05-22 | 2020-05-22 | Data fusion method and system based on data center |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111737252B (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101820449A (en) * | 2010-04-20 | 2010-09-01 | 江苏电力调度通信中心 | Cross-safety zone application service isolation platform |
| US7908448B1 (en) * | 2007-01-30 | 2011-03-15 | American Megatrends, Inc. | Maintaining data consistency in mirrored cluster storage systems with write-back cache |
| US20130042156A1 (en) * | 2011-08-12 | 2013-02-14 | Oracle International Corporation | Utilizing multiple storage devices to reduce write latency for database logging |
| CN104615759A (en) * | 2015-02-13 | 2015-05-13 | 厦门雅迅网络股份有限公司 | Data synchronization method for different business system platforms |
| CN105718570A (en) * | 2016-01-20 | 2016-06-29 | 北京京东尚科信息技术有限公司 | Data migration method and device used for database |
| CN106844390A (en) * | 2015-12-07 | 2017-06-13 | 北京航天长峰科技工业集团有限公司 | A kind of inter-sectional data resource cut-in method |
| KR20180037571A (en) * | 2016-10-04 | 2018-04-12 | 삼성전자주식회사 | Method of data storage and operating methode of datacenter cluster caching system |
| CN108647277A (en) * | 2018-05-03 | 2018-10-12 | 山东师范大学 | A kind of mobile campus comprehensive service platform and its working method |
| US20190007206A1 (en) * | 2017-06-30 | 2019-01-03 | Microsoft Technology Licensing, Llc | Encrypting object index in a distributed storage environment |
| US20190197027A1 (en) * | 2016-12-19 | 2019-06-27 | Tencent Technology (Shenzhen) Company Limited | Data management method and server |
-
2020
- 2020-05-22 CN CN202010442115.1A patent/CN111737252B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7908448B1 (en) * | 2007-01-30 | 2011-03-15 | American Megatrends, Inc. | Maintaining data consistency in mirrored cluster storage systems with write-back cache |
| CN101820449A (en) * | 2010-04-20 | 2010-09-01 | 江苏电力调度通信中心 | Cross-safety zone application service isolation platform |
| US20130042156A1 (en) * | 2011-08-12 | 2013-02-14 | Oracle International Corporation | Utilizing multiple storage devices to reduce write latency for database logging |
| CN104615759A (en) * | 2015-02-13 | 2015-05-13 | 厦门雅迅网络股份有限公司 | Data synchronization method for different business system platforms |
| CN106844390A (en) * | 2015-12-07 | 2017-06-13 | 北京航天长峰科技工业集团有限公司 | A kind of inter-sectional data resource cut-in method |
| CN105718570A (en) * | 2016-01-20 | 2016-06-29 | 北京京东尚科信息技术有限公司 | Data migration method and device used for database |
| KR20180037571A (en) * | 2016-10-04 | 2018-04-12 | 삼성전자주식회사 | Method of data storage and operating methode of datacenter cluster caching system |
| US20190197027A1 (en) * | 2016-12-19 | 2019-06-27 | Tencent Technology (Shenzhen) Company Limited | Data management method and server |
| US20190007206A1 (en) * | 2017-06-30 | 2019-01-03 | Microsoft Technology Licensing, Llc | Encrypting object index in a distributed storage environment |
| CN108647277A (en) * | 2018-05-03 | 2018-10-12 | 山东师范大学 | A kind of mobile campus comprehensive service platform and its working method |
Non-Patent Citations (3)
| Title |
|---|
| S. BYAN等: "Mercury: Host-side flash caching for the data center", 《2012 IEEE 28TH SYMPOSIUM ON MASS STORAGE SYSTEMS AND TECHNOLOGIES (MSST)》, pages 1 - 12 * |
| 曹靖华: "P公司互联网支付实时风险控制的架构方案设计", 《中国优秀硕士学位论文全文数据库信息科技辑》, no. 7, pages 139 - 69 * |
| 睡不醒的天天: "网吧无盘技术：支持镜像独立回写的设置方法", pages 1, Retrieved from the Internet <URL:《https://jingyan.baidu.com/article/5553fa821730e965a3393456.html》> * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111737252B (en) | 2023-10-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11416344B2 (en) | Partial database restoration | |
| CN102917072B (en) | For carrying out the equipment of Data Migration, system and method between data server cluster | |
| CN110309218B (en) | Data exchange system and data writing method | |
| CN107766575B (en) | Read-write separation database access method and device | |
| CN110209726A (en) | Distributed experiment & measurement system system, method of data synchronization and storage medium | |
| CN109634970B (en) | Table data synchronization method, apparatus, storage medium and device | |
| US20110258160A1 (en) | Data synchronization methods for synchronizing data in communication system and communication systems | |
| CN102272751B (en) | Data integrity in a database environment through background synchronization | |
| CN113111129A (en) | Data synchronization method, device, equipment and storage medium | |
| CN111143368A (en) | Relational database data comparison method and system | |
| CN104346373A (en) | Partition log queue synchronization management method and device | |
| CN106709066B (en) | Data synchronization method and device | |
| CN110083579A (en) | Incremental data synchronous method, apparatus, computer equipment and computer storage medium | |
| JP4511047B2 (en) | Protocol for synchronizing parallel processors in a mobile communication system | |
| CN111581030A (en) | Data synchronization system and method based on difference data | |
| CN112000850B (en) | Method, device, system and equipment for processing data | |
| CN105574026A (en) | Method and device for service supporting by using non-relational database | |
| CN111046246B (en) | Label updating method and device and distributed storage system | |
| CN112417043A (en) | Data processing system and method | |
| CN110502584B (en) | Data synchronization method and device | |
| US6738791B2 (en) | Data synchronizing device | |
| CN111737252A (en) | Data fusion method and system based on data center | |
| CN111147226B (en) | Data storage method, device and storage medium | |
| CN115104295A (en) | Data processing method, data processing device, electronic device and storage medium | |
| CN114490570A (en) | Production data synchronization method and device, data synchronization system and server |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |