CN111737252B - Data fusion method and system based on data center - Google Patents
Data fusion method and system based on data center Download PDFInfo
- Publication number
- CN111737252B CN111737252B CN202010442115.1A CN202010442115A CN111737252B CN 111737252 B CN111737252 B CN 111737252B CN 202010442115 A CN202010442115 A CN 202010442115A CN 111737252 B CN111737252 B CN 111737252B
- Authority
- CN
- China
- Prior art keywords
- data
- mirror image
- write
- database
- data table
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2358—Change logging, detection, and notification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a data fusion method and a system based on a data center, wherein the method comprises the following steps: acquiring a data table in a service system used by a network hall, constructing a mirror image database in a data center, and generating a corresponding mirror image data table; the data center receives the write-back data request sent by the network hall, carries out write-back operation on the corresponding mirror image data table according to the write-back data, generates a corresponding write-back operation record, starts a trigger to implement data exchange based on a preset rule, and synchronizes the data of the mirror image database to a service system; after the data synchronization of the mirror image database is completed, the data center cleans, converts and stores corresponding data into a main database of the data center; and the network office hall acquires display data from the main database. The invention realizes data fusion by converting the mirror image database into the write-back data, maintains the consistency of the data, and truly realizes one-stop business, service and data.
Description
Technical Field
The invention relates to the technical field of data storage, in particular to a data fusion method and system based on a data center.
Background
The online office usually adopts a data exchange technology to realize the exchange of the data center and the service system, but the online office only allows the data to be acquired from the data center, cannot write back the data to the data center, and cannot directly acquire or write back the data from the service system to the service system database, as shown in fig. 1, mainly considering the real-time property that the data is difficult to be regularly unified and synchronized when the data is written back to the data center or the service system, so that the data errors or inconsistencies of the data table of the data center and the service system are caused. Therefore, the online office hall adopts the traditional mode to exchange data, so that data fusion is difficult to achieve, information, data, office, service and the like are split, one-stop office, service and data cannot be truly realized, and the user experience is greatly influenced.
Disclosure of Invention
The present invention aims to solve at least one of the technical problems existing in the prior art. Therefore, the invention provides a data fusion method based on a data center, which can write data back to the data center or the data center to realize data fusion.
The invention also provides a data center-based data fusion system using the data center-based data fusion method.
According to a first aspect of the present invention, a data center-based data fusion method includes: acquiring a data table in a service system used by a network hall, constructing a mirror image database in a data center, and generating a corresponding mirror image data table; the data center receives the write-back data request sent by the network hall, carries out write-back operation on the corresponding mirror image data table according to the write-back data, generates a corresponding write-back operation record, starts a trigger to implement data exchange based on a preset rule, and synchronizes the data of the mirror image database to a service system; after the data synchronization of the mirror image database is completed, the data center cleans, converts and stores corresponding data into a main database of the data center; the network office hall is connected with the main database, and the display data are obtained from the main database.
The data fusion method based on the data center has the following advantages: by constructing the mirror image database, the mirror image database is used for transferring and storing the write-back data into the data center and the service system, so that data fusion is realized, the consistency of the data is maintained, and one-stop transaction, service and data are realized in a real sense.
According to some embodiments of the invention, the method of data exchange comprises: a mirror database data table updating step, namely comparing the mirror data table with the structural rule information of the corresponding data table in the service system, and updating the mirror data table according to a comparison result; and a service system data table synchronization step, wherein the data of the mirror image data table is synchronized to the service system according to the write-back operation record. The data table of the mirror image library is consistent with the data table structure and data of the service system.
According to some embodiments of the invention, the mirror library data table updating step includes: a table structure detection step of acquiring a first table structure of a corresponding data table in the service system according to the mirror image data table, and judging whether a second table structure of the mirror image data table is consistent with the first table structure or not to obtain a table structure detection result; a rule detection step, namely acquiring a first reference integrity rule of a corresponding data table in the service system according to the mirror image data table, and judging whether a second reference integrity rule of the mirror image data table is consistent with the first reference integrity rule or not to obtain a rule detection result; and updating the table structure of the mirror image data table if at least one of the table structure detection result and the rule detection result is inconsistent. And maintaining the mirror image library table structure consistent with the corresponding data table structure in the service system according to the table structure definition and the reference integrity rule.
According to some embodiments of the present invention, the method for writing the write-back data into the mirror database and generating a corresponding write-back operation record includes: and adding corresponding operation marks in corresponding records of the mirror image data table according to the type of the write-back data request, and recording operation starting positions and operation numbers. And the data operation record of the mirror image library is recorded, but is not deleted in practice, so that the data can be conveniently synchronized with the service system according to the whole preset rule, and the operation is not frequently requested.
According to some embodiments of the invention, the method for synchronizing the data of the mirror database to the service system according to the write-back operation record comprises: and according to the operation marks of the mirror image data table, positioning the operation marks to the operation starting position in the corresponding data table in the service system, and triggering corresponding updating operation on the data of the number of the operation. And according to the operation record of the mirror image database, carrying out corresponding operation on the service system data table, completing the write-back operation on the service system data table, and ensuring that the data of the service system data table is consistent with the data in the mirror image database.
According to some embodiments of the invention, the method of building a mirror database comprises: extracting a data table used by the network hall from the service system, and acquiring the first table structure and the first reference integrity rule of the data table; and generating the corresponding mirror image data table in the mirror image database according to the first table structure and the first reference integrity rule, and storing the second table structure and the second reference integrity rule of the mirror image data table. The mirror image data table is generated in advance according to the table structure definition of the service data table and the reference integrity rule, which is beneficial to ensuring that the data table of the mirror image database is consistent with the corresponding data table structure in the service system, avoiding error when the data is written back subsequently, reducing the synchronous quantity of the mirror image table structure caused by the structural rule change of the service data table and improving the processing speed of the subsequent write back.
According to some embodiments of the invention, the method for obtaining the display data in the web-handling hall comprises the following steps: the network office hall obtains the display data by initiating a data request to the main database, or the main database pushes the display data to the network office hall in real time; the preset rule comprises the following steps: the trigger is activated at preset time intervals. The flexibility is improved by the network office to acquire the display data; the write-back triggers the timing operation, so that the problem that the mirror database and the service system work normally is influenced when the data of the mirror database and the service system are inconsistent for a long time and frequent synchronization is avoided.
According to a second aspect of the present invention, a data fusion system based on a data center comprises a business system, a web hall and a data center, wherein the data center comprises: the mirror image database is used for storing mirror image data tables corresponding to the data tables in the business system processed by the network hall; a main database for storing the mirror data table in synchronization with the service system; and the data management module is used for carrying out corresponding operation on the mirror image database according to the write-back data request of the network hall and synchronizing the data of the mirror image database to the service system according to a preset rule.
The data fusion system based on the data center has the following advantages: the mirror image database is used as write-back transfer, the network office can change service system data, ensure the data consistency of the data center and the service system, realize data fusion, keep the data consistency and truly realize one-stop business, service and data.
According to some embodiments of the invention, the data management module comprises: the write-back data processing unit is used for adding corresponding operation marks in the corresponding records of the mirror image data table according to the type of the write-back data request, and recording the operation starting positions and the operation numbers; the mirror image data table structure synchronizing unit is used for updating the corresponding mirror image data table based on a preset rule according to a first table structure and a first reference integrity definition of the data table of the service system; and the business system data synchronization unit is used for positioning the operation starting position in a corresponding data table in the business system according to the operation mark based on a preset rule and triggering corresponding updating operation on the data of the number of the operation. The mirror image database processes the write-back data and adds an operation mark in the corresponding record, so that the synchronization to a service system is facilitated; the mirror image data table structure synchronizing unit enables the data table of the mirror image library to be consistent with the data table of the service system in structure, and is convenient for writing back data to be synchronized to the service system.
According to some embodiments of the invention, the web-handling lobby comprises: the data acquisition module is used for generating write-back data and sending a write-back request to the data center; and the data application module is used for acquiring data from the main database and displaying the data. The data acquisition module collects write-back data and is connected with the mirror image database; the data application module is used for displaying data; the network hall processes the data orderly in a classified manner, so that the efficiency is improved, and the error occurrence rate is reduced.
Drawings
The foregoing and/or additional aspects and advantages of the invention will become apparent and may be better understood from the following description of embodiments taken in conjunction with the accompanying drawings in which:
FIG. 1 is a schematic diagram of a prior art data exchange for an online office hall;
FIG. 2 is a schematic diagram of steps of a method according to an embodiment of the present invention;
FIG. 3 is a schematic diagram illustrating a processing procedure of write-back data according to a method of an embodiment of the invention;
FIG. 4 is a block schematic diagram of a system of an embodiment of the present invention;
FIG. 5 is a schematic block diagram of a switching management module in a system according to an embodiment of the present invention;
fig. 6 is a schematic diagram of data interaction of a system according to an embodiment of the present invention.
Reference numerals:
a data center 100, a business system 200, and a web lobby 300;
mirror database 110, master database data 120, data management module 130;
write-back data processing unit 131, mirror data table structure synchronizing unit 132, business system data synchronizing unit 133;
a data acquisition module 310, a data application module 320, and a lobby database 330.
Detailed Description
Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to like or similar elements or elements having like or similar functions throughout. The embodiments described below by referring to the drawings are illustrative only and are not to be construed as limiting the invention.
In the description of the present invention, a number means one or more, a number means two or more, and greater than, less than, exceeding, etc. are understood to not include the present number, and above, below, within, etc. are understood to include the present number. The description of the first and second is for the purpose of distinguishing between technical features only and should not be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated or implicitly indicating the precedence of the technical features indicated.
Referring to fig. 2, the method of an embodiment of the present invention includes: acquiring a data table in a service system used by a network hall, constructing a mirror database (the mirror database is also called as a mirror database in the invention) in a data center, and generating a corresponding mirror data table; the data center receives a write-back data request sent by the network hall, writes the write-back data into the mirror image database, carries out write-back operation on the corresponding mirror image data table, generates a corresponding write-back operation record, starts a trigger to implement data exchange based on a preset rule, and synchronizes the data of the mirror image database to the service system; after the data of the mirror image database is synchronized to the service system, the data center cleans, converts and stores the corresponding data into a main database of the data center; the network office is connected with the main database, and the display data are obtained from the main database. Wherein the write back operation records include, but are not limited to: operation time, operation type, operation start position, operation number, and the like.
The data exchange process of the method of the embodiment of the invention comprises the following steps: updating the mirror image database data table and synchronizing the service system data table. In the step of updating the mirror database data table, the structural rule information of the mirror database data table and the corresponding data table in the service system is compared, and the mirror database data table is updated according to the comparison result, including: and (3) detecting a table structure, detecting rules and updating. In the table structure detection step, a first table structure of a data table corresponding to the service system is obtained according to the mirror image data table, and whether a second table structure of the mirror image data table is consistent with the first table structure is judged to obtain a table structure detection result. For example: the data table A '(called mirror table A for short) in the mirror image library is updated with data, and the mirror image table A' is the mirror image of the data table A in the service system; when data in the mirror image table A 'is synchronized to the data table A, firstly judging whether the table structure of the data table A is consistent with the table structure of the mirror image table A' or not to obtain a table structure detection result; if not, an update to the mirror table A' is required. In the rule detection step, a first reference integrity rule of a corresponding data table in the service system is obtained according to the mirror image data table, and whether a second reference integrity rule of the mirror image data table is consistent with the first reference integrity rule is judged, so that a rule detection result is obtained. For example: the mirror image table A 'is a mirror image of the data table A in the service system, and whether the reference integrity of the mirror image table A' is consistent with the reference integrity of the data table A or not is checked and judged; if the table structures of the mirror table A 'and the mirror table related to the referential rule thereof are inconsistent, the table structures of the mirror table A' and the mirror table related to the referential rule thereof need to be updated. And the updating step judges whether the table structure detection result and the rule detection result are consistent, and if at least one item is inconsistent, the table structure of the corresponding mirror image data table in the mirror image library is updated. In the step of synchronizing the service system data table, the data of the mirror image library is synchronized to the service system according to the write-back operation record.
In some embodiments of the present invention, the second table structure of the mirror table found in the table structure detection step is inconsistent with the corresponding first table structure, i.e. an update to the mirror table is triggered. And then continuously checking whether the reference integrity rule of the mirror image table, including the update rule, the deletion rule and the insertion rule, is consistent with the reference integrity rule of the corresponding data table in the database of the service system, and if not, triggering updating of all table structures of the mirror image database and the mirror image table.
The write-back data processing procedure according to one embodiment of the present invention, referring to fig. 3, includes: the data center detects that the network hall has a write-back data request, a data exchange strategy is triggered, write-back data into the mirror image database, a trigger is started to implement the data exchange strategy according to a preset time interval, and mirror image database table data are synchronously updated to the service system database. The preset time interval can be set to be real-time and specific in minutes according to application requirements. When the write-back data is written into the mirror database, the type of the write-back data request, such as adding, inserting and deleting, is detected, and operation marks such as adding, inserting and deleting are added behind the corresponding records of the mirror database table, and the operation starting position and the operation number are recorded. In data exchange, it is first detected whether the table structure of the mirror database table is consistent with the table structure of the service system database table. If the two types of the images are inconsistent, triggering the updating of the table structure of the mirror image library; if so, detecting whether the reference integrity rule (including updating rule, deleting rule and inserting rule) of the mirror database table is consistent with the reference integrity rule of the service system database table, and if not, triggering the mirror database updating table structure. And if the table structure is consistent with the reference integrity rule, updating the data table data of the new service system according to the write-back operation record. The write operation record (including, but not limited to, operation time, operation type, operation starting position and operation number) is read and retrieved, the operation starting position is located in the corresponding data table in the service system according to the operation mark recorded in the mirror image data table, corresponding update operation (such as adding, inserting, deleting and the like) is triggered on the data of the operation number, and the data in the mirror image data table is synchronized to the service system. In other embodiments of the present invention, the mirror data table is directly read, and the corresponding data table in the service system is updated according to the operation mark.
The system of the embodiment of the invention comprises: data center 100, business system 200, and web-office hall 300. Referring to fig. 4, wherein the data center 100 includes: the mirror image database 110 is used for storing mirror image data tables corresponding to the data tables in the service system processed by the network hall; a master database 120 for storing a mirror data table that maintains synchronization with the service system; the data management module 130 is configured to perform corresponding operations on the mirror database according to the write-back data request of the network hall, and synchronize data of the mirror database to the service system according to a preset rule. In some embodiments of the present invention, the preset rule is to synchronize data to the service system according to a preset time interval; in some embodiments of the present invention, the preset rule is to synchronize data to the service system according to the write-back data amount; in other embodiments of the present invention, the preset rule is to synchronize data to the service system according to the operation record amount of the mirror data table.
Referring to fig. 5, the data management module 130 includes: write-back data processing unit 131, mirror data table structure synchronization unit 132, and business system data synchronization unit 133. The write-back data processing unit 131 is configured to receive the write-back data request, perform corresponding processing on the mirror image data table according to the type of the write-back data request, add relevant operation marks at corresponding records in the mirror image data table, and record an operation start position and an operation number. For example, after the new data is recorded in the mirror image data table, the new mark is added at the recorded position, the operation position and the operation number are recorded, and the insertion of the update data is similar; and deleting the data, namely searching the corresponding record, adding a deleting mark, recording the operation position and the operation number, and not performing mirror image library record deleting processing. It should be understood that the mirror database records are not deleted, but only deleted, so that the records in the mirror database can be asynchronously synchronized into the service system according to the preset rule. The mirror data table structure synchronization unit 132 is configured to update the corresponding mirror data table according to the first table structure and the first reference integrity definition of the data table of the service system based on the preset rule. The service system data synchronization unit 133 is configured to read an operation record (including, but not limited to, an operation time, an operation type, an operation start position, and an operation number) of the mirror image table based on a preset rule, locate, according to an operation flag recorded in the mirror image, the operation start position in a corresponding data table in the service system, and trigger a corresponding update operation for data of the same operand number in the record. In other embodiments of the present invention, the service system data synchronization unit 133 is configured to directly read the mirror data table based on a preset rule, and update the corresponding data table in the service system according to the operation mark.
Referring to fig. 4, a web-handling hall 300 in a system according to an embodiment of the present invention includes: the data acquisition module 310 is configured to generate write-back data, and send a write-back request to the data center; the data application module 320 is configured to obtain data from the main database and display the data to the user. In some systems of the present invention, referring to FIG. 6, the web-handling lobby 300 also includes a lobby database 330 for storing data that is received and processed during operation of the web-handling lobby 300.
Referring to fig. 6, a schematic diagram of data interaction after implementation of system deployment according to an embodiment of the present invention is shown. When the method is deployed, firstly, the mirror image database 110 is deployed in the data center, a data table used by the network hall is extracted from the service system, a first table structure and a first reference integrity rule of the data table are obtained, then a corresponding mirror image data table is generated in the mirror image database 110 according to the first table structure and the first reference integrity rule, and a second table structure and a second reference integrity of the mirror image data table are stored. Obviously, when the mirror database 110 is initially created, the second table structure of the mirror table is the same as the first table structure of the corresponding data table in the service system, and the second reference integrity rule of the mirror table is also the same as the first reference integrity rule of the corresponding data table in the service system. The web-office hall 300 obtains data from the main database 120 through the data application module 320, wherein the method for obtaining the data comprises the following steps: the data request is directly initiated to the main database 120 to obtain the display data, or the main database 120 is pushed to the web office hall 300 in real time, the hall database 330 of the web office hall 300 receives the push data, and the push data is read and displayed from the hall database 330 by the data application module 320. The data involved in the operation of the web-office hall 300 is stored in the hall database 330, and the data acquisition module 310 acquires write-back data from the hall database 330 and sends a write-back request to the mirror database 110. The mirror database 110 of the data center 100 receives the write-back request, writes the write-back data into the mirror database, starts data exchange according to a preset rule, and synchronizes the mirror data into the service database in the service system 200. After the mirror image data synchronization is completed, the data in the mirror image database is cleaned, converted and stored into the main database. In other embodiments of the present invention, the data acquisition module 310 obtains write-back data directly from the data application module 320. It should be appreciated that in the embodiment of the present invention, if the office hall 300 uses data tables in service databases in more than one service system, a mirror database of corresponding data may be generated according to the service system correspondence.
Although specific embodiments are described herein, those of ordinary skill in the art will recognize that many other modifications or alternative embodiments are also within the scope of the present disclosure. For example, any of the functions and/or processing capabilities described in connection with a particular device or component may be performed by any other device or component. In addition, while various exemplary implementations and architectures have been described in terms of embodiments of the present disclosure, those of ordinary skill in the art will recognize that many other modifications to the exemplary implementations and architectures described herein are also within the scope of the present disclosure.
Certain aspects of the present disclosure are described above with reference to block diagrams and flowchart illustrations of systems, methods, apparatuses and/or computer program products according to example embodiments. It will be understood that one or more blocks of the block diagrams and flowchart illustrations, and combinations of blocks in the block diagrams and flowchart illustrations, respectively, can be implemented by executing computer-executable program instructions. Also, some of the blocks in the block diagrams and flowcharts may not need to be performed in the order shown, or may not need to be performed in their entirety, according to some embodiments. In addition, additional components and/or operations beyond those shown in blocks of the block diagrams and flowcharts may be present in some embodiments.
Accordingly, blocks of the block diagrams and flowchart illustrations support combinations of means for performing the specified functions, combinations of elements or steps for performing the specified functions and program instruction means for performing the specified functions. It will also be understood that each block of the block diagrams and flowchart illustrations, and combinations of blocks in the block diagrams and flowchart illustrations, can be implemented by special purpose hardware-based computer systems that perform the specified functions, elements or steps, or combinations of special purpose hardware and computer instructions.
Program modules, applications, etc. described herein may include one or more software components including, for example, software objects, methods, data structures, etc. Each such software component may include computer-executable instructions that, in response to execution, cause at least a portion of the functions described herein (e.g., one or more operations of the exemplary methods described herein) to be performed. The software components may be encoded in any of a variety of programming languages.
The embodiments of the present invention have been described in detail with reference to the accompanying drawings, but the present invention is not limited to the above embodiments, and various changes can be made within the knowledge of one of ordinary skill in the art without departing from the spirit of the present invention.
Claims (9)
1. A data center-based data fusion method, comprising:
acquiring a data table in a service system used by a network hall, constructing a mirror image database in a data center, and generating a corresponding mirror image data table;
the data center receives the write-back data request sent by the network hall, carries out write-back operation on the corresponding mirror image data table according to the write-back data, generates a corresponding write-back operation record, starts a trigger to implement data exchange based on a preset rule, and synchronizes the data of the mirror image database to a service system;
after the data synchronization of the mirror image database is completed, the data center cleans, converts and stores corresponding data into a main database of the data center;
the network office hall is connected with the main database, and display data are obtained from the main database;
the data exchange method comprises the following steps:
a mirror database data table updating step, namely comparing the mirror data table with the structural rule information of the corresponding data table in the service system, and updating the mirror data table according to a comparison result;
and a service system data table synchronization step, wherein the data of the mirror image data table is synchronized to the service system according to the write-back operation record.
2. The data center-based data fusion method of claim 1, wherein the mirroring data table updating step comprises:
a table structure detection step of acquiring a first table structure of a corresponding data table in the service system according to the mirror image data table, and judging whether a second table structure of the mirror image data table is consistent with the first table structure or not to obtain a table structure detection result;
a rule detection step, namely acquiring a first reference integrity rule of a corresponding data table in the service system according to the mirror image data table, and judging whether a second reference integrity rule of the mirror image data table is consistent with the first reference integrity rule or not to obtain a rule detection result;
and updating the table structure of the corresponding mirror image data table in the mirror image database if at least one of the table structure detection result and the rule detection result is inconsistent.
3. The data center-based data fusion method of claim 1, wherein the method of writing the write-back data to the mirror database to generate the corresponding write-back operation record comprises:
and adding corresponding operation marks in corresponding records of the mirror image data table according to the type of the write-back data request, and recording operation starting positions and operation numbers.
4. A data center-based data fusion method according to claim 3, wherein the method of synchronizing the data of the mirror database to the business system according to the write-back operation record comprises:
and according to the operation marks of the mirror image data table, positioning the operation marks to the operation starting position in the corresponding data table in the service system, and triggering corresponding updating operation on the data of the number of the operation.
5. The data center-based data fusion method of claim 2, wherein the method of constructing a mirror database comprises:
extracting a data table used by the network hall from the service system, and acquiring the first table structure and a first reference integrity rule of the data table;
and generating the corresponding mirror image data table in the mirror image database according to the first table structure and the first reference integrity rule, and storing the second table structure and the second reference integrity rule of the mirror image data table.
6. The data center-based data fusion method of claim 1, wherein the method for acquiring presentation data in the web lobby comprises: the network office hall obtains the display data by initiating a data request to the main database, or the main database pushes the display data to the network office hall in real time; the preset rule comprises the following steps: the trigger is activated at preset time intervals.
7. A data fusion system based on a data center, using the method of any one of claims 1 to 6, comprising a business system, a web-office hall and a data center, characterized in that the data center comprises:
the mirror image database is used for storing mirror image data tables corresponding to the data tables in the business system processed by the network hall;
a main database for storing the mirror data table in synchronization with the service system;
and the data management module is used for carrying out corresponding operation on the mirror image database according to the write-back data request of the network hall and synchronizing the data of the mirror image database to the service system according to a preset rule.
8. The data center-based data fusion system of claim 7, wherein the data management module comprises:
the write-back data processing unit is used for adding corresponding operation marks in the corresponding records of the mirror image data table according to the type of the write-back data request, and recording the operation starting positions and the operation numbers;
the mirror image data table structure synchronizing unit is used for updating the corresponding mirror image data table based on a preset rule according to a first table structure and a first reference integrity definition of the data table of the service system;
and the business system data synchronization unit is used for positioning the operation starting position in a corresponding data table in the business system according to the operation mark based on a preset rule and triggering corresponding updating operation on the data of the number of the operation.
9. The data center-based data fusion system of claim 8, wherein the web-handling lobby comprises:
the data acquisition module is used for generating write-back data and sending a write-back request to the data center;
and the data application module is used for acquiring data from the main database and displaying the data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010442115.1A CN111737252B (en) | 2020-05-22 | 2020-05-22 | Data fusion method and system based on data center |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010442115.1A CN111737252B (en) | 2020-05-22 | 2020-05-22 | Data fusion method and system based on data center |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111737252A CN111737252A (en) | 2020-10-02 |
| CN111737252B true CN111737252B (en) | 2023-10-03 |
Family
ID=72647627
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010442115.1A Active CN111737252B (en) | 2020-05-22 | 2020-05-22 | Data fusion method and system based on data center |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111737252B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101820449A (en) * | 2010-04-20 | 2010-09-01 | 江苏电力调度通信中心 | Cross-safety zone application service isolation platform |
| US7908448B1 (en) * | 2007-01-30 | 2011-03-15 | American Megatrends, Inc. | Maintaining data consistency in mirrored cluster storage systems with write-back cache |
| CN104615759A (en) * | 2015-02-13 | 2015-05-13 | 厦门雅迅网络股份有限公司 | Data synchronization method for different business system platforms |
| CN105718570A (en) * | 2016-01-20 | 2016-06-29 | 北京京东尚科信息技术有限公司 | Data migration method and device used for database |
| CN106844390A (en) * | 2015-12-07 | 2017-06-13 | 北京航天长峰科技工业集团有限公司 | A kind of inter-sectional data resource cut-in method |
| KR20180037571A (en) * | 2016-10-04 | 2018-04-12 | 삼성전자주식회사 | Method of data storage and operating methode of datacenter cluster caching system |
| CN108647277A (en) * | 2018-05-03 | 2018-10-12 | 山东师范大学 | A kind of mobile campus comprehensive service platform and its working method |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8909996B2 (en) * | 2011-08-12 | 2014-12-09 | Oracle International Corporation | Utilizing multiple storage devices to reduce write latency for database logging |
| WO2018113580A1 (en) * | 2016-12-19 | 2018-06-28 | 腾讯科技（深圳）有限公司 | Data management method and server |
| US10764045B2 (en) * | 2017-06-30 | 2020-09-01 | Microsoft Technology Licensing, Llc | Encrypting object index in a distributed storage environment |
-
2020
- 2020-05-22 CN CN202010442115.1A patent/CN111737252B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7908448B1 (en) * | 2007-01-30 | 2011-03-15 | American Megatrends, Inc. | Maintaining data consistency in mirrored cluster storage systems with write-back cache |
| CN101820449A (en) * | 2010-04-20 | 2010-09-01 | 江苏电力调度通信中心 | Cross-safety zone application service isolation platform |
| CN104615759A (en) * | 2015-02-13 | 2015-05-13 | 厦门雅迅网络股份有限公司 | Data synchronization method for different business system platforms |
| CN106844390A (en) * | 2015-12-07 | 2017-06-13 | 北京航天长峰科技工业集团有限公司 | A kind of inter-sectional data resource cut-in method |
| CN105718570A (en) * | 2016-01-20 | 2016-06-29 | 北京京东尚科信息技术有限公司 | Data migration method and device used for database |
| KR20180037571A (en) * | 2016-10-04 | 2018-04-12 | 삼성전자주식회사 | Method of data storage and operating methode of datacenter cluster caching system |
| CN108647277A (en) * | 2018-05-03 | 2018-10-12 | 山东师范大学 | A kind of mobile campus comprehensive service platform and its working method |
Non-Patent Citations (2)
| Title |
|---|
| Mercury: Host-side flash caching for the data center;S. Byan等;《2012 IEEE 28th Symposium on Mass Storage Systems and Technologies (MSST)》;第1-12页 * |
| P公司互联网支付实时风险控制的架构方案设计;曹靖华;《中国优秀硕士学位论文全文数据库信息科技辑》(第7期);第I139-69页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111737252A (en) | 2020-10-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110209726A (en) | Distributed experiment & measurement system system, method of data synchronization and storage medium | |
| CN110347746B (en) | Heterogeneous database synchronous data consistency checking method and device | |
| US20120254120A1 (en) | Logging system using persistent memory | |
| EP2429134A1 (en) | Method and apparatus for checking and synchronizing data block in distributed file system | |
| CN106503158B (en) | Data synchronization method and device | |
| CN109933632B (en) | Data migration method, device and equipment for database | |
| EP1952283A2 (en) | Apparatus and method for creating a real time database replica | |
| CN101231652B (en) | Method and system for data storage resynchronization | |
| US20110137874A1 (en) | Methods to Minimize Communication in a Cluster Database System | |
| CN105574187A (en) | Duplication transaction consistency guaranteeing method and system for heterogeneous databases | |
| CN102272751A (en) | Data integrity in a database environment through background synchronization | |
| CN111858181B (en) | Method and tool for realizing mirror image synchronization of cross-region container | |
| CN115617571A (en) | Data backup method, device, system, equipment and storage medium | |
| US20230315713A1 (en) | Operation request processing method, apparatus, device, readable storage medium, and system | |
| CN110895547A (en) | Multi-source heterogeneous database data synchronization system and method based on DB2 federal characteristics | |
| CN111737252B (en) | Data fusion method and system based on data center | |
| CN117591345A (en) | Ceph-specific data synchronization method, device, equipment and storage medium | |
| CN112417043A (en) | Data processing system and method | |
| CN110502584B (en) | Data synchronization method and device | |
| WO2021189283A1 (en) | Method and device for data processing, electronic device, and storage medium | |
| CN113742420B (en) | Data synchronization method and device | |
| CN114490570A (en) | Production data synchronization method and device, data synchronization system and server | |
| CN110879812B (en) | Spark-based data synchronization method in e-commerce platform | |
| CN114281833A (en) | Method for rapidly keeping consistency of slave computer and host computer data under master-slave copy mode of OpenGauss database | |
| CN113190531A (en) | Database migration method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |