CN107357791B - Data sharing method and device - Google Patents
Data sharing method and device Download PDFInfo
- Publication number
- CN107357791B CN107357791B CN201610305168.2A CN201610305168A CN107357791B CN 107357791 B CN107357791 B CN 107357791B CN 201610305168 A CN201610305168 A CN 201610305168A CN 107357791 B CN107357791 B CN 107357791B
- Authority
- CN
- China
- Prior art keywords
- data
- application system
- standard
- incremental
- sending
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/445—Program loading or initiating
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/176—Support for shared access to files; File sharing support
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a data sharing method and a device, wherein the method comprises the following steps: acquiring incremental data of two adjacent backup data of a first application system; performing data processing on the incremental data to obtain standard data in a preset format; the standard data is sent to a second application system associated with the first application system. The invention realizes the data butt joint among all the distributed application systems, really integrates the information among all the systems to carry out the information interaction among the systems, ensures the accuracy of the data interaction, solves the problem of data synchronization under the distributed environment, and ensures that the change of the original system can be reduced to the minimum, thereby being capable of realizing the rapid implementation and generating the benefit.
Description
Technical Field
The present invention relates to the field of databases, and in particular, to a data sharing method and apparatus.
Background
In the process of enterprise information construction, because unified planning is lacked in the initial construction stage, most application systems are developed and implemented from the perspectives of departments and base levels, and global consideration is lacked, so that the developed application systems of all departments have no unified data standard, the data formats are different, and effective data sharing cannot be realized among the systems; the same type of data exists in a plurality of systems, and data synchronization cannot be realized among the systems; in the process of realizing data unification, certain workload is increased on the use and maintenance of information isolated islands, and unnecessary resource waste is caused, so that automatic data synchronization among systems is realized, information among the systems is really integrated to carry out information interaction among the systems and keep data consistency, and the method is a key problem to be solved in enterprise informatization construction.
One conventional way to keep data synchronized is an Enterprise Application Integration (EAI) implementation manner, which requires that each system provides a corresponding data interface for the system interacting with the system, and thus the development workload is large; and each application developer needs to coordinate the discussion work of the interface specification, the interface dependency is high, and the realization period is uncontrollable. Therefore, how to solve the problem of data synchronization in a distributed environment becomes an urgent problem to be solved in the data synchronization process of the cloud computing data center.
Disclosure of Invention
The invention provides a data sharing method and a data sharing device, and aims to solve the problem of data synchronization in a distributed environment.
In order to achieve the above object, an embodiment of the present invention provides a data sharing method, including:
acquiring incremental data of two adjacent backup data of a first application system;
performing data processing on the incremental data to obtain standard data in a preset format;
the standard data is sent to a second application system associated with the first application system.
Preferably, the step of obtaining incremental data of two adjacent backup data of the first application system comprises:
and acquiring newly added data and/or changed data in the two adjacent backup data of the first application system, wherein the later backup data is compared with the former backup data.
Preferably, the step of acquiring newly added data and/or changed data in two adjacent backup data of the first application system, where the data of the later backup is compared with the data of the earlier backup, includes:
copying source data in the first application system to a preset materialized view of the first application system at regular time according to a first preset period, and updating a materialized view log of the materialized view;
acquiring newly added data and/or changed data in the two adjacent backup data of the first application system according to the materialized view log, wherein the later backup data is compared with the former backup data;
and establishing an increment table of the first application system, and storing the increment data in the increment table.
Preferably, the step of performing data processing on the incremental data to obtain standard data in a preset format includes:
performing data format conversion on the incremental data in the incremental table to obtain standard data in a preset format;
storing the standard data in a theme library, and recording the time of data format conversion in the materialized view log.
Preferably, after the step of storing the standard data in the subject library, the method further comprises:
incremental data corresponding to the standard data is stored in a topic library.
Preferably, the step of sending the standard data to a second application system associated with the first application system comprises:
when a target query object in a session number in a received data query request of a second application system is a first application system, sending standard data to the second application system associated with the first application system through the session number; or
Generating a session number of a data synchronization request at regular time according to a first preset period, and sending standard data to a second application system associated with a first application system according to a preset corresponding relation between the first application system and the second application system; and the target synchronization object in the session number of the data synchronization request is a second application system associated with the first application system through the session number.
Preferably, after the step of sending the criteria data to a second application system associated with the first application system, the method further comprises:
the increment table is cleared.
In order to achieve the above object, an embodiment of the present invention further provides a data sharing apparatus, including:
the acquisition module is used for acquiring incremental data of the adjacent two backup data of the first application system;
the processing module is used for carrying out data processing on the incremental data to obtain standard data in a preset format;
and the sending module is used for sending the standard data to a second application system associated with the first application system.
Preferably, the acquisition module comprises:
and the obtaining submodule is used for obtaining newly added data and/or changed data in the two adjacent backup data of the first application system, wherein the data of the later backup is compared with the data of the former backup.
Preferably, the acquisition submodule includes:
the replication unit is used for periodically replicating the source data in the first application system to a preset materialized view of the first application system according to a first preset period and updating a materialized view log of the materialized view;
the acquiring unit is used for acquiring newly added data and/or changed data in the two adjacent backup data of the first application system according to the materialized view log, wherein the later backup data is compared with the former backup data;
and the storage unit is used for establishing an increment table of the first application system and storing the increment data in the increment table.
Preferably, the processing module comprises:
the conversion submodule is used for carrying out data format conversion on the incremental data in the incremental table to obtain standard data in a preset format;
and the recording submodule is used for storing the standard data in the theme library and recording the time of data format conversion in the materialized view log.
Preferably, the processing module further comprises:
and the storage submodule is used for storing the incremental data corresponding to the standard data in the theme library.
Preferably, the transmission module includes:
the first sending sub-module is used for sending the standard data to a second application system associated with the first application system through a session number when a target query object in the session number in a received data query request of the second application system is the first application system; or
The second sending submodule is used for regularly generating a session number of the data synchronization request according to a first preset period and sending the standard data to a second application system associated with the first application system according to a preset corresponding relation between the first application system and the second application system; and the target synchronization object in the session number of the data synchronization request is a second application system associated with the first application system through the session number.
Preferably, the apparatus further comprises:
a clearing module for clearing the standard data after the sending module sends the standard data to a second application system associated with the first application system,
the increment table is cleared.
The scheme of the invention at least comprises the following beneficial effects:
according to the data sharing method and device, the incremental data are obtained and the incremental table is established for the incremental data, and the change of the data of the first application system is firstly reflected in the incremental table; converting the data format of the incremental data to form standard data with a uniform format so as to overcome the problem of format difference among different application systems; establishing a subject library for the standard data, and issuing the subject library to a second associated application system to improve the real-time performance of data sharing; the invention realizes the data butt joint among all the distributed application systems, really integrates the information among all the systems to carry out the information interaction among the systems and ensures the accuracy of the data interaction; when data support service is provided for a new application system according to business requirements, only standard data in the embodiment of the invention needs to be acquired. The invention solves the problem of data synchronization in a distributed environment, and can minimize the change of the original system, thereby being capable of realizing rapid implementation and generating benefits.
Drawings
Fig. 1 is a flow chart illustrating basic steps of a data sharing method according to a first embodiment of the present invention;
FIG. 2 shows one of the scenarios for a second embodiment of the present invention;
FIG. 3 is a second exemplary scenario diagram of the second embodiment of the present invention;
fig. 4 is a schematic structural diagram of a shared data center according to a second embodiment of the present invention;
FIG. 5 is a flowchart illustrating the basic steps of accessing a shared data center by an application system or user according to a second embodiment of the present invention;
fig. 6 is a schematic structural diagram of a data sharing apparatus according to a third embodiment of the present invention.
Detailed Description
In order to make the technical problems, technical solutions and advantages of the present invention more apparent, the following detailed description is given with reference to the accompanying drawings and specific embodiments.
The invention provides a data sharing method and a data sharing device aiming at the existing problems.
First embodiment
Referring to fig. 1, a first embodiment of the present invention provides a data sharing method, including:
The incremental data comprises newly added data and/or changed data. Data sharing needs to be realized by uniformly managing data of each application system, and in order to reduce data redundancy, incremental data in two adjacent backup data of a first application system only needs to be acquired.
And 102, performing data processing on the incremental data to obtain standard data in a preset format.
The standard data is a unified model established for a data warehouse, the conversion of the data from a service model to an analysis model is realized through library functions, custom scripts and other modes, and the obtained final result is data which is externally provided and accords with the specification; because the distributed application systems have no uniform data standard and different data formats, effective data sharing cannot be realized among the systems, and in order to facilitate uniform management, the data of the application systems are subjected to data processing and converted into data with a uniform format.
The method comprises the steps that standard data corresponding to incremental data of a first application system are sent to a second application system associated with the first application system, so that data connection between the standard data and distributed application systems is achieved, information among the systems is really integrated to carry out information interaction among the systems, and accuracy of data interaction is guaranteed; when data support service is provided for a new application system according to business requirements, only standard data in the embodiment of the invention needs to be acquired.
Specifically, step 101 includes:
and acquiring newly added data and/or changed data in the two adjacent backup data of the first application system, wherein the later backup data is compared with the former backup data.
Wherein, compared with the data backed up before, the data backed up after and/or the data changed, namely the incremental data, are required to be acquired; and the changed data is the data obtained after the time change and needing to be synchronized with the associated application system, namely the data backed up later.
Preferably, the step of acquiring newly added data and/or changed data in two adjacent backup data of the first application system, where the data of the later backup is compared with the data of the earlier backup, includes:
the method comprises the steps of firstly, copying source data in a first application system to a preset materialized view of the first application system at regular time according to a first preset period, and updating a materialized view log of the materialized view.
The materialized view is a local copy of the remote data, the source data in the first application system are copied into the preset materialized view at regular time according to a first preset period, and the time of copying the source data each time is recorded into a materialized view log, so that unified management and data query can be conveniently carried out on the distributed application system.
And secondly, acquiring newly added data and/or changed data in the two adjacent backup data of the first application system compared with the previously backed up data in the materialized view log.
In the process of obtaining the incremental data, the incremental data in the two adjacent backup data is obtained by inquiring the time record of copying the source data in the materialized view log each time.
And thirdly, establishing an increment table of the first application system, and storing increment data in the increment table.
And establishing an increment table for the first application system, wherein the increment table is used for temporarily storing increment data.
Specifically, step 102 includes:
firstly, performing data format conversion on the incremental data in the incremental table to obtain standard data in a preset format.
And converting the data format of the increment data in the increment table to obtain standard data in a uniform format.
And secondly, storing the standard data in a theme library, and recording the time of data format conversion in a materialized view log.
The standard data are stored in the subject database so as to be convenient for uniformly storing, managing and maintaining the standard data corresponding to the incremental data of each distributed application system. The theme library stores standard data in a uniform format, and is used for issuing the standard data to be synchronized to other application systems, and realizing data docking with each distributed application system through the theme library, so that the accuracy of data interaction is ensured.
Further, the step of storing the standard data in the subject library in the second step may be followed by:
incremental data corresponding to the standard data is stored in a topic library.
The incremental data are stored in a subject library, and it needs to be explained that the incremental data and the standard data need to be stored correspondingly; the incremental data and the standard data are equivalent to stubs of interactive data, so that query and management are facilitated, and the accuracy and uniqueness of the data among various systems are ensured.
Preferably, step 103 comprises:
when a target query object in a session number in a received data query request of a second application system is a first application system, sending standard data to the second application system associated with the first application system through the session number;
when a data query request of a second application system is received, the data query request carries a session number, the session number comprises a target query object, and when the target query object of the second application system is the first application system, standard data is sent to the second application system.
Or
Generating a session number of a data synchronization request at regular time according to a first preset period, and sending standard data to a second application system associated with a first application system according to a preset corresponding relation between the first application system and the second application system; and the target synchronization object in the session number of the data synchronization request is a second application system associated with the first application system through the session number.
And the second application system is used as a target synchronization object in the session number of the data synchronization request according to the preset corresponding relation between the application systems, and the standard data is sent to the second application system.
It should be noted that, before the second application system for transmitting the standard data, the data format of the standard data needs to be converted according to the data format of the second application system.
Preferably, after step 103, the method further comprises:
the increment table is cleared.
After the data synchronization is completed, the incremental data are invalid, and the memory is cleared in time so as to be convenient to clean.
In the embodiment of the invention, the change of the first application system data is firstly reflected in the increment table by acquiring the increment data and establishing the increment table for the increment data; converting the data format of the incremental data to form standard data with a uniform format so as to overcome the problem of format difference among different application systems; establishing a subject library for the standard data, and issuing the subject library to a second associated application system to improve the real-time performance of data sharing; the invention realizes the data butt joint among all the distributed application systems, really integrates the information among all the systems to carry out the information interaction among the systems and ensures the accuracy of the data interaction; when data support service is provided for a new application system according to business requirements, only standard data in the embodiment of the invention needs to be acquired. The invention solves the problem of data synchronization in a distributed environment, and can minimize the change of the original system, thereby being capable of realizing rapid implementation and generating benefits.
Second embodiment
Referring to fig. 2, a second embodiment provides an embodiment of an Enterprise Service Bus (ESB) based shared data center. The shared data center comprises a Web service layer, an enterprise service bus, a service layer, a display layer, a logic layer, a persistence layer and a shared database;
the Web service layer provides access data service for other application systems, and comprises some operations for data synchronization of the shared data center; the presentation layer completes management of the shared data center, which includes functions of metadata management, system daily maintenance and the like.
The shared database comprises two parts of data, a stub of interactive data and metadata of a shared data center. The interactive data stub is mainly used for ensuring the accuracy and uniqueness of data among various systems, the data defines and manages a shared data center model, the normal operation of the shared data center is ensured, and the shared data center does not provide functions of inquiring and modifying the data and the like due to the consideration of safety, so that the problems of various systems caused by abnormal data are avoided.
Referring to fig. 3, the main function of the shared data center is to implement data sharing, i.e. synchronization and exchange of data. The shared data center copies the data of the application systems to the data exchange area, and stores the data in the exchange area after cleaning and converting the data in the theme library, and then distributes the data to each application system. The shared data center is divided into two main parts: a data exchange area and a theme library.
A data exchange area
The shared data center creates a database user for each application system, the resource owned by the user is called a data exchange area, and the following types of objects exist under the user:
(1) the method comprises the steps of creating a materialized view which is a local copy of remote data in a data exchange area according to a table which needs to be imported by an application system database, and refreshing the views in an ESB bus at regular time according to needs, for example, starting a regular refreshing task in a QuartZ (task scheduling framework, which is specially used for scheduling concurrent tasks).
(2) The materialized view log can be established to uniquely identify one physical address (ROWID) or primary key (PRIMARY KEY) type according to the requirements of quick refresh of different materialized views, and can also select whether to include a list of specified columns and the like.
(3) Before data cleaning, the increment table queries data which change from the last cleaning to the current time according to the materialized view log, and then stores the data into the increment table to wait for cleaning.
(4) And when the data of the application system is cleaned, the data is stored in the theme base, corresponding records are also distributed to the data to be transmitted corresponding to other application systems, and the data of the tables are transmitted to the corresponding application systems by the transmission task.
Second, theme library
The subject library is a core area of the shared data center, and stores data after the application system is cleaned, wherein the cleaning in the embodiment of the invention can also be understood as conversion, namely, the conversion of the data from a business model to an analysis model is realized by a library function, a custom script and the like aiming at a model established by a data warehouse, and the obtained final result is data which is provided externally and accords with the specification, namely the final shared data center data.
With continued reference to fig. 2 and 3, a series of data import (or data replication), flush, and dispatch actions of the shared data center are scheduled through the ESB bus, and include four main steps:
the method comprises the steps of firstly, copying application system data, and importing data (namely increment data) which is newly added or changed by the application system into a corresponding increment table of a shared data center data exchange area, wherein the step can be finished by calling a shared data center Web-Service through the application system, and can also be obtained from a materialized view by starting a timing task through QuartZ of the shared data center.
Step two, data cleaning, which is a process of transferring temporary data storage to a subject library, and at this stage, the storage process is called by a Unified Message Object (UMO) of the ESB to realize the data cleaning and storing process, so that the module mainly generates a data cleaning and storing process of transferring temporary data storage to central data storage.
Specifically, there are two ways to start a data cleaning task, one is to configure the cleaning task as OutBound, after data is imported by WebService or QuarZ task, the ESB bus immediately triggers the corresponding data packet to go out to the management module (OutBound) for data cleaning, in this process, the data cleaning task is executed passively (a passive trigger scenario, a WebService service is called by another application system to upload a request content and a request data format, and then a shared data center cleans or converts a response content into a required data format), and this way is generally adopted for data with higher real-time requirements; in another method, a data cleansing task is configured in QuartZ, and the QuartZ task is periodically started as a packet to a management module (InBound).
And thirdly, distributing the cleaned data from the theme base to a data exchange area (the theme base is a database, and the cleaned data needs to be taken out to the data exchange area), on one hand, updating the cleaned data to the theme base of the shared data center, and distributing the cleaned data to data tables to be transmitted of each application system needing the data, and deleting the increment tables of the corresponding systems after the distribution is finished.
And fourthly, transmitting the data in the data exchange area to an application system, and updating the data to be transmitted in the data exchange area of the shared data center to a database of the application system.
Referring to fig. 4, the structure of the shared data center of the present invention is divided into three layers, which are a data layer, a service layer and an application layer, and the functions are as follows:
(1) and the data layer is the bottom layer of the system and is responsible for describing and packaging the data into service according to the behavior of the data provided by the shared data center, then registering the service into the service registration center, and when the service is called, executing the corresponding behavior by the database and returning the result to the previous layer.
The enterprise service bus is used for packaging behavior description of data provided by a data source into a service; when the service is called by the system, the service is responsible for activating corresponding behaviors in the data source and returning the result. The service registry is used for registering the service according to the service description submitted by the data center.
(2) And the service layer is responsible for receiving the query request from the previous layer, calling the corresponding service according to the existing service registration information and transmitting the service to the previous layer.
The SOAP router is responsible for mapping requests with services of the data center. If the corresponding service exists, sending a request message for calling; if there is no corresponding message, the application is to be declared that the request cannot be executed. When the request message is sent, the SOAP router accepts the results returned by the enterprise service bus and passes them to the application.
(3) And the application layer is the topmost layer and provides an access interface for an external application system.
Referring to fig. 5, the basic steps of the application system or user accessing the shared data center are as follows:
the application program makes a request according to the requirement;
the request is matched with the service registered in the service registration center, and the result is recorded;
the SOAP router sends a service calling request to the shared data center according to the calling of the request, and the SOAP router is used for transmitting SOAP messages;
the enterprise service bus receives the service request, activates the service, acquires the executed result and returns the result to the SOAP router;
and the SOAP router packages the received call result into a SOAP message and transmits the SOAP message to the application program.
In the embodiment of the invention, the ESB-based shared data center meets the requirements of real-time performance of data transmission and low coupling performance and expandability between the shared data center and each application system, arranges the data of the existing system, gradually removes dirty data existing in each information isolated island, and ensures the authority of the data in the shared data center; data docking with the existing application systems is completed through the shared data center, and correctness of core data interaction between the systems is guaranteed; according to business requirements, data support services are provided for future application systems, namely, newly added systems can realize data synchronization with the existing system only through services provided by a data center.
Third embodiment
Referring to fig. 6, a third embodiment of the present invention provides a data sharing apparatus, including:
an obtaining module 601, configured to obtain incremental data of two adjacent backup data of a first application system;
the processing module 602 is configured to perform data processing on the incremental data to obtain standard data in a preset format;
a sending module 603, configured to send the standard data to a second application system associated with the first application system.
Preferably, the obtaining module 601 includes:
and the obtaining submodule is used for obtaining newly added data and/or changed data in the two adjacent backup data of the first application system, wherein the data of the later backup is compared with the data of the former backup.
Preferably, the acquisition submodule includes:
the replication unit is used for periodically replicating the source data in the first application system to a preset materialized view of the first application system according to a first preset period and updating a materialized view log of the materialized view;
the acquiring unit is used for acquiring newly added data and/or changed data in the two adjacent backup data of the first application system according to the materialized view log, wherein the later backup data is compared with the former backup data;
and the storage unit is used for establishing an increment table of the first application system and storing the increment data in the increment table.
Preferably, the processing module 602 includes:
the conversion submodule is used for carrying out data format conversion on the incremental data in the incremental table to obtain standard data in a preset format;
and the recording submodule is used for storing the standard data in the theme library and recording the time of data format conversion in the materialized view log.
Preferably, the processing module 602 further includes:
and the storage submodule is used for storing the incremental data corresponding to the standard data in the theme library.
Preferably, the sending module 603 includes:
the first sending sub-module is used for sending the standard data to a second application system associated with the first application system through a session number when a target query object in the session number in a received data query request of the second application system is the first application system; or
The second sending submodule is used for regularly generating a session number of the data synchronization request according to a first preset period and sending the standard data to a second application system associated with the first application system according to a preset corresponding relation between the first application system and the second application system; and the target synchronization object in the session number of the data synchronization request is a second application system associated with the first application system through the session number.
Preferably, the apparatus further comprises:
a clearing module for clearing the first application system after the sending module 603 sends the standard data to a second application system associated with the first application system,
the increment table is cleared.
In the embodiment of the present invention, the obtaining module 601 obtains the incremental data and establishes the incremental table for the incremental data, and the change of the first application system data is firstly reflected in the incremental table; the processing module 602 performs data format conversion on the incremental data to form standard data with a uniform format, so as to overcome the problem of format difference among different application systems; a theme base is established for the standard data and is issued to the associated second application system through the sending module 603, so that the real-time performance of data sharing is improved; the invention realizes the data butt joint among all the distributed application systems, really integrates the information among all the systems to carry out the information interaction among the systems and ensures the accuracy of the data interaction; when data support service is provided for a new application system according to business requirements, only standard data in the embodiment of the invention needs to be acquired. The invention solves the problem of data synchronization in a distributed environment, and can minimize the change of the original system, thereby being capable of realizing rapid implementation and generating benefits.
It should be noted that the data sharing apparatus provided in the embodiments of the present invention is an apparatus applying the above method, that is, all embodiments of the above method are applicable to the apparatus, and can achieve the same or similar beneficial effects.
While the foregoing is directed to the preferred embodiment of the present invention, it will be understood by those skilled in the art that various changes and modifications may be made without departing from the spirit and scope of the invention as defined in the appended claims.
Claims (8)
1. A data sharing method, characterized in that the method is applied to a shared data center based on an enterprise service bus, and the shared data center is divided into two main parts: the method comprises the following steps that a data exchange area and a theme base are arranged, wherein the data exchange area is a resource owned by a database user created by the shared data center for each first application system, and the data exchange area corresponds to each first application system, and the method comprises the following steps:
acquiring incremental data of two adjacent backup data of a first application system;
performing data processing on the incremental data to obtain standard data in a preset format;
sending the standard data to a second application system associated with the first application system;
the step of obtaining incremental data of two adjacent backup data of the first application system comprises the following steps:
acquiring newly added data and/or changed data in the two adjacent backup data of the first application system, wherein the later backup data is compared with the former backup data;
in the two adjacent backup data of the first application system, the step of acquiring newly added data and/or changed data in the later backup data compared with the earlier backup data comprises:
copying source data in a first application system to a preset materialized view of the first application system at regular time according to a first preset period, and updating a materialized view log of the materialized view;
acquiring newly added data and/or changed data in the two adjacent backup data of the first application system according to the materialized view log, wherein the later backup data is compared with the former backup data;
establishing an increment table of the first application system, and storing the increment data in the increment table;
the step of performing data processing on the incremental data to obtain standard data in a preset format comprises:
performing data format conversion on the incremental data in the incremental table to obtain standard data in a preset format;
storing the standard data in a theme library, and recording the time of the data format conversion in the materialized view log.
2. The data sharing method of claim 1, wherein the step of storing the standard data in a subject library is followed by:
storing incremental data corresponding to the standard data in a subject library.
3. The data sharing method according to claim 1, wherein the step of sending the standard data to a second application system associated with the first application system comprises:
when a target query object in a session number in a received data query request of a second application system is the first application system, sending the standard data to the second application system associated with the first application system through the session number; or
Generating a session number of a data synchronization request at regular time according to a first preset period, and sending the standard data to a second application system associated with a first application system according to a preset corresponding relation between the first application system and the second application system; and the target synchronization object in the session number of the data synchronization request is a second application system associated with the first application system through the session number.
4. The data sharing method according to claim 1, wherein after the step of sending the standard data to a second application system associated with the first application system, further comprising:
clearing the increment table.
5. A data sharing device, wherein the device is disposed in a data exchange area of a shared data center based on an enterprise service bus, and the shared data center is divided into two main parts: the data exchange area and the theme library, the data exchange area is a resource owned by a database user created by the shared data center for each first application system, and the device comprises:
the acquisition module is used for acquiring incremental data of the adjacent two backup data of the first application system;
the processing module is used for carrying out data processing on the incremental data to obtain standard data in a preset format;
a sending module, configured to send the standard data to a second application system associated with the first application system;
wherein the acquisition module comprises:
the acquisition submodule is used for acquiring newly added data and/or changed data in the adjacent twice backup data of the first application system, wherein the later backup data is compared with the former backup data;
wherein the acquisition submodule comprises:
the system comprises a copying unit, a judging unit and a judging unit, wherein the copying unit is used for copying source data in a first application system to a preset materialized view of the first application system at regular time according to a first preset period and updating a materialized view log of the materialized view;
the acquiring unit is used for acquiring newly added data and/or changed data in the two adjacent backup data of the first application system according to the materialized view log, wherein the later backup data is compared with the former backup data;
the storage unit is used for establishing an increment table of the first application system and storing the increment data in the increment table;
wherein the processing module comprises:
the conversion submodule is used for carrying out data format conversion on the increment data in the increment table to obtain standard data in a preset format;
and the recording sub-module is used for storing the standard data in a theme library and recording the time of data format conversion in the materialized view log.
6. The data sharing apparatus according to claim 5, wherein the processing module further comprises:
and the storage submodule is used for storing the incremental data corresponding to the standard data in the theme library.
7. The data sharing apparatus according to claim 5, wherein the sending module comprises:
the first sending sub-module is used for sending the standard data to a second application system associated with the first application system through the session number when a target query object in the session number in a received data query request of the second application system is the first application system; or
The second sending submodule is used for regularly generating a session number of the data synchronization request according to a first preset period and sending the standard data to a second application system associated with the first application system according to a preset corresponding relation between the first application system and the second application system; and the target synchronization object in the session number of the data synchronization request is a second application system associated with the first application system through the session number.
8. The data sharing apparatus according to claim 5, further comprising:
a clearing module, configured to clear the increment table after the sending module sends the standard data to a second application system associated with the first application system.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610305168.2A CN107357791B (en) | 2016-05-10 | 2016-05-10 | Data sharing method and device |
| PCT/CN2017/079739 WO2017193739A1 (en) | 2016-05-10 | 2017-04-07 | Data sharing method and apparatus |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610305168.2A CN107357791B (en) | 2016-05-10 | 2016-05-10 | Data sharing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107357791A CN107357791A (en) | 2017-11-17 |
| CN107357791B true CN107357791B (en) | 2021-10-15 |
Family
ID=60266887
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610305168.2A Active CN107357791B (en) | 2016-05-10 | 2016-05-10 | Data sharing method and device |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN107357791B (en) |
| WO (1) | WO2017193739A1 (en) |
Families Citing this family (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110309218B (en) * | 2018-02-09 | 2021-07-30 | 杭州数梦工场科技有限公司 | Data exchange system and data writing method |
| CN108897880A (en) * | 2018-07-05 | 2018-11-27 | 南方电网科学研究院有限责任公司 | Electric energy data sharing method and system |
| CN110287255A (en) * | 2019-05-23 | 2019-09-27 | 深圳壹账通智能科技有限公司 | Data sharing method, device and computer equipment based on user behavior |
| CN112825072B (en) * | 2019-11-21 | 2023-02-17 | 青岛海信移动通信技术股份有限公司 | Communication terminal and data sharing method |
| CN111639130A (en) * | 2020-05-26 | 2020-09-08 | 深圳壹账通智能科技有限公司 | Data access method and device based on materialized view and computer equipment |
| CN113537558B (en) * | 2021-06-03 | 2024-06-04 | 广州快批信息科技有限公司 | Processing method, device and terminal equipment for automatic upstream synchronization of material demands |
| CN114079668B (en) * | 2022-01-20 | 2022-04-08 | 檀沐信息科技(深圳)有限公司 | Information acquisition and arrangement method and system based on internet big data |
| CN116633691B (en) * | 2023-07-24 | 2023-10-27 | 四川航天天盛科技有限公司 | Cloud platform system based on SaaS and data storage method thereof |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101266606A (en) * | 2007-03-15 | 2008-09-17 | 阿里巴巴公司 | On-line data migration method based on Oracle database |
| CN101674495A (en) * | 2009-10-20 | 2010-03-17 | 中兴通讯股份有限公司 | Method and device for preprocessing data disaster tolerance |
| CN103617585A (en) * | 2013-11-06 | 2014-03-05 | 梧州学院 | Data sharing platform-based data processing method |
| CN105183860A (en) * | 2015-09-10 | 2015-12-23 | 北京京东尚科信息技术有限公司 | Data synchronization method and system |
| CN105338078A (en) * | 2015-10-26 | 2016-02-17 | 北京百度网讯科技有限公司 | Data storage method and device used for storing system |
Family Cites Families (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6532520B1 (en) * | 1999-09-10 | 2003-03-11 | International Business Machines Corporation | Method and apparatus for allocating data and instructions within a shared cache |
| US20060059140A1 (en) * | 2004-09-14 | 2006-03-16 | Logical Information Machines, Inc. | Correcting data warehouse with prioritized processing for integrity and throughput |
| US8595382B2 (en) * | 2010-06-07 | 2013-11-26 | Salesforce.Com, Inc. | System, method and computer program product for performing a synchronization of data |
| CN102508835B (en) * | 2011-09-22 | 2015-04-15 | 用友软件股份有限公司 | Incremental data real-time synchronizing device based on log management packet and incremental data real-time synchronizing method based on same |
| CN103136231B (en) * | 2011-11-25 | 2016-03-02 | 中国移动通信集团江苏有限公司 | Method of data synchronization between a kind of heterogeneous database and system |
| CN102693324B (en) * | 2012-01-09 | 2015-03-18 | 西安电子科技大学 | Distributed database synchronization system, synchronization method and node management method |
| KR101662212B1 (en) * | 2014-05-30 | 2016-10-04 | 주식회사 알티베이스 | Database Management System providing partial synchronization and method for partial synchronization thereof |
| US9858425B2 (en) * | 2014-07-07 | 2018-01-02 | Qualcomm Incorporated | Method and apparatus for incrementally sharing greater amounts of information between user devices |
| CN105447083A (en) * | 2015-11-06 | 2016-03-30 | 深圳市中润四方信息技术有限公司 | Data convergence and divergence method and system for multi-source heterogeneous database |
| CN105677384A (en) * | 2015-12-28 | 2016-06-15 | 上海爱数信息技术股份有限公司 | System supporting information synchronization of organizations and users between different application systems |
-
2016
- 2016-05-10 CN CN201610305168.2A patent/CN107357791B/en active Active
-
2017
- 2017-04-07 WO PCT/CN2017/079739 patent/WO2017193739A1/en active Application Filing
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101266606A (en) * | 2007-03-15 | 2008-09-17 | 阿里巴巴公司 | On-line data migration method based on Oracle database |
| CN101674495A (en) * | 2009-10-20 | 2010-03-17 | 中兴通讯股份有限公司 | Method and device for preprocessing data disaster tolerance |
| CN103617585A (en) * | 2013-11-06 | 2014-03-05 | 梧州学院 | Data sharing platform-based data processing method |
| CN105183860A (en) * | 2015-09-10 | 2015-12-23 | 北京京东尚科信息技术有限公司 | Data synchronization method and system |
| CN105338078A (en) * | 2015-10-26 | 2016-02-17 | 北京百度网讯科技有限公司 | Data storage method and device used for storing system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107357791A (en) | 2017-11-17 |
| WO2017193739A1 (en) | 2017-11-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107357791B (en) | Data sharing method and device | |
| CN102037463B (en) | Log based replication of distributed transactions using globally acknowledged commits | |
| KR20200100173A (en) | Data replication and data failover within the database system | |
| CN111078504A (en) | Distributed call chain tracking method and device, computer equipment and storage medium | |
| US20090157762A1 (en) | Dynamic Data Reorganization to Accommodate Growth Across Replicated Databases | |
| CN100371900C (en) | Method and system for synchronizing data | |
| CN103605698A (en) | Cloud database system used for distributed heterogeneous data resource integration | |
| CN102262674A (en) | Data synchronization method and system for distributed memory database | |
| CA2341382A1 (en) | System and method for a master scheduler | |
| CN105468720A (en) | Method for integrating distributed data processing systems, corresponding systems and data processing method | |
| CN101763347A (en) | GIS (Geographical Information System) interface platform as well as network GIS management system and management method | |
| KR101545626B1 (en) | System for interoperation between dds and dbms | |
| CN112632035A (en) | Autonomous controllable database migration method and storage medium | |
| CN111008244A (en) | Database synchronization and analysis method and system | |
| US20190251096A1 (en) | Synchronization of offline instances | |
| CN107145673A (en) | A kind of combined simulation system and method | |
| CN112685499A (en) | Method, device and equipment for synchronizing process data of work service flow | |
| JP5065259B2 (en) | Apparatus, system, method, and computer program useful for communication between enterprise information system and client (device, system, and method useful for communication between enterprise information system and client) | |
| CN117677943A (en) | Data consistency mechanism for hybrid data processing | |
| CN103365740B (en) | A kind of data cold standby method and device | |
| CN109120445A (en) | A kind of network log data synchronous system and method | |
| CN109800069B (en) | Method and device for realizing data management | |
| CN114416868A (en) | Data synchronization method, device, equipment and storage medium | |
| CN103634374A (en) | Method and device for processing concurrent access requests | |
| CN112699118A (en) | Data synchronization method, corresponding device, system and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |