CN112612783A

CN112612783A - Method for realizing cross-platform data sharing

Info

Publication number: CN112612783A
Application number: CN202011525975.8A
Authority: CN
Inventors: 赵杰岩; 索凤桐; 冯畅; 张莹; 刘燃
Original assignee: Aisino Corp
Current assignee: Aisino Corp
Priority date: 2020-12-22
Filing date: 2020-12-22
Publication date: 2021-04-06

Abstract

The invention discloses a method and a system for realizing cross-platform data sharing, which comprises the following steps: configuring a data source of the conversion file according to the database information and the server parameter information; newly building a table input script file, and compiling an sql statement in the table input script file to extract table information to obtain the table information; newly building an inserted updating script file, configuring field information and loading target table information, and loading the table information into a corresponding target table according to the configured field information; newly building an SQL script file, compiling a deletion statement, and sequentially connecting all the script files according to a preset sequence; and newly building a job task, setting job scheduling time, and converting all script files in sequence according to the job task to realize cross-platform data sharing. The method can more flexibly realize cross-platform data sharing, maximally reduce code writing, use a graphical interface for configuration and improve later maintainability of the system.

Description

Method for realizing cross-platform data sharing

Technical Field

The present invention relates to the field of data sharing technologies, and in particular, to a method and a system for implementing cross-platform data sharing.

Background

With the proposal of sharing concept, tax sharing also comes, and in order to realize tax sharing, different subsystems need to be integrated, so as to realize tax sharing. However, the situation that databases among subsystems are not matched can be encountered in the integration process, the codes need to be changed if the databases are changed can be processed in a java code mode, and the flexibility is not high.

Therefore, there is a need for a data processing system that can be implemented by a tool and an interface configuration, which is more flexible and relatively easy to maintain.

Disclosure of Invention

The invention provides a method and a system for realizing cross-platform data sharing, which aim to solve the problem of how to realize the cross-platform data sharing.

In order to solve the above problem, according to an aspect of the present invention, there is provided a method of implementing cross-platform data sharing, the method including:

configuring a data source of the conversion file according to the database information and the server parameter information; wherein the data source comprises: a source library data source and a target library data source;

newly building a table input script file, and compiling an sql statement in the table input script file to extract table information to obtain the table information;

newly building an inserted updating script file, configuring field information and loading target table information, and loading the table information into a corresponding target table according to the configured field information;

newly building an SQL script file, compiling a deletion statement, and sequentially connecting all the script files according to a preset sequence;

and newly building a job task, setting job scheduling time, and converting all script files in sequence according to the job task to realize cross-platform data sharing.

Preferably, wherein the method further comprises:

after the table information is obtained, a java script file is created, the java script file is used for carrying out non-null processing on the extracted table information, and the table information which is subjected to the non-null processing is obtained.

Preferably, the loading the table information into the corresponding target table according to the configured field information includes:

matching the table field information of different platforms, comparing whether each piece of data is updated or deleted by setting a main key, and loading the non-empty processed table information into a corresponding target table according to an updating or deleting result.

Preferably, the database information includes: a database name and a database type; the server parameter information includes: host name, port number, user name, and password.

Preferably, wherein the method further comprises:

setting user authority, determining whether a user has the operation authority of the data to be operated according to the user authority, and allowing the user to operate when the user is determined to have the operation authority of the data to be operated; otherwise, the operation of the user is refused.

According to another aspect of the present invention, there is provided a system for implementing cross-platform data sharing, the system comprising:

the configuration unit is used for configuring a data source of the conversion file according to the database information and the server parameter information; wherein the data source comprises: a source library data source and a target library data source;

the table information acquisition unit is used for newly building a table input script file and compiling an sql statement in the table input script file so as to extract table information and acquire the table information;

the table information loading unit is used for newly building an inserted updating script file, configuring field information and loading target table information, and loading the table information into a corresponding target table according to the configured field information;

the script file connecting unit is used for newly building an SQL script file, compiling a deletion statement and sequentially connecting all script files according to a preset sequence;

and the job task execution unit is used for newly building a job task, setting job scheduling time, and converting all script files in sequence according to the job task to realize cross-platform data sharing.

Preferably, wherein the system further comprises:

and the table information processing unit is used for creating a java script file after the table information is acquired, and performing non-null processing on the extracted table information by using the java script file to acquire the table information subjected to the non-null processing.

Preferably, the loading unit of table information loads the table information into the corresponding target table according to the configured field information, including:

Preferably, wherein the system further comprises:

the authority judging unit is used for setting user authority, determining whether the user has the operation authority of the data to be operated according to the user authority, and allowing the user to operate when the user is determined to have the operation authority of the data to be operated; otherwise, the operation of the user is refused.

The invention provides a method and a system for realizing cross-platform data sharing.A system operator can create new authority information such as roles and users in real time in a system with centralized authority management according to actual operation conditions or input new data information; then, the data is synchronously shared among different databases under different systems by an ETL (extract transform and load) synchronization method, so that the method does not occupy the operating resources of each system, greatly reduces the operating pressure of each system, realizes real-time synchronous sharing and reduces the risk caused by untimely synchronization due to data change; the sharing can be realized quickly only by determining the data source and target to be shared, a large amount of codes are not needed to be written, the configuration is carried out by using a graphical interface, and the later maintainability of the system is improved.

Drawings

A more complete understanding of exemplary embodiments of the present invention may be had by reference to the following drawings in which:

FIG. 1 is a flow diagram of a method 100 for implementing cross-platform data sharing according to an embodiment of the present invention;

FIG. 2 is a diagram of an example configuration data source according to an embodiment of the present invention;

FIG. 3 is an exemplary diagram of an insert update script configuration interface according to an embodiment of the present invention;

FIG. 4 is a schematic diagram of all switching order placements according to an embodiment of the invention;

fig. 5 is a schematic structural diagram of a system 500 for implementing cross-platform data sharing according to an embodiment of the present invention.

Detailed Description

The exemplary embodiments of the present invention will now be described with reference to the accompanying drawings, however, the present invention may be embodied in many different forms and is not limited to the embodiments described herein, which are provided for complete and complete disclosure of the present invention and to fully convey the scope of the present invention to those skilled in the art. The terminology used in the exemplary embodiments illustrated in the accompanying drawings is not intended to be limiting of the invention. In the drawings, the same units/elements are denoted by the same reference numerals.

Unless otherwise defined, terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Further, it will be understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense.

The data warehouse technology ETL is a process of data extraction (Extract), Cleaning (Cleaning), transformation (Transform), and loading (Load), and is an important part for constructing a data warehouse, and a user extracts required data from a data source, and loads the data into the data warehouse according to a predefined data warehouse model after data Cleaning. While a button is an ETL tool, which is used very frequently in a data warehouse project, the button can also be applied in the following scenarios: integrating data among different applications or databases, exporting data in the databases to text files, loading mass data into the databases, cleaning data, and integrating application-related projects for use. The keytle is very simple to use, realizes what business is done through graphical interface design, and does not need to write codes to realize, so the keytle is designed in a way of orienting metadata. The keylet supports a wide variety of input and output formats, including text files, data sheets, and commercial and free database engines. In addition, the powerful conversion function of the button makes it very convenient for you to manipulate data. There are two script files in the button, transformation and job, the transformation completes the basic conversion for the data, and the job completes the control of the whole workflow. Changing: data transfer (Data transfer) is the process of changing Data from one representation to another, and when Data is transferred from one system to another, it is necessary to convert the Data to a format that the system can recognize.

The system applied by the method is a set of integrity system which is produced by the operation in the sharing service mode, and some enterprises apply a single tax management system platform, but the system is not butted and fused with the financial sharing service platform, and the data between the systems can not be shared, so that some data are respectively input in two or more systems, the repeated low-efficiency labor is increased, and the tax management cost is increased. Meanwhile, as the system is huge and has various functions, the data needing synchronization is complicated and various, and the synchronous data of a common interface (such as a webservice data synchronization interface) occupies too much system resources, so that the response time of the system is too long, the conventional financial sharing service platform is reasonably utilized and optimized, the repeated investment of informatization is reduced, the multi-platform data sharing is realized, and the integrated management of tax and finance is the transformation selection of the conventional financial sharing service center, so that the invention uses a method for realizing cross-platform data sharing based on the ETL technology.

FIG. 1 is a flow chart of a method 100 for implementing cross-platform data sharing according to an embodiment of the present invention. As shown in fig. 1, the method for implementing cross-platform data sharing provided in the embodiment of the present invention only needs to determine the source and target of data to be shared, and can implement sharing quickly without writing a large amount of code, and use a graphical interface for configuration, thereby improving the post-maintainability of the system. The method 100 for realizing cross-platform data sharing provided by the embodiment of the invention starts from step 101, and configures a data source of a conversion file according to database information and server parameter information in step 101; wherein the data source comprises: a source library data source and a target library data source.

In step 102, a new table input script file is created, and an sql statement is written in the table input script file to extract table information, thereby obtaining the table information.

Preferably, wherein the method further comprises:

In an embodiment of the present invention, first configuring a data source of a transformation file includes: a source repository and a target repository data source are created separately. As shown in fig. 2, includes: configuring host names, port numbers, user names, passwords and the like.

When data extraction is carried out, an 'table input' script is newly built, an sql statement is written in the script to realize data extraction, a java script is created to process the extracted data, and data cleaning is realized. Correspondingly, table information is extracted through SQL in the background, extracted data is processed through java codes, partial fields are subjected to non-empty processing, and influence on the system is caused by eliminating empty fields.

In step 103, an update script file is newly created, field information and loading target table information are configured, and the table information is loaded into the corresponding target table according to the configured field information.

In step 104, newly creating an SQL script file, writing a delete statement, and sequentially connecting all the script files according to a preset sequence.

In step 105, a job task is newly established, job scheduling time is set, and all script files are converted in sequence according to the job task, so that cross-platform data sharing is realized.

Preferably, wherein the method further comprises:

In the invention, data conversion and loading are also needed to realize data sharing. First, by creating a "insert update" script, and configuring fields and loading target table information, the data that has been extracted and processed before is loaded into the corresponding table, and the specific process is as shown in fig. 3. The method comprises the steps of setting a main key for comparison by corresponding table fields of different platforms, and judging whether the data is newly added and updated or deleted according to the main key. Then, by newly creating an SQL script, a delete statement is compiled, and all scripts are sequentially connected according to the sequence of the SQL script, the table input, the java script and the insertion update script. And finally, newly building a job, placing all the conversions in sequence, as shown in fig. 4, setting job timing scheduling time, and realizing data sharing according to the set time.

In addition, in order to ensure the safety of data sharing, the invention sets user authority, determines whether the user has the operation authority of the data to be operated according to the user authority, and allows the user to operate when determining that the user has the operation authority of the data to be operated; otherwise, the operation of the user is refused.

The invention realizes the cross-platform sharing of data based on the ETL technology, realizes the unified management of data sharing among different platforms of a plurality of systems, only needs to determine the source and the target of the data to be shared, can quickly realize the sharing, and does not need to compile a large amount of codes. A system operator can create new authority information such as roles and users in real time in the authority centralized management system or input new data information according to actual operation conditions, and then the data is synchronously shared among different databases under different systems by an ETL (extract transform and load) synchronization method.

Fig. 5 is a schematic structural diagram of a system 500 for implementing cross-platform data sharing according to an embodiment of the present invention. As shown in fig. 5, a system 500 for implementing cross-platform data sharing according to an embodiment of the present invention includes: a configuration unit 501, a table information acquisition unit 502, a table information loading unit 503, a script file connection unit 504, and a job task execution unit 505.

Preferably, the configuration unit 501 is configured to configure a data source of the conversion file according to the database information and the server parameter information; wherein the data source comprises: a source library data source and a target library data source.

Preferably, the table information obtaining unit 502 is configured to newly create a table input script file, and write an sql statement in the table input script file, so as to extract table information and obtain table information.

Preferably, wherein the system further comprises:

Preferably, the table information loading unit 503 is configured to newly create an insert update script file, configure field information and load target table information, and load the table information into a corresponding target table according to the configured field information.

Preferably, the loading unit 503 of the table information loads the table information into the corresponding target table according to the configured field information, and includes:

Preferably, the script file connecting unit 504 is configured to create an SQL script file, write a delete statement, and sequentially connect all script files according to a preset sequence.

Preferably, the job task execution unit 505 is configured to create a job task, set job scheduling time, and sequentially convert all script files according to the job task, so as to implement cross-platform data sharing.

Preferably, wherein the system further comprises:

The system 500 for implementing cross-platform data sharing according to the embodiment of the present invention corresponds to the method 100 for implementing cross-platform data sharing according to another embodiment of the present invention, and is not described herein again.

The invention has been described with reference to a few embodiments. However, other embodiments of the invention than the one disclosed above are equally possible within the scope of the invention, as would be apparent to a person skilled in the art from the appended patent claims.

Generally, all terms used in the claims are to be interpreted according to their ordinary meaning in the technical field, unless explicitly defined otherwise herein. All references to "a/an/the [ device, component, etc ]" are to be interpreted openly as referring to at least one instance of said device, component, etc., unless explicitly stated otherwise. The steps of any method disclosed herein do not have to be performed in the exact order disclosed, unless explicitly stated.

As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

Finally, it should be noted that: the above embodiments are only for illustrating the technical solutions of the present invention and not for limiting the same, and although the present invention is described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that: modifications and equivalents may be made to the embodiments of the invention without departing from the spirit and scope of the invention, which is to be covered by the claims.

Claims

1. A method for implementing cross-platform data sharing, the method comprising:

2. The method of claim 1, further comprising:

3. The method of claim 2, wherein the loading the table information into the corresponding target table according to the configured field information comprises:

4. The method of claim 1, wherein the database information comprises: a database name and a database type; the server parameter information includes: host name, port number, user name, and password.

5. The method of claim 1, further comprising:

6. A system for enabling cross-platform data sharing, the system comprising:

7. The system of claim 6, further comprising:

8. The system of claim 7, wherein the table information loading unit loads the table information into the corresponding target table according to the configured field information, and comprises:

9. The system of claim 6, wherein the database information comprises: a database name and a database type; the server parameter information includes: host name, port number, user name, and password.

10. The system of claim 6, further comprising: