CN115525614A - Data access method, device, equipment, system and storage medium - Google Patents
Data access method, device, equipment, system and storage medium Download PDFInfo
- Publication number
- CN115525614A CN115525614A CN202211203998.6A CN202211203998A CN115525614A CN 115525614 A CN115525614 A CN 115525614A CN 202211203998 A CN202211203998 A CN 202211203998A CN 115525614 A CN115525614 A CN 115525614A
- Authority
- CN
- China
- Prior art keywords
- data
- data access
- platform
- access center
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/16—File or folder operations, e.g. details of user interfaces specifically adapted to file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/11—File system administration, e.g. details of archiving or snapshots
- G06F16/116—Details of conversion of file system types or formats
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a data access method, a device, equipment, a system and a storage medium. The invention relates to the technical field of big data. The method comprises the following steps: the data access center acquires the interface document according to the identification information of the interface document; the interface document is a document containing a preset incidence relation between the data source platform and the data access center; the data access center acquires platform data of a data source platform and converts the platform data into a first data model of the data access center according to the interface document; and generating a second data model of the data access center according to the first data model and a preset model rule of the data access center, and converting the generated first data model into the second data model. The technical scheme of the invention can achieve the technical effects of not needing to carry out customized data access development aiming at each platform, reducing the data access cost and improving the data access efficiency.
Description
Technical Field
The embodiment of the invention relates to the technical field of big data, in particular to a data access method, a device, equipment, a system and a storage medium.
Background
In the prior art, a data center needs to be continuously accessed with massive platform data in the construction process so as to perform comprehensive processing.
When data is accessed, platform data needs to be converted into data meeting the requirements of a data center, and because the data of each platform has differences in format and the like, each platform needs to have an independent logic conversion rule, and usually customized development needs to be performed on each platform, so that a large amount of resources are consumed and the efficiency of data access is reduced.
Disclosure of Invention
Embodiments of the present invention provide a data access method, apparatus, device, system, and storage medium, so as to solve the problems that a large amount of resources are consumed for data access and the efficiency is low.
In a first aspect, an embodiment of the present invention provides a data access method, including:
the data access center acquires the interface document according to the identification information of the interface document; the interface document is a document containing a preset association relation between a data source platform and the data access center;
the data access center acquires platform data of the data source platform and converts the platform data into a first data model of the data access center according to the interface document;
and generating a second data model of the data access center according to the first data model and a preset model rule of the data access center.
In a second aspect, an embodiment of the present invention further provides a data access apparatus, where the apparatus includes:
the interface document acquisition module is used for acquiring the interface document by the data access center according to the identification information of the interface document; the interface document is a document containing a preset association relationship between a data source platform and the data access center;
the first model conversion module is used for the data access center to acquire platform data of the data source platform and convert the platform data into a first data model of the data access center according to the interface document;
and the second model generation module is used for generating a second data model of the data access center according to the first data model and a preset model rule of the data access center.
In a third aspect, an embodiment of the present invention further provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor executes the computer program to implement the data access method according to any one of the embodiments of the present invention.
In a fourth aspect, the embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the data access method according to any one of the embodiments of the present invention.
In a fifth aspect, embodiments of the present invention further provide a computer program product, which includes a computer program and when executed by a processor, the computer program implements the data access method according to any of the embodiments of the present invention.
In the embodiment of the invention, a data access center acquires an interface document according to identification information of the interface document; the interface document is a document containing a preset association relation between the data source platform and the data access center; the data access center obtains platform data of a data source platform, and converts the platform data into a first data model of the data access center according to the interface document, so that the converted platform data meets the data use requirements of the data access center, the data standards are unified, the interface document can be directly modified when the data standards are changed, customized development for each platform is not needed, the data access cost is reduced, and the data access efficiency is improved. The data of different data models and different data rules in different data source platforms can be accessed to the same data access center, and the rapid construction of the data access center is realized.
And generating a second data model of the data access center according to the first data model and a preset model rule of the data access center, and converting the generated first data model into the second data model, so that the data is suitable for the data access center, the data access center is convenient to directly use subsequently, and the effectiveness of data access is improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained according to the drawings without inventive efforts.
Fig. 1 is a first flowchart of a data access method according to an embodiment of the present application;
fig. 2 is a second flowchart of a data access method according to an embodiment of the present application;
fig. 3 is a flow chart three of a data access method provided in the embodiment of the present application;
fig. 4 is a fourth flowchart of a data access method provided in the embodiment of the present application;
fig. 5 is a fifth flowchart of a data access method according to an embodiment of the present application;
fig. 6 is a schematic structural diagram of a data access apparatus according to an embodiment of the present invention;
fig. 7 is a schematic structural diagram of an electronic device for implementing an embodiment of the invention.
Detailed Description
The present invention will be described in further detail with reference to the drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some structures related to the present invention are shown in the drawings, not all of them.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures. Meanwhile, in the description of the present invention, the terms "first", "second", and the like are used only for distinguishing the description, and are not to be construed as indicating or implying relative importance. According to the technical scheme, the data acquisition, storage, use, processing and the like meet relevant regulations of national laws and regulations.
The following describes the technical solutions of the present application and how to solve the above technical problems with specific embodiments. The following several specific embodiments may be combined with each other, and details of the same or similar concepts or processes may not be repeated in some embodiments.
Fig. 1 is a first flowchart of a data access method according to an embodiment of the present application. As shown in fig. 1, the data access method includes:
step 110, the data access center acquires an interface document according to the identification information of the interface document; the interface document is a document containing a preset association relationship between the data source platform and the data access center.
The data access center is used for uniformly accessing data of each platform, and may be a user data center, which is not limited in this embodiment. Each data access center may access data for one or more platforms.
The identification information of the interface document may be a document ID or the like for uniquely identifying the document identity. The interface document is obtained according to the identification information, and the interface document may be obtained by querying from a preset location where each interface document is stored according to the identification information before obtaining the platform data of the platform.
The interface document includes a preset association relationship between the data source platform and the data access center, where the data source platform is a platform for providing a data source, and the preset association relationship may include a physical model relationship between the data source platform and the data access center, a conversion relationship between data rules and code values, and the like, and this embodiment does not limit this.
Optionally, before the data access center acquires the interface document according to the identification information of the interface document, the method further includes:
generating an interface document according to a preset incidence relation between a data source platform and a data access center;
and importing the interface document into a second database of the data access center, and determining the identification information of the interface document.
And generating an interface document according to a preset incidence relation between the data source platform and the data access center, wherein the preset incidence relation can be predetermined by the data source platform and the data access center.
And importing the generated interface document into a second database of the data access center, and identifying the unique identification information of the interface document. Interface documents corresponding to different data source platforms can be stored in the same database, so that the interface documents can be conveniently and uniformly inquired according to the identification information when being used subsequently, and the inquiry efficiency of the interface documents is improved.
And step 120, the data access center acquires platform data of the data source platform, and converts the platform data into a first data model of the data access center according to the interface document.
The platform data of the data source platform is data that the data source platform needs to provide to the data access center, and the obtaining mode may be that the data source platform transmits to a data storage location of the data access center, which is not limited in this embodiment.
The platform data is converted into a first data model of the data access center according to the interface document, a data processing conversion script of the data source platform can be automatically generated for the data access center according to the content of the interface document, the script is called, the platform data of the data source platform is subjected to standardization processing, for example, the platform data is converted into data conforming to the data format of the data access center, the first data model is formed by the data subjected to the standardization processing, and the first data model can be formed by different tables containing data.
Optionally, the obtaining, by the data access center, platform data of the data source platform includes:
the data source platform generates platform data in a preset format according to the data acquisition requirement of the data access center;
the data access center acquires platform data in a preset format from the data source platform.
The data source platform generates platform data in a preset format according to a data acquisition requirement of the data access center, where the data acquisition requirement may be a data format acquisition requirement, and illustratively, if the data acquisition requirement of the data access center is information requiring generation of an EXCEL format, the data source platform generates a copy of the platform data in the EXCEL format and sends the copy to the data access center, so that the data access center acquires the platform data in the preset format.
The platform data is generated into the preset format, so that the platform data of each data source platform is transmitted to the platform data unified format of the data access center, the problem that if the obtained platform data formats are not unified, the data access platforms need to adopt multiple modes to process data, and the processing difficulty is increased is solved, and the data processing efficiency of the data access center is improved through format unification.
And step 130, generating a second data model of the data access center according to the first data model and a preset model rule of the data access center.
The preset model rule is a rule for processing data in the first data model, for example, merging and splitting the data or reserving the data as it is, and the like, for example, if the first data model includes a table a and a table B, the second data model of the data access center is generated according to the first data model and the preset model rule of the data access center, the table a and the table B may be merged, the table a may be split into A1 and A2, and then the A1 is merged with other tables of the data center, or the original tables a and B and the like may be reserved, which is not limited in this embodiment.
And adjusting the first data model according to a preset model rule to generate a second data model of the data access center.
The embodiment obtains the interface document through the data access center according to the identification information of the interface document; the interface document is a document containing a preset incidence relation between the data source platform and the data access center; the data access center obtains platform data of a data source platform, converts the platform data into a first data model of the data access center according to the interface document, enables the converted platform data to meet the data use requirements of the data access center, achieves unification of data standards, can directly modify the interface document when the data standards are changed, does not need to carry out customized development on each platform, reduces the cost of data access and improves the efficiency of data access. The data of different data models and different data rules in different data source platforms can be accessed to the same data access center, and the rapid construction of the data access center is realized.
And generating a second data model of the data access center according to the first data model and a preset model rule of the data access center, and converting the generated first data model into the second data model, so that the data is suitable for the data access center, the data access center is convenient to directly use subsequently, and the effectiveness of data access is improved.
Fig. 2 is a second flowchart of a data access method according to an embodiment of the present disclosure. As shown in fig. 2, the data access method includes the following steps:
step 210, the data access center obtains an interface document according to the identification information of the interface document; the interface document is a document containing a preset association relation between the data source platform and the data access center.
Step 220, the data access center obtains data information and table structure information from the data source platform.
The data information is data that the data source platform needs to access to the data access center, and examples of the data information include account information, personal basic information, authentication information, and related enterprise information. The range of the data information may be predetermined, for example, all data in the data source platform may be accessed, or part of categories of data may be accessed, which is not limited in this embodiment.
The table structure information is structure information of a table storing data in the data source platform, and includes, for example, a table name, an included field, a field name of each field, a field type, a width, and the like. The table structure information of the table storing the range data information may be acquired according to the range of the acquired data information determined.
Optionally, acquiring data information from the data source platform includes:
acquiring data information from a data source platform to a preset storage position of a data access center;
and reading the data information from the preset storage position to the local part of the data access center in response to the data reading operation.
The data information is acquired from the data source platform to a preset storage position of the data access center, and the preset storage position can be a directory specified by the data source platform for sending the data information to the data access center, wherein the preset storage position can be located at a cloud end of the data access center.
Responding to a data reading operation to read data information from a preset storage position to the local of a data access center, wherein the data reading operation can be sent by the data access center and used for reading corresponding data information, and the data reading operation can be sent at regular time or when the data information needs to be acquired; in addition, when the data information is read, all the unread data may be read, or a part of the unread data may be read, which is not limited in this embodiment.
The data source platform acquires the data information to a preset storage position of the data access center, the data information is temporarily stored, the data information is read from the preset storage position to the local part of the data access center in response to data reading operation, the required data information is read to the local part of the data access center, pressure of the data access center caused by the fact that a large amount of data is read at one time is avoided from being increased, the data information is read from the preset storage position when the data access center actually needs the data information, and the data reading speed is increased.
Optionally, loading the data information into the target table includes:
and loading data information related to the target table into the target table from the data access center locally according to the table information of the target table.
And loading data information related to the target table into the target table from the data access center locally according to the table information of the target table. The table information may be table name information of the target table, the data information may have a preset association relationship with the table name information, and the data information related to the target table is determined locally by the data access center according to the table name information of the target table, so that the data information related to the target table is loaded into the target table. The data information loaded in the target table is associated with the target table, and the accuracy of data loading is improved.
Step 230, generating a table building statement according to the table structure information, and executing the table building statement in the first database of the data access center to obtain the target table.
And generating a table building statement according to the table structure information, wherein the table building statement is used for establishing a new table, the table name information of the table can be acquired, the table structure information corresponding to the table is acquired according to the table name information, and the table building statement can be automatically generated according to the acquired table structure information.
Executing the table building statement in the first database of the data access center to obtain a target table, namely creating the target table in the first database of the data access center by executing the table building statement, wherein the target table is a table created by the table building statement.
It should be noted that the first database and the second database may be the same database or different databases, which is not limited in this embodiment.
And 240, loading the data information into the target table, and converting the data information into a first data model of the data access center according to the interface document.
And loading the data information into the target table, wherein the data information can be loaded into the target table correspondingly according to the position of the data information in the table of the data source platform. And carrying out normalized processing on the data information in the target table according to the interface document, and converting the data information into a first data model of the data access center.
And 250, generating a second data model of the data access center according to the first data model and a preset model rule of the data access center.
By acquiring data information and table structure information; and generating a table building statement according to the table structure information, and executing the table building statement in a first database of the data access center to obtain a target table, so that the established target table can restore a corresponding table in the data source platform, and accordingly, when data information is loaded into the target table, the data information can be correspondingly loaded to the position of the original data in the table, loading omission is avoided, data information of the data source platform can be completely acquired, and the integrity of information acquisition is improved.
Fig. 3 is a third flowchart of a data access method according to an embodiment of the present application. As shown in fig. 3, the data access method includes:
step 310, the data access center acquires an interface document according to the identification information of the interface document; the interface document is a document containing a preset association relationship between the data source platform and the data access center.
And step 320, the data access center acquires platform data of the data source platform and converts the platform data into a first data model of the data access center according to the interface document.
And 330, generating a second data model of the data access center according to the first data model and a preset model rule of the data access center.
And 340, judging whether the platform data is the same data as the historical center data of the data access center.
Optionally, steps 340 to 360 may be executed synchronously in the process of acquiring the platform data, or may be executed synchronously in the process of executing step 330, which is not limited in this embodiment.
Whether the platform data is the same as the historical center data of the data access center is judged, wherein the historical center data of the data access center is the data stored in the data access center in a historical mode, whether the platform data is the same as the historical center data is judged, and the judgment can be carried out through a data identifier.
And 350, if so, acquiring a first data identifier, and adding the first data identifier to the platform data.
If the platform data is the same data as the historical center data of the data access center, it indicates that the data access center has obtained the platform data, and the platform data may be obtained from the data source platform or obtained from other data source platforms, which is not limited in this embodiment.
And acquiring a first data identifier, wherein the first data identifier can be an existing identifier, the first data identifier is an identifier indicating that data already exist, and the first data identifier is added to the platform data which is the same as the historical center data of the data access center.
And step 360, if not, generating a second data identifier, and adding the second data identifier to the platform data.
And if the platform data is different from the historical center data of the data access center, the platform data is the newly added data, a second data identifier is generated, the second data identifier is the identifier indicating that the platform data is the newly added data, and the second data identifier is added to the platform data different from the historical center data of the data access center.
Optionally, after determining whether the platform data is the same data as the historical center data of the data access center, the method further includes:
if so, determining whether the data access center acquires the same data or not according to a preset data processing rule.
And if the platform data is the same data as the historical center data of the data access center, determining whether the data access center acquires the platform data or not according to a preset data processing rule.
The preset data processing rule is a rule for determining whether to acquire the same data. For example, if the preset data processing rule is to acquire the same data, the platform data may be acquired to the data access center, so as to facilitate subsequent processing such as cross-certification in the data access center.
And determining whether the data access center acquires platform data according to a preset data processing rule, so as to avoid the problem that the subsequent data processing is influenced because the same data is directly filtered. And the effectiveness of data acquisition is improved.
The first data identifier is added to the same data in the platform data, which is the same as the historical center data of the data access center, so that the subsequent analysis of each platform data is facilitated. And adding the second data identifier to data which is different from the historical center data of the data access center in the platform data, so that the data can be summarized and sorted conveniently, and the effectiveness of data acquisition is improved.
Fig. 4 is a fourth flowchart of a data access method according to an embodiment of the present application. As shown in fig. 4, the data access method includes:
step 410, the data access center acquires an interface document according to the identification information of the interface document; the interface document is a document containing a preset association relation between the data source platform and the data access center.
Step 420, the data access center obtains platform data of the data source platform, and converts the platform data into a first data model of the data access center according to the interface document.
And 430, generating a second data model of the data access center according to the first data model and a preset model rule of the data access center.
Step 440, responding to the changing operation of the preset model rule, changing the preset model rule.
The operation of changing the preset model rule may be executed by the data access center, and the change of the preset model rule may be addition, deletion, modification, and the like of the preset model rule, which is not limited in this embodiment.
And step 450, reading the change information of the preset model rule, and updating the second data model according to the change information.
And reading the change information of the preset model rule, wherein the change information is the information of the part of the preset model rule, which is changed. The second data model is updated according to the change information, which may be only the changed part is updated correspondingly, or may be the second data model is updated as a whole, and this embodiment does not limit this.
And updating the second data model by changing the preset model rule, so that the changed preset model rule is automatically adapted to the second data model generated by the platform, and the updating efficiency of the second data model is improved. And when the platform data generates the second data model, the second data model can be automatically generated according to the latest preset model rule without changing the platform one by one, so that the model generation efficiency is improved, and the method is suitable for the rapid iteration of a data access center.
Fig. 5 is a fifth flowchart of a data access method according to an embodiment of the present application. As shown in fig. 5, the data access method includes:
step 510, the data access center acquires an interface document according to the identification information of the interface document; the interface document is a document containing a preset association relationship between the data source platform and the data access center.
And step 520, the data access center acquires platform data of the data source platform, and determines new data associated with the data source platform in the data access center according to the interface document.
According to the interface rules in the interface documents, a statistical mode is determined, platform data accessed to the data access center are automatically counted, and the statistics can be performed exemplarily by the dimensions of accounts, enterprises, users and the like.
Step 530, determining the output data of the data source platform according to the interface document.
According to the interface rules in the interface document, a statistical mode is determined, platform data output by the source platform are automatically counted, and statistics can be performed exemplarily according to dimensions such as accounts, enterprises and users.
And 540, judging whether the newly added data and the output data have difference so as to obtain a judgment result.
And comparing the newly added data obtained in the same statistical mode with the output data, and judging whether a difference exists. The obtained judgment result may be: if the difference exists, the part with the difference can be identified; and if no difference exists, determining that the judgment result is a pass.
And 550, converting the platform data into a first data model of the data access center according to the judgment result and the interface document.
And converting the platform data into a first data model of the data access center according to the judgment result and the interface document, wherein if the data amount identified in the judgment result exceeds a preset threshold value, a corresponding prompt can be performed to determine whether to continue the conversion operation of the first data model.
If the judgment result is that the platform data passes or the data volume identified in the judgment result is smaller than the preset threshold value, the platform data can be converted into a first data model of the data access center according to the interface document.
And 560, generating a second data model of the data access center according to the first data model and a preset model rule of the data access center.
Determining new data associated with a data source platform in a data access center according to the interface document; determining output data of a data source platform according to the interface document; and judging whether the newly added data is different from the output data, realizing automatic data verification of data access center data and data source platform data, and avoiding the platform data from generating events such as missing transmission and error transmission in the transmission process to influence subsequent data processing. And the newly added data and the output data are determined through the interface document, the statistical standard of the newly added data and the output data is unified, and the accuracy of data verification is improved.
Fig. 6 is a schematic structural diagram of a data access apparatus according to an embodiment of the present invention. The device can be realized in a hardware and/or software mode, can execute the data access method provided by any embodiment of the invention, and has corresponding functional modules and beneficial effects of the execution method. As shown in fig. 6, the apparatus includes:
an interface document obtaining module 610, configured to, by a data access center, obtain an interface document according to identification information of the interface document; the interface document is a document containing a preset association relationship between a data source platform and the data access center;
a first model conversion module 620, configured to the data access center obtain platform data of the data source platform, and convert the platform data into a first data model of the data access center according to the interface document;
a second model generating module 630, configured to generate a second data model of the data access center according to the first data model and a preset model rule of the data access center.
Optionally, the platform data includes: data information and table structure information;
the first model conversion module comprises:
an information obtaining unit, configured to obtain, by the data access center, the data information and the table structure information from the data source platform;
the table building statement executing unit is used for generating a table building statement according to the table structure information and executing the table building statement in a first database of the data access center to obtain a target table;
the information loading unit is used for loading the data information into the target table;
correspondingly, the first model conversion module comprises:
and the first model conversion unit is used for converting the data information into a first data model of the data access center according to the interface document.
Optionally, the information obtaining unit includes:
the information acquisition subunit is used for acquiring the data information from the data source platform to a preset storage position of the data access center;
and the information reading subunit is used for responding to data reading operation to read the data information from the preset storage position to the local part of the data access center.
Optionally, the information loading unit includes:
and the information loading subunit is used for locally loading the data information related to the target table into the target table from the data access center according to the table information of the target table.
Optionally, the first model conversion module includes:
the data generating unit is used for generating the platform data in a preset format by the data source platform according to the data acquisition requirement of the data access center;
and the data acquisition unit is used for acquiring the platform data in the preset format from the data source platform by the data access center.
Optionally, the apparatus further comprises:
the same data judgment module is used for judging whether the platform data are the same data as the historical center data of the data access center;
the first identifier adding module is used for acquiring a first data identifier if the same data judging module judges that the same data is true, and adding the first data identifier to the platform data;
and the second identifier adding module is used for generating a second data identifier and adding the second data identifier to the platform data if the same data judging module judges that the same data is not the same data.
Optionally, the apparatus further comprises:
and the data acquisition determining module is used for determining whether the data access center acquires the same data or not according to a preset data processing rule if the same data judging module judges that the data access center acquires the same data.
Optionally, the apparatus further comprises:
the rule changing module is used for responding to changing operation of the preset model rule and changing the preset model rule;
and the model updating module is used for reading the change information of the preset model rule and updating the second data model according to the change information.
Optionally, the apparatus further comprises:
the newly added data determining module is used for determining newly added data which is associated with the data source platform in the data access center according to the interface document after the data access center of the first model converting module obtains platform data of the data source platform for execution;
the output data determining module is used for determining the output data of the data source platform according to the interface document;
and the data difference judging module is used for judging whether the newly added data and the output data have difference so as to obtain a judging result.
Optionally, the apparatus further comprises:
an interface document generating module, configured to generate the interface document according to the preset association relationship between the data source platform and the data access center before the interface document acquiring module;
and the identification information determining module is used for importing the interface document into a second database of the data access center and determining the identification information of the interface document.
FIG. 7 illustrates a schematic diagram of an electronic device 10 that may be used to implement an embodiment of the invention. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed herein.
As shown in fig. 7, the electronic device 10 includes at least one processor 11, and a memory communicatively connected to the at least one processor 11, such as a Read Only Memory (ROM) 12, a Random Access Memory (RAM) 13, and the like, wherein the memory stores a computer program executable by the at least one processor, and the processor 11 can perform various suitable actions and processes according to the computer program stored in the Read Only Memory (ROM) 12 or the computer program loaded from a storage unit 18 into the Random Access Memory (RAM) 13. In the RAM 13, various programs and data necessary for the operation of the electronic apparatus 10 may also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other via a bus 14. An input/output (I/O) interface 15 is also connected to the bus 14.
A number of components in the electronic device 10 are connected to the I/O interface 15, including: an input unit 16 such as a keyboard, a mouse, or the like; an output unit 17 such as various types of displays, speakers, and the like; a storage unit 18 such as a magnetic disk, an optical disk, or the like; and a communication unit 19 such as a network card, modem, wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.
In some embodiments, the data access method may be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device 10 via the ROM 12 and/or the communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the data access method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the data access method by any other suitable means (e.g. by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, the distributed storage system, the at least one input device, and the at least one output device.
Computer programs for implementing the methods of the present invention can be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions/acts specified in the flowchart and/or block diagram block or blocks to be performed. A computer program can execute entirely on a machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of the present invention, a computer-readable storage medium may be a tangible medium that can contain, or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium may be a machine readable signal medium. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user may provide input to the electronic device. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), blockchain networks, and the Internet.
The computing system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical host and VPS service are overcome.
It should be understood that various forms of the flows shown above, reordering, adding or deleting steps, may be used. For example, the steps described in the present invention may be executed in parallel, sequentially, or in different orders, and are not limited herein as long as the desired results of the technical solution of the present invention can be achieved.
Embodiments of the present invention further provide a computer program product, including a computer program, which, when executed by a processor, implements the data access method provided in any of the embodiments of the present application.
Computer program product in implementing the computer program code for carrying out operations of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments illustrated herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims (15)
1. A method for accessing data, comprising:
the data access center acquires the interface document according to the identification information of the interface document; the interface document is a document containing a preset association relation between a data source platform and the data access center;
the data access center acquires platform data of the data source platform and converts the platform data into a first data model of the data access center according to the interface document;
and generating a second data model of the data access center according to the first data model and a preset model rule of the data access center.
2. The method of claim 1, wherein the platform data comprises: data information and table structure information;
the data access center acquires the platform data of the data source platform, and the method comprises the following steps:
the data access center acquires the data information and the table structure information from the data source platform;
generating a table building statement according to the table structure information, and executing the table building statement in a first database of the data access center to obtain a target table;
and loading the data information into the target table.
Correspondingly, the converting the platform data into the first data model of the data access center according to the interface document includes:
and converting the data information into a first data model of the data access center according to the interface document.
3. The method of claim 2, wherein obtaining data information from the data source platform comprises:
acquiring the data information from the data source platform to a preset storage position of the data access center;
and reading the data information from the preset storage position to the local part of the data access center in response to a data reading operation.
4. The method of claim 3, wherein loading the data information into the target table comprises:
and loading data information related to the target table into the target table from the data access center locally according to the table information of the target table.
5. The method of claim 1, wherein the data access center obtains platform data of the data source platform, and comprises:
the data source platform generates the platform data in a preset format according to the data acquisition requirement of the data access center;
and the data access center acquires the platform data in the preset format from the data source platform.
6. The method of claim 1, further comprising:
judging whether the platform data are the same as the historical center data of the data access center or not;
if so, acquiring a first data identifier, and adding the first data identifier to the platform data;
and if not, generating a second data identifier, and adding the second data identifier to the platform data.
7. The method of claim 6, after determining whether the platform data is the same data as historical center data of the data access center, further comprising:
if so, determining whether the data access center acquires the same data or not according to a preset data processing rule.
8. The method of claim 1, further comprising:
changing the preset model rule in response to the changing operation of the preset model rule;
and reading the change information of the preset model rule, and updating the second data model according to the change information.
9. The method of claim 1, after the data access center obtains the platform data of the data source platform, further comprising:
determining newly added data associated with the data source platform in the data access center according to the interface document;
determining output data of the data source platform according to the interface document;
and judging whether the newly added data and the output data have difference or not to obtain a judgment result.
10. The method according to claim 1, wherein before the data access center obtains the interface document according to the identification information of the interface document, the method further comprises:
generating the interface document according to the preset incidence relation between the data source platform and the data access center;
and importing the interface document into a second database of the data access center, and determining the identification information of the interface document.
11. A data access apparatus, comprising:
the interface document acquisition module is used for acquiring the interface document by the data access center according to the identification information of the interface document; the interface document is a document containing a preset association relation between a data source platform and the data access center;
the first model conversion module is used for the data access center to acquire platform data of the data source platform and convert the platform data into a first data model of the data access center according to the interface document;
and the second model generation module is used for generating a second data model of the data access center according to the first data model and a preset model rule of the data access center.
12. The apparatus of claim 11, wherein the platform data comprises: data information and table structure information;
the first model conversion module comprises:
an information obtaining unit, configured to the data access center obtain the data information and the table structure information from the data source platform;
the table building statement executing unit is used for generating a table building statement according to the table structure information and executing the table building statement in a first database of the data access center to obtain a target table;
the information loading unit is used for loading the data information into the target table;
accordingly, the first model conversion module comprises:
and the first model conversion unit is used for converting the data information into a first data model of the data access center according to the interface document.
13. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the data access method of any one of claims 1-10 when executing the computer program.
14. A computer-readable storage medium, on which a computer program is stored, which program, when being executed by a processor, is adapted to carry out the data access method according to any one of claims 1-10.
15. A computer program product comprising a computer program, wherein the computer program, when executed by a processor, implements a data access method according to any one of claims 1-10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211203998.6A CN115525614A (en) | 2022-09-29 | 2022-09-29 | Data access method, device, equipment, system and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211203998.6A CN115525614A (en) | 2022-09-29 | 2022-09-29 | Data access method, device, equipment, system and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115525614A true CN115525614A (en) | 2022-12-27 |
Family
ID=84698780
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211203998.6A Pending CN115525614A (en) | 2022-09-29 | 2022-09-29 | Data access method, device, equipment, system and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115525614A (en) |
-
2022
- 2022-09-29 CN CN202211203998.6A patent/CN115525614A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112184154A (en) | Business approval method and device | |
| CN111125496A (en) | Price query method, device and system | |
| CN116611411A (en) | Business system report generation method, device, equipment and storage medium | |
| CN115617888A (en) | Data import method, device, equipment, storage medium and product | |
| CN115421922A (en) | Current limiting method, device, equipment, medium and product of distributed system | |
| CN112579632A (en) | Data verification method, device, equipment and medium | |
| CN115578170A (en) | Financial batch certificate making method, device, equipment and storage medium | |
| CN114978944A (en) | Pressure testing method, device and computer program product | |
| CN115525614A (en) | Data access method, device, equipment, system and storage medium | |
| CN115422275A (en) | Data processing method, device, equipment and storage medium | |
| CN114218313A (en) | Data management method, device, electronic equipment, storage medium and product | |
| CN114091909A (en) | Collaborative development method, system, device and electronic equipment | |
| CN115795322B (en) | Railway configuration data comparison method, device, electronic equipment and storage medium | |
| CN117272970B (en) | Document generation method, device, equipment and storage medium | |
| CN115167855A (en) | Front-end page generation method, device and equipment applied to matching transaction system | |
| CN115658026A (en) | Service processing method, device, equipment and storage medium | |
| CN116382640A (en) | Method, device, equipment and storage medium for managing micro-service | |
| CN115834729A (en) | Data transmission method, device, equipment and medium | |
| CN115794609A (en) | Script sharing method and device, electronic equipment and storage medium | |
| CN115640345A (en) | Data processing method, device, system, equipment, storage medium and product | |
| CN114996243A (en) | Database operation method and device, electronic equipment and storage medium | |
| CN114896322A (en) | JSON (Java Server object notation) -based configuration data conversion method, device, equipment and medium | |
| CN118093958A (en) | Control method, device, equipment, storage medium and program product for data query | |
| CN114844920A (en) | Internet of things equipment checking method and device, electronic equipment and storage medium | |
| CN114911860A (en) | Data interface generation method, data access method, data interface generation device, data access device, electronic equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |