CN117149886A - Dynamically configurable multi-data source data docking method and management platform - Google Patents

Dynamically configurable multi-data source data docking method and management platform Download PDF

Info

Publication number
CN117149886A
CN117149886A CN202311427213.8A CN202311427213A CN117149886A CN 117149886 A CN117149886 A CN 117149886A CN 202311427213 A CN202311427213 A CN 202311427213A CN 117149886 A CN117149886 A CN 117149886A
Authority
CN
China
Prior art keywords
data
data source
interface
dynamically configurable
source
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311427213.8A
Other languages
Chinese (zh)
Inventor
陈溪源
蔡金煌
何伟坡
郭望
郭劲军
林起柄
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Xiamen Sunsharing Information Technology Co ltd
Original Assignee
Xiamen Sunsharing Information Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xiamen Sunsharing Information Technology Co ltd filed Critical Xiamen Sunsharing Information Technology Co ltd
Priority to CN202311427213.8A priority Critical patent/CN117149886A/en
Publication of CN117149886A publication Critical patent/CN117149886A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/25Integrating or interfacing systems involving database management systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/245Query processing
    • G06F16/2455Query execution
    • G06F16/24552Database cache management
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/445Program loading or initiating
    • G06F9/44505Configuring for program initiating, e.g. using registry, configuration files

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention relates to the technical field of data docking, in particular to a dynamically configurable multi-data source data docking method and a management platform, which comprise the following steps: adding a data source from a database or a data interface according to the requirements of a service system, and configuring dynamic data on the basis of the data source; defining a data source Key value and a type identifier through a visual interface according to the source type of a data source, and storing the data source Key value and the type identifier into a service system; and the service system calls corresponding data information to be provided for a calling party according to the Key value and the type identifier, and completes data butt joint. Therefore, by defining unique Key value and type identifier for each dynamic data, the service system directly calls each configured and verified dynamic data according to the Key value and the type identifier, the effect of instant use is achieved, the calling party only needs to adjust the corresponding configuration when having variation, no code is needed to be adjusted, and the use efficiency is improved.

Description

Dynamically configurable multi-data source data docking method and management platform
Technical Field
The invention relates to the technical field of data docking, in particular to a dynamically configurable multi-data source data docking method and a management platform.
Background
Currently there are two common ways of data interfacing: interface docking and database docking:
interface interfacing primarily provides an API interface by the service provider and platform, and developers use API integration to dynamically acquire the required data.
Database interfacing mainly adopts a database connection pool manner to connect with a target database and adopts SQL (structured query language) to query data.
However, in the conventional service system database and interface docking, the data of the same data interface/the same target database need to be repeatedly docked in different service systems; if the target data interface is adjusted, each service system needs to synchronously change codes, so that the method is time-consuming, labor-consuming, low in efficiency, easy to make mistakes, not provided with uniform docking specifications and unfavorable for uniform management of the interfaces.
It should be noted that the information disclosed in this background section is only for the purpose of increasing the understanding of the general background of the invention and should not be taken as an acknowledgement or any form of suggestion that this information forms the prior art already known to a person of ordinary skill in the art.
Disclosure of Invention
In order to solve the above-mentioned problems in the background art, the present invention provides a data docking method for dynamically configurable multiple data sources, which includes:
adding a data source from a database or a data interface according to the requirements of a service system, and configuring dynamic data on the basis of the data source;
defining a data source Key value and a type identifier through a visual interface according to the source type of a data source, and storing the data source Key value and the type identifier into a service system;
and the service system calls corresponding data information to be provided for a calling party according to the Key value and the type identifier, and completes data butt joint.
Further, adding a data source from the database comprises the steps of:
querying whether a connection object of the data source has been created according to the name of the data source: if the existence is present, directly entering the next step; if not, creating a new object and caching the new object in a memory;
according to the object type, calling a corresponding data query method by using a connection object;
and converting the format of the data according to the configuration, and if the calculation logic is configured, performing corresponding calculation logic.
Further, adding a data source from the data interface comprises the steps of:
acquiring an interface token;
carrying a token call to acquire a data interface;
and converting the format of the data according to the configuration, and if the calculation logic is configured, performing corresponding calculation logic.
Further, the configuration of the dynamic data includes accessibility configuration of a database type data source and a data interface type data source.
Further, based on the data source of the database type, according to the initial preset database data source connection configuration information, the connection information is registered in the connection pool, so that the dynamic data query script can be used for connecting and executing an external database based on the connection pool.
Further, based on the data source of the data interface type, the interface service is registered in the data interface according to the data source request message of the initially preset data interface and the initially preset dynamic data request message, so that the preset calling party accesses the service interface through the external network.
Further, the Key value is a unique value.
The invention also provides a dynamically configurable multi-data source data docking management platform for implementing the data docking method described in any one of the above.
Further, the dynamically configurable multiple data source data docking management platform comprises:
the data layer comprises a database, a data interface and a data cache;
the calling layer is used for realizing database calling inquiry and calling of a service interface, and supporting local authorization calling and unified calling logic, namely, an authorization code is in a project, and a request is realized through reflection calling;
and the processing layer performs logic operation on the call result.
Further, the database at least comprises mysql and oracle, and the data interface at least comprises a collaborative interface and a restful interface.
Based on the above, the method for interfacing data with multiple dynamically configurable data sources provided by the invention integrates two sources of data in a data layer, configures unique Key and identifier for each dynamic data, uniformly stores the Key and identifier in a service system, and each configured and verified dynamic data can be directly provided for the service system to be directly invoked, so that the service system can be used without writing interfacing codes again, and the service system can be opened for use; if the third party has adjustment, only the corresponding configuration is needed to be adjusted, the service system does not need to adjust any code, and the use efficiency is improved.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be apparent from the description. The objects and other advantages of the invention may be realized and obtained by means of the instrumentalities particularly pointed out in the specification and the like.
Drawings
For a clearer description of embodiments of the invention or of the solutions of the prior art, a brief description will be given below of the drawings that are needed in the description of the embodiments or of the prior art, it being obvious that the drawings in the description below are some embodiments of the invention, from which other drawings can be obtained, without the aid of inventive labour, for a person skilled in the art; in the following description, the positional relationship described in the drawings is based on the orientation of the components shown in the drawings unless otherwise specified.
FIG. 1 is a flow chart of a dynamically configurable multi-data-source data docking method according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a framework of a dynamically configurable data docking management platform for multiple data sources according to an embodiment of the present invention;
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more clear, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention; the technical features designed in the different embodiments of the invention described below can be combined with each other as long as they do not conflict with each other; all other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
In the description of the present invention, it should be understood that the terms "center," "lateral," "upper," "lower," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," and the like indicate orientations or positional relationships based on the orientation or positional relationships shown in the drawings, merely to facilitate describing the present invention and simplify the description, and do not indicate or imply that the devices or components referred to must have a specific orientation or be configured and operated in a specific orientation, and thus should not be construed as limiting the present invention. Furthermore, the terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include one or more such feature. In the description of the present invention, unless otherwise indicated, the meaning of "a plurality" is two or more. In addition, the term "comprising" and any variations thereof are meant to be "at least inclusive".
Example 1
In order to solve the above-mentioned technical problem of low efficiency, the present invention provides a dynamically configurable data docking method for multiple data sources, as shown in fig. 1, comprising the following steps:
adding a data source from a database or a data interface according to the requirements of a service system, and configuring dynamic data on the basis of the data source;
defining a data source Key value and a type identifier through a visual interface according to the source type of a data source, and storing the data source Key value and the type identifier into a service system;
and the service system calls corresponding data information to be provided for a calling party according to the Key value and the type identifier, and completes data butt joint.
Further, adding a data source from the database comprises the steps of:
querying whether a connection object of the data source has been created according to the name of the data source: if the existence is present, directly entering the next step; if not, creating a new object and caching the new object in a memory;
according to the object type, calling a corresponding data query method by using a connection object;
and converting the format of the data according to the configuration, and if the calculation logic is configured, performing corresponding calculation logic.
Further, adding a data source from the data interface comprises the steps of:
acquiring an interface token;
carrying a token call to acquire a data interface;
and converting the format of the data according to the configuration, and if the calculation logic is configured, performing corresponding calculation logic.
The invention also provides a data docking management platform based on multiple data sources, as shown in fig. 2, comprising:
the data layer comprises a database, a data interface and a data cache;
the calling layer is used for realizing database calling inquiry and calling of a service interface, and supporting local authorization calling and unified calling logic, namely, an authorization code is in a project, and a request is realized through reflection calling;
and the processing layer performs logic operation on the call result.
Further, the database comprises mysql, oracle and other databases, and the data interfaces comprise a collaboration interface, a restful interface and other interfaces.
Based on the signaling of the service system, dynamic data and data sources are managed, maintained and collected through a visual interface, and based on different data sources, the data sources are divided into two types: the interface type and the database type, both types define a data source type identification that identifies whether the data source is an interface or a database.
Dynamic data are maintained based on preset data source information, different types of data sources are different in configuration information to be maintained, each dynamic data needs to be defined with a unique Key value, the Key value is defined manually and automatically, and the Key value is unique and is stored in a service system.
The platform initiates a data acquisition request to a data layer, and queries whether a connection object of the data source is created according to the name of the data source at the database layer: if yes, directly calling a corresponding data query method; if not, creating a new object and caching the new object in a memory; according to the object type, calling a corresponding data query method by using a connection object; and converting the format of the data according to the configuration, wherein the system transmits a flag bit and a specific logic operation rule ID of whether logic operation is needed or not, and if the calculation logic is configured, the platform processing layer carries out corresponding calculation logic.
Acquiring an interface token at an interface data layer; carrying a token call to acquire a data interface; and converting the format of the data according to the configuration, wherein the system transmits a flag bit and a specific logic operation rule ID of whether logic operation is needed or not, and if the calculation logic is configured, the platform processing layer carries out corresponding calculation logic.
It should be noted that, the configuration of the calculation rule is implemented by configuring an expression evaluation script, and an Aviator expression engine of google open source is adopted, for example:
fn add(x, y) {
return x + y;
}
println(c);
println(count(c));
println(c[1]);
three = add(a, b);
println(three);
return three;
after the data source acquisition is completed, the platform calling layer performs accessibility configuration on the data source, and based on the data source of the interface type, the interface service is registered in the data interface according to an initial preset interface data source request message and an initial preset dynamic data request message, so that a preset calling party accesses a service interface through an external network.
Based on the data source of the database type, according to the initial preset database data source connection configuration information, the connection information is registered in the connection pool, so that the dynamic data query script can be used for connecting and executing an external database based on the connection pool.
After the accessibility configuration is completed, the call request message and the description information (comprising the unique identifier and the Key value) of the dynamic data are provided for a service system, the service system carries the identifier Key and the actual request message to call the dynamic data, the data source type is identified according to the data source identifier associated with the dynamic data, and the external data is called and accessed.
From the aspect of performance, if a query interface is provided, the data returned by each query is the same, a buffer memory within a certain time range can be set, for example, 1 day, and the data query and call within the 1 day can be directly and simply and quickly carried out through the data buffer memory without running the whole call flow.
In addition, the data interface can be called by the request after authorization, so the calling layer also supports the realization of the authorization requests of different butting parties, and then unified calling is performed.
Example two
Assuming that the calling party is in butt joint with the employment information database, inquiring the personnel information of employment for more than three months:
the platform adds the data source configuration information of the employment information database, wherein the data source type is a database, a connection string, a user account number password and the like;
the dynamic data configuration information of personnel with employment more than three months is newly added, and comprises a data source type identifier which is a database, wherein the data source selects a employment information database, and a unique Key is an overThreeMontthsPeole, a database script is inquired, and a message is requested;
the service system designates the method for acquiring the dynamic data provided by the identification Key and the actual request message calling platform, and the platform acquires the data source information related to the dynamic data and the configuration according to the dynamic data Key, acquires the data source configuration information of the employment information database, performs database connection and calls the data to return the result.
And the platform converts the calling result according to the result configuration, returns the converted calling result to the service system, and provides the service system for the calling party to complete the data butt joint query.
Example III
Assuming that the calling party is in butt joint with the public security household registration population information interface, inquiring public security household registration population information:
the platform adds data source configuration information of public security household registration population information interfaces, wherein the data source configuration information comprises data source type identifiers such as interfaces, interface addresses, request messages Wen Moban and the like;
dynamic data configuration information for newly inquiring public security registration personnel information is added: the method comprises the steps of taking the type of a data source as an interface, selecting employment information database by the data source, and requesting message parameter configuration information by a unique Key;
the service system designates the method for acquiring dynamic data provided by the parameter information calling platform in the identification Key and the request message, the platform acquires the data source information related to the dynamic data and configuration according to the dynamic data Key, acquires a request interface and a request message template of the data source configuration information of the public security registration personnel information interface, converts the request message, calls the public security registration personnel information interface and returns a query result;
and the platform converts the calling result according to the result configuration, returns the converted calling result to the service system, and provides the service system for the calling party to complete the data butt joint query.
In summary, the dynamically configurable multi-data source data docking method and management platform provided by the invention integrate two sources of data in a data layer, and configure unique Key and identifier for each dynamic data, the Key and identifier are uniformly stored in a service system, each configured and verified dynamic data can be directly provided for the service system to be directly invoked, and the service system does not need to write docking codes again, so that the service system can be used after unpacking; if the third party has adjustment, only the corresponding configuration is needed to be adjusted, the service system does not need to adjust any code, and the use efficiency is improved.
In addition, it should be understood by those skilled in the art that although many problems exist in the prior art, each embodiment or technical solution of the present invention may be modified in only one or several respects, without having to solve all technical problems listed in the prior art or the background art at the same time. Those skilled in the art will understand that nothing in one claim should be taken as a limitation on that claim.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the invention.

Claims (10)

1. A dynamically configurable data docking method for multiple data sources is characterized in that: the dynamically configurable multi-data source data docking method comprises the following steps:
adding a data source from a database or a data interface according to the requirements of a service system, and configuring dynamic data on the basis of the data source;
defining a data source Key value and a type identifier through a visual interface according to the source type of a data source, and storing the data source Key value and the type identifier into a service system;
and the service system calls corresponding data information to be provided for a calling party according to the Key value and the type identifier, and completes data butt joint.
2. The dynamically configurable multiple data source data docking method of claim 1, wherein: adding a data source from a database comprises the steps of:
querying whether a connection object of the data source has been created according to the name of the data source: if the existence is present, directly entering the next step; if not, creating a new object and caching the new object in a memory;
according to the object type, calling a corresponding data query method by using a connection object;
and converting the format of the data according to the configuration, and if the calculation logic is configured, performing corresponding calculation logic.
3. The dynamically configurable multiple data source data docking method of claim 1, wherein: adding a data source from a data interface comprises the steps of:
acquiring an interface token;
carrying a token call to acquire a data interface;
and converting the format of the data according to the configuration, and if the calculation logic is configured, performing corresponding calculation logic.
4. The dynamically configurable multiple data source data docking method of claim 1, wherein: the configuration of the dynamic data includes accessibility configuration of a database type data source and a data interface type data source.
5. The dynamically configurable multiple data source data docking method of claim 4, wherein: based on the data source of the database type, according to the initial preset database data source connection configuration information, the connection information is registered in a connection pool, so that the dynamic data query script can be used for connecting and executing an external database based on the connection pool.
6. The dynamically configurable multiple data source data docking method of claim 4, wherein: based on the data source of the data interface type, registering interface service into the data interface according to a data source request message of an initial preset data interface and an initial preset dynamic data request message, so that a preset calling party accesses a service interface through an external network.
7. The dynamically configurable multiple data source data docking method of claim 1, wherein: the Key value is a unique value.
8. A dynamically configurable multiple data source data docking management platform, characterized by: the dynamically configurable multiple data source data docking management platform is configured to implement the data docking method of any one of claims 1-7.
9. The dynamically configurable multiple data source data docking management platform of claim 8, wherein: the dynamically configurable multi-data source data docking management platform comprises:
the data layer comprises a database, a data interface and a data cache;
the calling layer is used for realizing database calling inquiry and calling of a service interface, and supporting local authorization calling and unified calling logic, namely, an authorization code is in a project, and a request is realized through reflection calling;
and the processing layer performs logic operation on the call result.
10. The dynamically configurable multiple data source data docking management platform of claim 9, wherein: the database at least comprises mysql and oracle, and the data interface at least comprises a collaborative interface and a restful interface.
CN202311427213.8A 2023-10-31 2023-10-31 Dynamically configurable multi-data source data docking method and management platform Pending CN117149886A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311427213.8A CN117149886A (en) 2023-10-31 2023-10-31 Dynamically configurable multi-data source data docking method and management platform

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311427213.8A CN117149886A (en) 2023-10-31 2023-10-31 Dynamically configurable multi-data source data docking method and management platform

Publications (1)

Publication Number Publication Date
CN117149886A true CN117149886A (en) 2023-12-01

Family

ID=88912523

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311427213.8A Pending CN117149886A (en) 2023-10-31 2023-10-31 Dynamically configurable multi-data source data docking method and management platform

Country Status (1)

Country Link
CN (1) CN117149886A (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040133543A1 (en) * 2002-07-26 2004-07-08 Shlaes Marc J. Method and system of unifying data
CN111198901A (en) * 2020-01-06 2020-05-26 北京邮电大学 Unified data service opening method and system based on multiple data sources
CN115269629A (en) * 2022-06-30 2022-11-01 启明信息技术股份有限公司 Data query method and system supporting multiple data sources
CN115309566A (en) * 2022-08-09 2022-11-08 医利捷(上海)信息科技有限公司 Dynamic management method and system for service interface
CN115374116A (en) * 2022-08-22 2022-11-22 中国铁建重工集团股份有限公司 High-allocable multi-data-source instant heterogeneous fusion method and system

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040133543A1 (en) * 2002-07-26 2004-07-08 Shlaes Marc J. Method and system of unifying data
CN111198901A (en) * 2020-01-06 2020-05-26 北京邮电大学 Unified data service opening method and system based on multiple data sources
CN115269629A (en) * 2022-06-30 2022-11-01 启明信息技术股份有限公司 Data query method and system supporting multiple data sources
CN115309566A (en) * 2022-08-09 2022-11-08 医利捷(上海)信息科技有限公司 Dynamic management method and system for service interface
CN115374116A (en) * 2022-08-22 2022-11-22 中国铁建重工集团股份有限公司 High-allocable multi-data-source instant heterogeneous fusion method and system

Similar Documents

Publication Publication Date Title
CN108717861B (en) Medical data sharing method based on block chain
CN107480198B (en) Distributed NewSQL database system and full-text retrieval method
CN109087004B (en) Public workflow engine system based on domain model
US6601071B1 (en) Method and system for business to business data interchange using XML
US8045995B2 (en) Centralized location broker
US5995946A (en) System and method for establishing and managing links among customer accounts maintained within a telecommunications system
US9342572B2 (en) Workflow processing system and method with database system support
US7213208B2 (en) Data container for interaction between a client process and software applications
JP3282652B2 (en) OSI multi-layer management system
US11966384B2 (en) Generating external identifiers for data entities using a data catalog system
US9515948B2 (en) Techniques for generically accessing data
CN110049503B (en) Method and equipment for acquiring data
CN103946841A (en) Systems and methods for dynamic service integration
CN102054032A (en) Medical data information processing method and system
CN110209719A (en) A kind of multitype database based on micro services framework uniformly accesses system and method
JPH09508734A (en) Distributed database system
US8161075B1 (en) Systems and methods for managing integrated and customizable data
CN111381820A (en) Method and device for automatically generating API based on GUI
CN116955399A (en) Unified SQL query method, system and medium based on Calcite
CN111064602B (en) Resource quota management method under public cloud platform
CN106028311A (en) Terminal register method and device
CN117149886A (en) Dynamically configurable multi-data source data docking method and management platform
CN115686497A (en) Business development data management method, development engine, electronic device, and storage medium
CN107145606A (en) A kind of data managing method and device
WO2022111356A1 (en) Data migration method and system, and server and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination