CN117271441A

CN117271441A - Real-time data network system and method for inquiring and processing distributed data

Info

Publication number: CN117271441A
Application number: CN202210667295.2A
Authority: CN
Inventors: 沈唐秀蓉
Original assignee: Aikunde Beijing Technology Co ltd
Current assignee: Aikunde Beijing Technology Co ltd
Priority date: 2022-06-13
Filing date: 2022-06-13
Publication date: 2023-12-22

Abstract

The invention discloses a real-time data network system and a method for inquiring and processing distributed data, relates to the technical field of distributed data, and aims to solve the difficult problems of inquiring and processing large-scale distributed multi-structure real-time data in real time. The system comprises: one or more invocation systems; and one or more access enforcement systems; each calling system is connected with one or more access execution systems respectively; each access execution system is respectively connected with one or more data storage systems; each calling system is used for providing an interface for accessing and processing distributed data for an external application program, receiving an instruction for accessing and processing the distributed data from the external application program through the interface for accessing and processing the distributed data, and accessing and processing the distributed data stored by the data storage system through one or more access execution systems connected with the interface.

Description

Real-time data network system and method for inquiring and processing distributed data

Technical Field

The invention relates to the technical field of distributed data processing, in particular to a real-time data network system and a method for inquiring and processing large-scale distributed multi-structure real-time data (comprising unstructured data, semi-structured data and structured data).

Background

Information present in an information-based society can be divided into structured data, semi-structured data, and unstructured data. Structured data is data organized in a Structured Query Language (SQL) database, has a certain logical structure and physical structure, and is generally data realized by logically expressing a two-dimensional table structure; semi-structured data refers to data that has a degree of structure and is stored in a non-relational (noSQL) database or document (e.g., a spreadsheet). The semi-structured data structure is greatly changed, and analysis processing cannot be performed completely according to the unstructured or structured data processing mode; unstructured data has no predictable structure and is stored in any document without a specific structure, such as images, audio/video information, etc.

Achieving sharing and processing of multi-structure data (structured, semi-structured, and/or unstructured data) is an urgent need in the information-based society. Many geographically separated organizations, or companies operating separately but requiring collaboration, need to share real-time data. For example, within a large enterprise group, a subsidiary needs to allow a headquarter to retrieve and review its financial data and other files; large banks need to collect data from branches and synchronize the data to a central unit; the healthcare facility needs to share patient medical data and files; business partners share certain data in B2B mode in real time, etc., as required by the contract. Such information to be shared includes not only structured data, but also more and more unstructured data and semi-structured data, which are required to be shared and processed in real time.

However, currently used data sharing solutions do not solve the problem of sharing multi-structure real-time data well, and in many cases the demand cannot be met. The difficulty is mainly how to process data and documents stored by using different technologies and different data structure formats, and eliminate differences on a standard network platform to realize seamless connection of data. Another challenge is the limitation of Information Technology (IT) resources, which in ideal cases can be collected and processed at a central unit when needed if the network bandwidth, storage and computing resources are unlimited. However, in the real world, this is not the case. For example, when it is desired to search and extract information from a large number of documents containing semi-structured and unstructured data, it is practically impossible to collect all the documents to a central unit. Instead, the request needs to be processed at the edge where the data resides.

Data sharing is an important component in management work between enterprises and within large enterprise groups, so that scattered solutions designed for these institutions have emerged, which share data in a semi-real-time or non-real-time manner, and only meet specific application requirements. The user eventually has to build multiple solutions custom to handle different usage scenarios separately. None of these solutions solves the real-time sharing and processing problems of large-scale multi-structure data. The following are several typical solutions currently used by large organizations:

Non-real-time electronic means: in some large enterprise groups, to evaluate risk and aggregate financial data, the large enterprise groups design forms and provide them to their subsidiaries for periodic filling. Enterprises or subsidiaries fill in the data into forms and submit them to the management authorities or headquarters on a regular basis. The frequency of data submission may be daily, weekly, monthly, quarterly, or yearly, and the business may submit filled forms through a web site or FTP web site. In some cases, online data entry may be performed using a spreadsheet.

This solution has many drawbacks, one of which is the inability to directly access real-time data, which is only suitable for regular data submissions in the manner in which data producers push data to data consumers, and does not support the use of scenarios in which data consumers autonomously acquire data from the data producers. Another drawback is that this solution causes the enterprise to expend a great deal of manual labor to prepare the data and ensure that the data is submitted successfully. In many cases, the data must be manually entered and verified, and the data package must be decompressed first after it reaches the central unit, after which the data is cleaned up and entered into the central database. Because data can only be submitted periodically, and because of network and centralized storage resource limitations, the amount of data is typically small. The data collected in this way cannot be shared in real time and is generally not suitable for complex analysis.

Structured data is shared in real time through REST SQL: in a B2B environment, an agreement may also be reached between enterprises, allowing one party to utilize the data of the other party. This type of usage scenario requires that the data producer allow the data consumer to automatically retrieve the data via a query or download.

In the above usage scenario, organizations and organizations that produce data typically provide portions of the data through REST SQL Application Programming Interfaces (APIs) for sharing with the public or business partners. In this way, the public and B2B partners can spontaneously issue data query requests as needed.

However, this solution has a significant technical limitation: often, data producers will only work well if they are small. For example, only one data producer provides data to the public, or two partners share data with each other.

Sharing data through REST SQL API can be very difficult if there are many data producers. This is because, lacking a single standard REST SQL API, all database providers have to create their own SQL access interfaces.

As shown in fig. 1, which is an example of sharing structured data and semi-structured data in a conventional manner, for example, a data owner would typically synchronize a subset of its data into a PG or MySQL database and then use REST SQL API to open share its replicated data, the disadvantage of this approach is also apparent, firstly the shared data is non-real-time, and the open source software REST SQL API used by PG and MySQL has stopped development since 2018 and does not support orderby, groupby, etc. instructions. The business REST SQL API is accomplished independently by different database providers without following a particular standard. In the multiple partner B2B scenario, unless all data producers use exactly the same business product, the collaboration network may cease to work because REST SQL data query APIs created using different vendors are not identical. This solution does not support the sharing of semi-structured and unstructured data.

Other existing database tools also face similar problems in handling data sharing, e.g., APIs developed by data producers using Oracle are non-standard, self-developed, and therefore cannot support unified queries for real-time multi-structure data.

Combination of structured data aggregation and real-time document access: most conventional data sharing solutions are created for sharing structured data. These solutions cannot support sharing of multi-structure data. The multi-structure data includes structured, semi-structured, and unstructured text and media content, where the addresses of unstructured content files are stored in structured and semi-structured databases.

Fig. 2 is an example of a conventional method of accessing distributed structured data and unstructured data. In this solution, structured and semi-structured data are integrated into the central unit, but unstructured data files are still stored distributed and are not integrated into the central unit. The reason for not integrating unstructured data files into a central unit is due to cost and technical considerations. Integrating a large number of files requires significant storage and network resource investment. Furthermore, since the document links are stored as structured data in the database, once the file is relocated to the central unit, these links stored in the database must be updated to the new central unit. Updating a database is not only a huge task, but may also destroy the database.

In this solution, structured and semi-structured data 0224-a, b, c are periodically integrated into a central database-fusion library 0232 by various methods. One of the data integration methods is to have each data generating node 0220-a, b, c prepare incremental data packets 0226-a, b, c in a particular format. The incremental data packets are then submitted 0228 to the centralized server 0230 by pre-integrating the structured data. At the centralized server 0230, these packets are collected, cleaned and fused to finally integrate into a central database-fusion library 0232. As shown in FIG. 2, the application 0210 may call structured and semi-structured data 0212, including structured information, document metadata, and links, to a fusion library 0232. Because the addresses of the distributed documents are stored in the structured database, the addresses of the unstructured data files can be extracted from the fusion library 0232. As long as these files can be accessed through some file sharing service 0222-a, b, c (e.g., webDAV, HTTP, or FTP), the application 0210 can retrieve the unstructured files stored in the distributed file storage file system 0225-a, b, c by way of the direction of the unstructured data file address, call 0214-a, b, c.

A significant problem with this solution is its semi-real time data sharing nature. Structured and semi-structured data tend to be out of sync with unstructured data files because the data in the structured and semi-structured databases is collected periodically, while unstructured data files are in real-time. When files are relocated in the distributed data storage file system 0225-a, b, c, their links in the central database-fusion repository 0232 are broken. Because unstructured data changes in real time, the content and the address of the distributed document also change, but the distributed document address originally stored in the structured database cannot directly correspond to the real-time change condition because of periodic collection, and the unstructured data cannot be called by using the wrong document address information.

This solution also often places a lot of workload on the IT personnel of the data producer and the central data consumer. When an error occurs in database integration, the process of rebuilding the central database will result in a long data downtime.

Disclosure of Invention

The embodiment of the disclosure provides a real-time data network system and a method for inquiring and processing distributed data, relates to the technical field of distributed data, and can solve the difficult problem of inquiring and processing large-scale distributed multi-structure real-time data in real time.

The real-time data network system according to the embodiment of the disclosure may include:

one or more invocation systems; and

one or more access enforcement systems; wherein,

each calling system is respectively connected with one or more access execution systems;

each access execution system is respectively connected with one or more data storage systems;

each calling system is used for providing an interface for accessing and processing distributed data for an external application program, receiving an instruction for accessing and processing the distributed data from the external application program through the interface for accessing and processing the distributed data, and accessing and processing the distributed data stored by the data storage system through one or more access execution systems connected with the interface.

In some embodiments of the present disclosure, each calling system installs a first plug-in service and a second plug-in service; wherein, the calling system is used for: receiving a first application programming interface instruction sent by the external application program through the interface for accessing and processing the distributed data; determining an execution place of the first application programming interface instruction; in response to determining that the execution site is an access execution system, initiating a first plug-in service associated with the first application programming interface instruction to cause the calling system to send second application programming interface instructions to one or more access execution systems associated therewith and access distributed data stored by the data storage system through the one or more access execution systems; and in response to determining that the execution place is a calling system, starting a second plug-in service program related to the first application programming interface instruction, acquiring data according to data processing logic set by the second plug-in service program, processing the acquired data, and feeding the processed data back to the external application program through the interface for accessing and processing the distributed data.

In some embodiments of the present disclosure, each access execution system is installed with a third plug-in service; the access execution system receives a second application programming interface instruction sent by the calling system, and starts a third plug-in service program related to the second application programming interface instruction, so that the access execution system reads the distributed data from the data storage system, and processes the distributed data according to data processing logic set by the third plug-in service program to obtain first response data.

In some embodiments of the present disclosure, each calling system is installed with a fourth plug-in service; the calling system is further used for receiving the first response data from the access execution system, starting a fourth plug-in service program related to the first application programming interface instruction, processing the first response data to obtain second response data, and feeding back the second response data to the external application program through the interface for accessing and processing the distributed data.

In some embodiments of the present disclosure, the access enforcement system is configured with a data adapter; the access execution system matches the distributed data through the data adapter and reads the distributed data from the data storage system.

In some embodiments of the present disclosure, the access execution system is further installed with a data integration program; the access execution system converts the data model of the distributed data into the data model required by the third plug-in service by starting the data integration program.

In some embodiments of the present disclosure, each access execution system installs a third plug-in service and a fifth plug-in service; the access execution system receives a second application programming interface instruction sent by the calling system and starts a fifth plug-in service program related to the second application programming interface instruction; the fifth plug-in service program calls the third plug-in service program so that the access execution system reads the distributed data from the data storage system and processes the distributed data according to the data processing logic set by the third plug-in service program to obtain intermediate data; and processing the intermediate data according to the data processing logic set by the fifth plug-in service program to obtain first response data.

In some embodiments of the present disclosure, the system described above further comprises: an intermediate processing node and an intermediate storage system; the intermediate processing node is connected with a calling system and one or more access execution systems, and is used for receiving the instructions for accessing and processing the distributed data from the calling system, sending the instructions for accessing and processing the distributed data to the one or more access execution systems, receiving response data from the one or more access execution systems, processing the response data and sending the response data to the calling system; the intermediate storage system is used for storing response data processed by the intermediate processing node.

In some embodiments of the present disclosure, the above system further comprises:

an intermediate layer access execution system;

a middle layer calling system; and

a middle layer data storage system; wherein,

the access execution system of the middle layer is connected with a calling system, the calling system of the middle layer and the data storage system of the middle layer;

and the calling system of the middle layer is connected with one or more access execution systems.

In some embodiments of the present disclosure, the above system further comprises: the data network plug-in service management system is used for managing all plug-in service programs installed in the calling system and the access execution system; wherein the data network plug-in service management system comprises: a plug-in service programming framework for providing a programming framework for developing plug-in service packages for developers; and the program management module is used for managing and releasing the plug-in service program package developed by the developer so that the calling system and the access execution system install the plug-in service program package developed by the developer.

In some embodiments of the present disclosure, the access enforcement system includes: a data integration programming framework for providing a programming framework for developing a data integration program for a developer; and a data adapter configuration framework for providing an interface for a developer to enter an adapter name field, select and reference a data adapter micro-service, and enter address and access information fields of a data storage system.

In some embodiments of the present disclosure, the above system further comprises: the data network management system is used for managing each system in the real-time data network; wherein the data network management system comprises: the calling system and the access execution system account management module are used for creating or deleting the calling system and the access execution system; and the calling system and the access execution system network topology management module is used for adding or deleting the calling system and the access execution system in the real-time data network system and managing the network connection relationship between the calling system and the access execution system.

In some embodiments of the present disclosure, the above system further comprises: a plug-in service programming framework for providing a programming template for developing a plug-in service package for a developer; wherein the programming template allows the developer to define an Application Program Interface (API) of each plug-in service in the plug-in service package, define execution sites of each plug-in service in the plug-in service package, and create code for each plug-in service executed in the calling system and/or the access execution system; wherein the API is programming interface instructions used by the external application.

Corresponding to the real-time data network system, the embodiment of the disclosure also discloses a method for querying and processing distributed data, which is applied to the real-time data network system, wherein the real-time data network system comprises: one or more invocation systems; and one or more access enforcement systems; wherein the method of querying and processing distributed data comprises: each calling system receives an access instruction from an external application program and an instruction for processing distributed data respectively; and each calling system accesses and processes the distributed data stored by the data storage system through one or more access execution systems connected with the calling system.

In some embodiments of the present disclosure, the instructions for each calling system to receive access from an external application and process distributed data include: the calling system receives a first application programming interface instruction sent by the external application program through an interface for accessing and processing the distributed data; and each calling system accesses and processes the distributed data stored by the data storage system through one or more access execution systems connected with the calling system, wherein the distributed data comprises: the calling system determines the execution place of the first application programming interface instruction; in response to determining that the execution site is an access execution system, the calling system initiates a first plug-in service associated with the first application programming interface instruction, sends second application programming interface instructions to one or more access execution systems associated with the first plug-in service, and accesses distributed data stored by the data storage system through the one or more access execution systems; and in response to determining that the execution place is a calling system, starting a second plug-in service program related to the first application programming interface instruction, acquiring data according to data processing logic set by the second plug-in service program, processing the acquired data, and feeding the processed data back to the external application program through the interface for accessing and processing the distributed data.

In some embodiments of the present disclosure, the accessing, by the one or more access enforcement systems, the distributed data stored by the data storage system includes: the access execution system receives a second application programming interface instruction sent by the calling system; starting a third plug-in service program related to the second application programming interface instruction; reading the distributed data from the data storage system; and processing the distributed data according to the data processing logic set by the third plug-in service program to obtain first response data.

In some embodiments of the present disclosure, the accessing, by the one or more access enforcement systems, the distributed data stored by the data storage system further comprises: the calling system receives the first response data from the access execution system, starts a fourth plug-in service program related to the first application programming interface instruction, processes the first response data according to data processing logic set by the fourth plug-in service program to obtain second response data, and feeds the second response data back to the external application program through the interface for accessing and processing the distributed data.

In some embodiments of the present disclosure, the accessing, by the one or more access enforcement systems, the distributed data stored by the data storage system further comprises: and the third plug-in service program of the one or more access execution systems converts the data model of the distributed data into the data model required by the third plug-in service program by starting the data integration program.

In some embodiments of the present disclosure, the accessing, by the one or more access enforcement systems, the distributed data stored by the data storage system includes: the access execution system receives a second application programming interface instruction sent by the calling system; starting a fifth plug-in service related to the second application programming interface instruction; the fifth plug-in service program calls a third plug-in service program; reading the distributed data from the data storage system; processing the distributed data according to the data processing logic set by the third plug-in service program to obtain intermediate data; and processing the intermediate data according to the data processing logic set by the fifth plug-in service program to obtain first response data.

In some embodiments of the present disclosure, the above method further comprises: acquiring an account of a calling system and the topological position of the calling system in a real-time data network system; registering a calling system in a real-time data network management system according to the calling system account and the topological position of the calling system in the real-time data network system; and receiving a first security token corresponding to the calling system from the real-time data network management system; determining whether the calling system has an associated access execution system according to the topological position of the calling system in a real-time data network system; and in response to determining that the associated access execution systems exist, notifying all of the associated access execution systems to be associated with the calling system, and establishing a communication connection between the access execution system and the calling system.

In some embodiments of the present disclosure, the above method further comprises: acquiring an account of an access execution system and a topological position of the access execution system in a data network system; registering an access execution system in a real-time data network management system according to the account of the access execution system and the topological position of the access execution system in the data network system; receiving a second security token corresponding to the access execution system from the real-time data network management system; and associating the access execution system to a calling system associated with the access execution system based on the topological position of the access execution system in the data network system, and establishing communication connection between the access execution system and the calling system.

In some embodiments of the present disclosure, establishing a communication connection between the access execution system and the invocation system includes: the access execution system provides the second security token to the calling system; the calling system provides a first access token to the access execution system; the access execution system provides a second access token to the upper layer calling system; wherein the first access token and the second access token are used for communication between the calling system and the access execution system.

In some embodiments of the present disclosure, the above method further comprises: sending an installation request to a calling system to install the plug-in service package; the calling system installs all or part of plug-in service programs in the plug-in service program package into a pre-installed program library; the calling system informs all the associated access execution systems to install the plug-in service package; each access execution system installs all or part of plug-in service programs in the plug-in service program package into an installation program library of the plug-in service program package; the access execution systems each configure a data adapter for the plug-in service to access distributed data stored by a data storage system connected to the access execution system and associate the plug-in service with the data adapter.

In some embodiments of the present disclosure, the above method further comprises: each access execution system provides a programming framework of a data integration program and acquires the data integration program from the programming framework of the data integration program; the data integration program is used for converting the data model of the distributed data into a data model required by the third plug-in service program; the third plug-in service is associated with the data integration program.

In some embodiments of the present disclosure, the above method further comprises: providing a programming template for developing a plug-in service package for a developer; wherein the programming template allows the developer to define an Application Program Interface (API) of each plug-in service in the plug-in service package, define execution locations of each plug-in service in the plug-in service package, and write code of each plug-in service executed in the calling system and/or the access execution system; wherein the API is programming interface instructions used by the external application.

According to the technical scheme provided by the embodiment of the invention, unified multi-structure data format and matching of the regional data storage mode are realized through joint debugging of the plug-in service program, the integrated program and the data adapter, and the purpose of accessing and processing large-scale distributed data in real time is realized through cooperation of the calling system and the access execution system, so that the problem that the existing data sharing solution cannot well solve the problem of sharing multi-structure distributed real-time data is solved.

Drawings

In order to more clearly illustrate the technical solutions of the present disclosure or related art, the drawings required for the embodiments or related art description will be briefly described below, and it is apparent that the drawings in the following description are only embodiments of the present disclosure, and other drawings may be obtained according to these drawings without inventive effort to those of ordinary skill in the art.

FIG. 1 is an example of a conventional structured and semi-structured data sharing method;

FIG. 2 is an example of a conventional method of accessing distributed structured and unstructured data;

FIG. 3 is a system component of one of the architectural embodiments of a real-time data network;

FIG. 4 is a first embodiment of a real-time data network system component layout;

FIG. 5 is a second embodiment of a real-time data network system component layout;

FIG. 6 is a third embodiment of a real-time data network system component layout;

FIG. 7 is a fourth embodiment of a real-time data network system component layout comprising two data sharing networks and a service management system;

FIG. 8 is a fifth embodiment of a real-time data network system component layout including a plurality of data sharing networks with service management systems;

FIG. 9 is an embodiment of a layout of components of a real-time data network system having a multi-layer data sharing network;

FIG. 10 is a diagram of one embodiment of a function of a real-time data network management system;

FIG. 11 is a diagram of one embodiment of a real-time data network plug-in service management system functionality;

FIG. 12 is one of the call system and access enforcement system architecture functional implementation of a real-time data network;

FIG. 13 is one embodiment of a real-time data network plug-in service programming framework;

FIG. 14 is a second embodiment of a real-time data network plug-in service programming framework;

FIG. 15 is an example of a scenario in which distributed multi-structure data is invoked in real-time;

FIG. 16 is an example of a scenario in which distributed multi-structure data is processed in real-time;

FIG. 17 is a diagram of an embodiment of a method of communication of components of a real-time data network system;

FIGS. 18a and 18b are, respectively, embodiments of a method for forming a real-time data network by adding a calling system and an access execution system;

FIGS. 19a and 19b are embodiments of a method for managing a real-time data network by deleting a calling system and an access execution system, respectively;

FIGS. 20a and 20b are, respectively, embodiments of a method of installing and integrating plug-in services into a real-time data network;

FIGS. 21a, 21b and 21c are each an embodiment of a method of executing a plug-in service in a real-time data network;

FIG. 22 is an example of a real-time data network application to a large organization;

FIG. 23 is an example of a network configuration when a real-time data network is applied to a large organization;

FIG. 24 is an example of a real-time data call network composition applied to a large organization;

FIG. 25 is an example of performing data retrieval and analysis processing over a real-time data network;

fig. 26 is an example of an application of a real-time data network to the field of internet of things.

Detailed Description

For the purposes of promoting an understanding of the principles and advantages of the disclosure, reference will now be made to the embodiments illustrated in the drawings and specific language will be used to describe the same.

It should be noted that unless otherwise defined, technical or scientific terms used in the embodiments of the present disclosure should be given the ordinary meaning as understood by one of ordinary skill in the art to which the present disclosure pertains. The terms "first," "second," and the like, as used in embodiments of the present disclosure, do not denote any order, quantity, or importance, but rather are used to distinguish one element from another. The word "comprising" or "comprises", and the like, means that elements or items preceding the word are included in the element or item listed after the word and equivalents thereof, but does not exclude other elements or items. The terms "connected" or "connected," and the like, are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "upper", "lower", "left", "right", etc. are used merely to indicate relative positional relationships, which may also be changed when the absolute position of the object to be described is changed.

Example 1

As shown in FIG. 3, the present invention provides an embodiment of a system for querying or processing distributed data in a coordinated manner. It should be noted that, in embodiments of the present disclosure, the distributed data described above may refer to distributed data stored at one or more geographic locations.

In the above embodiments, the above system may be referred to as a real-time data network 0310. Specifically, the real-time data network 0310 may be composed of four system modules as follows: a data network management system 0311, a calling system 0313 (hereinafter may be simply referred to as GDDS), an access execution system 0314 (hereinafter may be simply referred to as DSS), and a data network plug-in service management system 0315.

Wherein the data network plug-in service management system 0315 manages plug-in services, which allows developers to publish and install plug-in services on the data access and processing network 0312, which is composed of the calling system 0313 and the access executing system 0314.

It should be noted that the above-described embodiments organize these functions into one system composed of the above-described four systems. In other embodiments of the present disclosure, the above-described functions may also be reorganized in more or less than four systems, for example, a real-time data network 0310 composed of three systems of a call-only system 0313, an access execution system 0314, and a data network plug-in service management system 0315 may be constructed.

Nodes distributed in various geographical locations may comprise a data storage system 0320 connected to an access enforcement system 0314, the data access functions being performed by the access enforcement system 0314. The data storage system 0320 stores distributed multi-structure data including structured data 0321, unstructured data (e.g., files) 0322, and semi-structured data 0323. The number of the access execution systems 0314 may be plural, and they are deployed in different geographical locations. The access execution system 0314 is configured with a data adapter and/or a data integration program, so as to implement a real-time access function of distributed multi-structure data. The invocation system 0313 provides a unified interface for external applications 0330 to unified query and invoke real-time data. The interface may be an interface for accessing and processing distributed data. The calling system 0313 is connected to the access execution system 0314, and transmits a query and a data retrieval instruction issued by the external application 0330 to the access execution system 0314 located at one or more sites, so as to call and process the multi-structure data in the data storage system 0320 connected to the access execution system 0314 in real time. That is, the calling system 0313 may receive an instruction to access and process distributed data from the external application 0330 through the above-described interface to access and process distributed data, and access and process distributed data stored in the data storage system 0320 through one or more access execution systems 0314 connected thereto.

A developer may write plug-in service programs 0317-a, b, c using a plug-in service programming framework 0318 in the data network plug-in service management system 0315 and install into the data access and processing network 0312 for real-time data access and processing. The data access and processing network 0312 consists of a calling system 0313 and an access execution system 0314. The plug-in service 0317-a, b, c may be installed in the calling system 0313 and the access execution system 0314, or may be installed in only the calling system 0313. Plug-in service 0317-a, b, c may also install only part of the program into calling system 0313 and part of the program into access execution system 0314. The developer written services are called plug-in services 0317-a, b, c, which plug-in services 0317-a, b, c provide APIs for external applications 0330, such that the external applications 0330 can trigger real-time execution functions of the installed plug-in services 0317-a, b, c to query, retrieve and process real-time data.

The data network management system 0311 is responsible for managing all other modules or components in the real-time data network 0310.

The above-described fig. 3 is but one embodiment of the present disclosure. Furthermore, the system components may be flexibly organized, integrated, or partitioned to form different real-time data sharing and processing networks. Additional embodiments are provided in fig. 4-9 to illustrate how different network configurations may be implemented.

According to the technical scheme provided by the embodiment of the invention, the data files with multiple structures can be flexibly processed by providing the data network plug-in module, so that the distributed multi-structure data can be accessed and processed in real time, and the problem that the large-scale distributed multi-structure data in the prior art cannot be accessed and processed in real time is solved.

Example two

As shown in FIG. 4, the present invention provides a first embodiment of a layout of a system for querying or processing distributed data in a coordinated manner. This embodiment illustrates one layout of the real-time data network system depicted in fig. 3. As shown in fig. 4, the real-time data network has three access execution system nodes. Of course, any number of access execution system nodes may be set according to actual situations. Wherein each access performing system 0314-a, b, c may be deployed in a different geographical location, respectively. At each access enforcement system node, each access enforcement system 0314-a, b, c is connected to a local data storage system 0320-a, b, c, respectively, where the multi-structure data is stored. In this embodiment, the real-time data network has two invocation systems 0313-a, b. Of course, a single calling system may be clustered, or more calling systems may be provided. Each calling system can provide an API interface for an external application 0420, so that the purposes of calling and processing multi-structure data in real time are realized. The application 0420 may call and remotely process distributed data connected to the access execution system 0314-a, b, c by calling the system 0313-a, b.

In the arrangement shown in this embodiment, the data network plug-in service management system 0315 is a stand-alone system. The developer may download and use the plug-in service programming framework 0318 and program it in a framework to create plug-in service 0317-d. The plug-in service 0317-d may be uploaded to the data network plug-in service management system 0315. The plug-in service 0317-a, b, c uploaded in the data network plug-in service management system 0315 may issue and install the complete plug-in service or a subset 0317-a, b, c thereof to all the calling systems 0313-a, b and the access execution systems 0314-a, b, c in the data access and processing network 0312 through any one of the calling systems 0313-a, b. After installation, the application may call the real-time data.

The data access and processing network 0312 consists of a calling system 0313-a, b and an access execution system 0314-a, b, c. All real time data network system components 0315,0313-a,0313-b,0314-a, 0314-b, 0314-c are managed by the real time data network management system 0311.

According to the technical scheme provided by the embodiment of the invention, the independent data network plug-in service management system provides plug-in program management and uploading for a developer, and the plug-in service program can be issued to part or all of the calling system and the access execution system according to the situation, so that the access and processing of real-time multi-structure data is more flexible and elastic. Meanwhile, by setting up a plurality of independent calling systems, the problem that different users need to call different data respectively is solved.

Example III

As shown in FIG. 5, the present invention provides a second embodiment of a layout of a system for querying or processing distributed data in a coordinated manner. This embodiment is very similar to the layout of the individual components of the real-time data network system in fig. 4. In this embodiment, the real-time data network has only one calling system, and in addition, unlike the calling system 0313 in fig. 4, the calling system 0510 in this embodiment embeds the function of the data network plug-in service management system 0315. The developer may create a plug-in service program 0317-d using a plug-in service programming framework within the data network plug-in service management system 0315, upload and install to some or all of the calling system 0510 and the access execution systems 0314-a, b, c. After installation, the application 0520 issues call and process data instructions to the call system 0510 to access the data storage systems 0320-a, b, c and process data.

According to the technical scheme provided by the embodiment of the invention, the function of the data network plug-in service management system is embedded into the calling system, so that the structure is simplified.

Example IV

As shown in FIG. 6, the present invention provides a third embodiment of a layout of a system for querying or processing distributed data in a coordinated manner. This embodiment is very similar to the layout of the individual components of the real-time data network system in fig. 5. In this embodiment, the real-time data network has only one calling system, and the functions of the data network plug-in service management system 0315 and the real-time data network management system 0311 are embedded in the calling system 0610.

According to the technical scheme provided by the embodiment of the invention, the functions of the data network plug-in service management system and the real-time data network management system are integrated into the calling system, so that the structure is simpler.

Example five

As shown in FIG. 7, the present invention provides a fourth embodiment of a layout of a system for querying or processing distributed data in a coordinated manner. This embodiment illustrates an embodiment of the layout of a real-time data network system consisting of a plurality of data access and processing networks. In this embodiment, the real-time data network system comprises two data access and processing networks 0312-a, b, a real-time data network management system 0311, and a plug-in service management system 0315. Of course, this embodiment is also suitable for the case of any number of data access and processing networks. The data access and processing network 0312-a and the data access and processing network 0312-b in fig. 7 are two independent networks, each having its own independent invocation system and access execution system. Depending on the choice of the target and the embodiment, the same invoking system and the same access performing system may either register with both networks or be restricted to participate in only one network. In this example, only one real-time data network management system 0311 manages two data access and processing networks 0312-a, b. There is also a data network plug-in service management system 0315 that manages plug-in services for both networks.

Example six

As shown in FIG. 8, the present invention provides a fifth embodiment of a layout of a system for querying or processing distributed data in a coordinated manner. This embodiment illustrates an embodiment of the layout of a real-time data network system consisting of a plurality of data access and processing networks. The layout of the components in this embodiment is similar to fig. 7, wherein two data access and processing networks 0312-a,0312-b are managed by one real-time data network management system 0311. In contrast, the plug-in services of each data access and processing network 0312-a,0312-b are managed by its own data network plug-in service management system 0315-a, 0315-b. By having independent plug-in service management systems 0315-a,0315-b, these data access and processing networks 0312-a,0312-b do not need to share the same plug-in service. Of course, the present embodiment is also suitable for any of a plurality of data access and processing networks, and any combination of a plurality of data network plug-in service management systems.

The above embodiments are directed to two layer data networks, wherein the access enforcement systems 0314-a, b, c are stored as underlying connection area data, and the calling systems 0313-a, b are provided as upper layers with data in a unified manner to the application.

While two-layer data networks address many of the data sharing and processing needs, in some special cases, a multi-layer network may be necessary. In more than two layers of data networks, data stores are connected to the bottom layer system, data is processed in the bottom layer system, the processed results are collected to the middle layer system, and the upper layer calling system can access and process the data from the middle layer system. The layout example in fig. 9 is intended to illustrate that this objective may be achieved by constructing a multi-layer data network.

Example seven

As shown in FIG. 9, the present invention provides a sixth embodiment of a layout of a system for querying or processing distributed data in a coordinated manner. This embodiment illustrates a three-tier data access and processing network 0900. Wherein the regional data storage systems 0320-a, b, c, d, e are coupled to the underlying access execution systems 0314-a, b, c, d, e, forming an underlying system. In this embodiment, there are two intermediate layers 0910 and 0920, wherein one intermediate layer 0910 includes an intermediate processing node 0912 and an intermediate storage system 0320-i. The intermediate processing node 0912 is coupled to the underlying access execution system 0314-a, b. The other middle tier 0920 includes a back-to-back middle tier access execution system 0314-j and a middle tier call system 0313-g and a middle tier data storage system 0320-j. The calling system 0313-g is connected to the access execution system 0314-c, d, e of the bottom layer, and the uppermost layer contains one calling system 0313-e of the top layer. A real-time data network management system 0311 and a data network plug-in service management system 0315 provide network and plug-in service management services to data access and processing network 0900.

When an application 0950 instructs the top level calling system 0313-e to execute a pre-installed plug-in, a request may be sent to the intermediate processing node 0912 and the intermediate access execution system 0314-j. In one scenario, the invocation system 0313-g of the middle tier may, in cooperation with the access execution system 0314-c, d, e of the bottom tier, invoke data from the regional data storage systems 0320-c, d, e in advance and store it in the data storage system 0320-j of the middle tier. The access enforcement systems 0314-j of the middle tier may access these data. In another scenario, the intermediate processing node 0912 may process requests from the top level invocation system 0313-e and then issue requests to all of the bottom level access execution systems 0314-a, b to which it is connected, the bottom level access execution systems 0314-a, b may reprocess the data fetched from the regional data storage systems 0320-a, b at the intermediate processing node 0912. Intermediate processing node 0912 may also store data in middle tier data storage system 0320-i. Fig. 14 shows a plug-in service programming framework that may be applied to the present processing scenario. Further details regarding the programming framework will be provided later in fig. 13 and 14.

Although fig. 9 only shows a three-tier data network embodiment, other tiers may be added in the data access and processing network. For example, in another topology, two data access and processing networks may be stacked to form a multi-layer network. The call system 0313-e of the top layer may also be connected to the access execution system 0314-j of the middle layer, which is in turn connected to the call system 0313-g of the middle layer, which is in turn connected to the access execution system 0314-c, d, e of the bottom layer, similar to the embodiment shown in fig. 9. In this alternative configuration, the top tier call systems 0313-e may be connected to multiple middle tier access execution systems, which are each connected to a middle tier call system, which is connected to multiple bottom tier access execution systems, and so on, thereby forming a multi-tier real-time data access and processing network with any number of tiers above three. In a multi-layer data network there may also be a plurality of plug-in service management systems 0315.

According to the technical scheme provided by the embodiment of the invention, the intermediate processing result is generated and stored for users of related levels by setting up a multi-layer data access and processing network.

Example eight

As shown in fig. 10, the present invention provides one embodiment of the functional modules included in the real-time data network management system 0311. In this embodiment, the real-time data network management system 0311 is composed of four functional modules, namely, a real-time data network management module 1010, a calling system and access execution system account management module 1020 (hereinafter referred to as GDDS and DSS account management modules), a calling system and access execution system network topology management module 1030 (hereinafter referred to as GDDS and DSS network topology management modules), and a plug-in service system management module 1040.

Wherein the real-time data network management module 1010 tracks and manages each data access and processing network 0312. For example, in fig. 4, 5, 6 and 9, there is only one data access and processing network 0312 or 0900. Whereas in fig. 7 and 8 there are two data access and processing networks 0312-a, b, the functions provided by the data network management module 1010 include, for example, creating and deleting network management resources (e.g., creating or deleting data networks) and maintaining a data access and processing network 0312 deployment configuration. Wherein each data access and processing network 0312 has a unique identifier (referred to as a network id) and a network name.

The functions provided by the GDDS and DSS account management module 1020 in this embodiment include managing information for each calling system and access execution system. Wherein each of the calling system 0313 and the access executing system 0314 may create an account in the GDDS and DSS account management module 1020 and manage the account by the GDDS and DSS account management module 1020. All account information of the calling system 0313 and the access executing system 0314 is stored in its resources. As shown in fig. 10, the invocation system 0313 and the access execution system 0314 account may contain the following information: a network id for indicating to which data access and processing network 0312 the calling system 0313 or the access executing system 0314 is configured; a node id for generating a unique identifier for the calling system 0313 or the access performing system 0314; the node type is used for indicating whether the system is called, the execution system is accessed or the intermediate processing node is accessed; a node address indicating a network address of the system; a password for logging into the GDDS and DSS account management module 1020; security tokens for communication security, and the like. The system administrator may also provide credentials for nodes joining the network by creating an account (i.e., creating a node) that invokes system 0313 or access enforcement system 0314 through the functionality of GDDS and DSS account management module 1020. When the call system 0313 or the access execution system 0314 stops using, the account may delete the corresponding account (i.e., delete node) through the functions of the GDDS and DSS account management module 1020. When the calling system 0313 or the access execution system 0314 is configured, a system administrator inputs the provided credentials of the calling system 0313 or the access execution system 0314 joining the network into the calling system 0313 or the access execution system 0314, and the calling system 0313 or the access execution system 0314 performs registration with the GDDS and DSS account management module 1020. When the invoking system 0313 or the access executing system 0314 needs to refresh its security token, a new token can be obtained through the functions of the GDDS and DSS account management module 1020. When a system attempts to join a network or refresh token, the GDDS and DSS account management module 1020 uses its authentication function (verification node) to verify credentials provided by the calling system 0313 or the access enforcement system 0314. The flow of addition and deletion of the calling system 0313 or the access execution system 0314 is shown in fig. 18 and 19. It should be noted that although these figures only show management of the invocation system 0313 or the access enforcement system 0314, the system may add intermediate processing nodes 0912 by adjusting the embodiments.

The GDDS and DSS network topology management module 1030 in this embodiment manages the network link organization of the invocation system 0313 or the access execution system 0314 and possibly the intermediate system 0912 in the data access and processing network 0312. In resources, the GDDS and DSS network topology management module 1030 tracks the interrelation (e.g., parent node ids and child node ids) between the invocation system 0313 or the access execution system 0314 to form a network hierarchy of the particular data access and processing network 0312 identified by the network ids. The GDDS and DSS network topology management module 1030 provides functionality including: an insertion node for specifying a new location of the calling system 0313 or the access execution system 0314; a delete node for removing the calling system 0313 or the access executing system 0314 from the data access and processing network 0312; an acquire topology function for retrieving a topology or a slave topology of the data access and processing network 0312; the sub-node obtaining function is used for obtaining a subsystem of the calling system 0313, and the subsystem can be an access execution system or an intermediate layer node; the get parent function is used to get the parent system of the specific access execution system 0314, which may be the calling system 0313 or other functions.

The plug-in service system management module 1040 tracks and manages the plug-in service management system 0315. As shown in fig. 4 to 9, the data access and processing network 0312 relies on a plug-in service management system 0315, and a developer can build plug-in service programs and install these programs into the calling system 0313 or the access execution system 0314. When there are multiple data access and processing networks 0312, these data access and processing networks 0312 may share a plug-in service management system 0315. Alternatively, each data access and processing network 0312 may have its own separate plug-in service management system 0315. The resources of the plug-in service system management module 1040 contain information of the plug-in service management system 0315 in the network, including a network id for indicating that the plug-in service management system 0315 is serving the data access and processing network 0312; service id, which is a unique identifier of the plug-in service management system 0315; the server address is the network address of the plug-in service management system 0315; the server name is the name of the plug-in service management system 0315; the password is a security password for the plug-in service management system 0315 to connect to the real-time data network management system 0311. The plug-in service system management module 1040 also includes the following functions: add or delete plug-in service management system 0315 (i.e., add or delete service nodes); the add-on service management system 0315 joins the network using a registration function; other system components of the real-time data network may retrieve network information of the plug-in service management system 0315 using the get service system address function. As shown in fig. 5 and 6, if the search plug-in service management system 0315 is embedded in the calling system 0313, it is not necessary for the plug-in service system management module 1040 to manage it.

Fig. 10 is just one example of real-time data network system management. This example manages the real-time data network system through the data network management system 0311. Alternative embodiments may divide the management function into a plurality of modules or may combine some of the functional modules into one module. For example, the GDDS and DSS account management module 1020 may be combined with the GDDS and DSS network topology management module 1030 into one subsystem. Another embodiment may incorporate some or all of these functional modules into other system components of the real-time data network. For example, as shown in fig. 6, the data network management system 0311 functions are embedded in the calling system 0313. Thus, in the embodiment presented in FIG. 6, registration of the calling system may not be required.

Example nine

As shown in FIG. 11, the present invention provides an embodiment of a data network plug-in service management system 0315, which has three functional modules, namely a data network client module 1110, a plug-in service programming framework service module 1112, and a program management module 1116.

As shown in fig. 10, a system administrator may create an account for a plug-in service management system 0315 from plug-in service system management module 1040. In this embodiment, when the system administrator enters the created account into the newly deployed plug-in service management system 0315, the new system connects 1121 the real-time data network with the plug-in service system management module 1040 by registration. The data network client module 1110 is responsible for registering, maintaining connections and communications 1121 with the plug-in service system management module 1040.

Plug-in service programming framework service module 1112 manages plug-in service programming framework 0318. Through this module 1122, a system administrator may maintain the latest version of the programming framework and a developer may download a plug-in service programming framework 0318 to develop plug-in service programs 0317-a, b, c.

The program management module 1116 allows uploading and managing 1124 versions of the plug-in service 0317-a, b, c, which the developer also relies on to manage the libraries. When the developer decides to install a program into the data access and processing network 0312, this module instructs the calling system 0313 of the data access and processing network 0312 to distribute and install 1123 the plug-in service program 0317-a, b, c into all calling systems 0313 and access execution systems 0314-a, b, c systems in the network.

This embodiment is only one example of how the plug-in service 0317-a, b, c may be created and managed in a real-time data network, and alternative embodiments may further comprise partitioning or merging functional modules and integrating these modules into other system components of the real-time data network. For example, as shown in fig. 5 and 6, the functions of the plug-in service management system 0315 are embedded in the calling system 0313. In the embodiment of fig. 6, since the modules of the data network management system 0311 are also incorporated into the calling system 0313-a, the data network client module 1110 does not need to register and connect to the real-time data network management system 0311.

Examples ten

As shown in FIG. 12, the present invention provides a detailed component layout embodiment of a system architecture for querying or processing distributed data in a coordinated manner. An implementation of the invocation system 0313 and the access execution system 0314 of the real-time data network is shown. In the invocation system 0313 and the access execution system 0314, there are four main functions: the first is the formation of a data network; second is a secure communication between the calling system 0313 and the access execution system 0314-a, b, c; thirdly, the installation and management of the service program; and finally, the execution management of the service program.

In this embodiment, the management and maintenance of the data network involves invoking the system network management module 1220, DSS network management module 1270 and real-time data network management system 0311. When adding a new invocation system 0313 to the real-time data network, a invocation system account is first created in the GDDS and DSS account management module 1020, and the new invocation system is located (i.e. the node is inserted) in the GDDS and DSS network topology management module 1030 of the real-time data network management system 0311, as shown in fig. 10. The calling system account information is then provided to the calling system 0313 by the calling system network management module 1220. The system network management module 1220 is then invoked to register 1230 with the real-time data network management system 0311 and obtain a security token, as shown in fig. 10.

Accordingly, when adding a new access enforcement system 0314 to the real-time data network, an access enforcement system account is first created in the GDDS and DSS account management module 1020, and the new access enforcement system is located (i.e. the node is inserted) in the GDDS and DSS network topology management module 1030 of the real-time data network management system 0311, as shown in fig. 10. The access enforcement system account information is then provided to the access enforcement system 0314 by way of the access enforcement system network management module 1270. The access enforcement system network management module 1270 then registers 1257 with the real-time data network management system 0311. The data network formation process includes adding and deleting call system 0313 and access execution system 0314 in the data network topology, as shown in fig. 18 and 19.

The secure communication channel 1208 between the calling system 0313 and the access execution system 0314-a, b, c is handled by the secure protocol service module 1222 of the calling system 0313 and the secure protocol service module 1272 of the access execution system 0314-a, b, c. For illustration purposes, the service installation communication 1207 and the service execution communication are drawn outside of the secure communication channel 1208, respectively. In practice, all communications are conducted over the secure communications channel 1208.

In this embodiment, the function module for installation management in the calling system 0313 is the plug-in service installation management module 1210. The developer uses the plug-in service programming framework 0318 to write plug-in services. Plug-in service 0317-a, b, c may be uploaded to plug-in service management system 0315. When a developer installs 1201 the program, the plug-in service installation management module 1210 in the calling system 0313 installs 1202 the plug-in service 0317-a, b, c to the plug-in service library 1212, forming plug-in service 1213-a, b, c. It should be noted that, according to embodiments, the plug-in service 0317-a, b, c and the plug-in service 1213-a, b, c may be the same program, or the plug-in service 1213-a, b, c may be a subset of the plug-in service 0317-a, b, c, e.g., the plug-in service 1213-a, b, c may include only the programs 0317-a, b, c written in the program property template 1310 and invoking the plug-in service in the system program template 1320. The plug-in service 1213-a, b, c is shown with different labels installed in the calling system 0313.

As part of the installation, the plug-in service installation management module 1210 further sends plug-in service programs 0317-a, b, c to all access execution systems 0314-a, b, c, communications 1207 via secure protocol service channels 1222, 1272, along with an installation indication 1207. After receiving the installation instruction and the program, the plug-in service installation management module 1260 in the access execution system 0314-a, b, c installs 1256 the plug-in service 0317-a, b, c to the plug-in service library 1281 to form plug-in service 1282-a, b, c. It should be noted that, according to embodiments, the plug-in service program 0317-a, b, c and the plug-in service program 1282-a, b, c may be the same program, or the plug-in service program 1282-a, b, c may be a subset of the plug-in service program 0317-a, b, c, for example, the plug-in service program 1213-a, b, c may only include the program 0317-a, b, c written in the program attribute template 1310 and the plug-in service in execution system program template 1330. The various labels are shown to illustrate that plug-in services 1282-a, b, c are installed into access execution system 0314.

To enable the plug-in service 1282-a, b, c to access the local data storage system 0320-a, b, c, the local access execution system 0314-a, b, c selects the appropriate data adapter micro-service 1286-a, b, c and uses the data adapter configuration framework 1288 to configure the address and other linking information 1258 of the local data storage system 0320-a, b, c for the selected data adapter micro-service 1286-a, b, c, creating a data adapter 1287-x, y, z to connect to the local data storage system 0320-a, b, c. For example, if the local store contains one Oracle database and one NFS file server, then one Oracle adapter micro-service and one NFS client micro-service will be selected at the same time. They are allowed to connect to local NFS and Oracle servers through configuration on the microservices. Associations between plug-in services 1282-a, b, c and their data adapters 1287-x, y, z are created in an access execution system plug-in local access service module 1280.

Furthermore, if the structured data in the local data storage system 0320-a, b, c does not have the same data model as required by the plug-in service 1282-a, b, c, the local access execution system 0314-a, b, c can add data format conversion logic through the access execution system integrated programming service module 1283, creating a data integration program 1285-a, b, c using the data integration programming framework 1284. The association between the plug-in service 1282-a, b, c and its data integration 1285-a, b, c is managed and implemented by the access execution system plug-in local access service module 1280. Not all of the plug-in services 1282-a, b, c need be associated with the data adapter 1287-x, y, z or the data integration program 1285-a, b, c, and if the data model of the structured data required by the plug-in services 1282-a, b, c is the same as the data model in the local data storage system 0320-a, b, c, then the data integration program 1285-a, b, c is not required. In addition, the plug-in service 1282-a, b, c may request data from other plug-in services 1282-a, b, c, rather than from the local data storage system 0320-a, b, c, in which case the data adapter 1287-x, y, z is not required. The process of installing and configuring a plug-in service is shown in fig. 20.

In this embodiment, the APIs 1312 of the installed plug-in service 1213-a, b, c,1282-a, b, c are validated by calling the system plug-in service API interface module 1216 and the access execution system plug-in service API interface module 1261, and the plug-in service 1213-a, b, c,1282-a, b, c are ready for execution. Execution management of plug-in services involves invoking system plug-in service API interface module 1216, invoking system plug-in service execution engine 1214, access execution system plug-in service API interface module 1261, and access execution system plug-in service execution engine 1262. When an external application 1299 issues an API instruction to the plug-in service 1213-a, b, c by calling the system plug-in service API interface module 1216, the interface module validates the API. Once verified, the request is forwarded to the calling system plug-in service execution engine 1214, which is responsible for launching the plug-in service 1213-a, b, c and executing according to API instructions. Depending on the implementation of the plug-in service 1213-a, b, c, the invoking system plug-in service execution engine 1214 may simply execute the program 1213-a, b, c or it may also request all or a selected access execution system 0314-a, b, c to execute the associated plug-in service program 1282-a, b, c, which is transmitted via the secure communication channel between the invoking system 0313 and the security protocol service module 1222,1272 of the access execution system 0314-a, b, c. More detailed plug-in service 1213-a, b, c;1282-a, b, c can be seen with reference to fig. 13 and 14.

As shown in fig. 12, upon receipt of an access execution system request in the access execution system 0314-a, b, c, the access execution system plug-in service API interface module 1261 validates the request to ensure that the requested plug-in service is installed in the plug-in service database 1281, that the necessary integration has been completed 1285-a, b, c and that the necessary data adapters 1287-x, y, z have been configured. Once the verification is complete, the access execution system plug-in service execution engine 1262 initiates 1255 an associated plug-in service 1282-a, b, c execution request. The plug-in service 1282-a, b, c executes its associated data integration program 1285-a, b, c as necessary and accesses local data using its associated data adapter 1287-x, y, z as necessary.

Although fig. 12 shows a two-tier network with only the call system 0313 and the access execution systems 0314-a, b, c, a similar approach may be used to install and execute plug-in services in a multi-tier network with intermediate service nodes 0912. Fig. 14 illustrates a plug-in service programming framework that allows plug-in service programs to be created, installed, and executed in more than two layers of networks.

Example eleven

As shown in FIG. 13, the present invention provides an embodiment of a real-time data network programming framework. The programming frameworks include a plug-in service programming framework 0318, a data integration programming framework 1284, and a data adapter configuration framework 1288.

Referring to fig. 13, a developer may download a plug-in service programming framework 0318 to create 1302 a plug-in service program 0317,1213,1282. Wherein plug-in service programs 0317-a, b, c are defined as programs uploaded to plug-in service management module 0315, which have not been installed yet; plug-in service 1213-a, b, c is described as a program installed in calling system 0313; plug-in service 1282-a, b, c is described as a program installed in access execution system 0314.

Depending on the embodiment, these plug-in services 0317,1213,1282 may be identical programs, or it is possible that the program 1213 installed in the calling system 0313 and the program 1282 installed in the access execution system 0314 are only a subset of the plug-in services 0317. For example, the plug-in service 1213-a, b, c may include only the program 0317 written in the program properties template 1310 and the calling system program template 1320 for the plug-in service associated with the calling system 0313. Plug-in service 1282-a, b, c may contain only programs 0317 that write plug-in services associated with access execution system 0314 in program properties template 1310 and access execution system program template 1330.

In this embodiment, the plug-in service programming framework is made up of three parts, namely a program properties template 1310, a calling system program template 1320, and an access execution system program template 1330.

Within the calling system program template 1320 portion may be one or more function programming templates 1322, 1324, 1325, 1326 in which a developer may fill in logic that is desired to be executed in the calling system 0313. Such as GDDS verification template 1322 (gdds_auth) allows a developer to enter a code, check the credentials of the requesting application 1299 and its end user; the GDDS preprocessing templates 1324 (GDDS_Pre_Process), GDDS Post-processing templates 1325 (GDDS_post_Process) and GDSS execution templates 1326 (GDSS_Execute) allow developers to enter code, execute in the calling system 0313, the execution behavior may be defined in the execution place 1314 attributes, and the developers may add more subroutines in the calling system template 1320 section.

One or more function programming templates 1332 may be included in the access execution system program template 1330, such as a developer may fill in the DSS execution template 1332 with logic that is desired to be executed in the access execution system 0314. In addition, the developer may add more subroutines in the access execution system program template 1330 section. Also available in the access execution system program template 1330 is a query programming API interface for access execution system integration programs 1334 and access execution system adapters 1336 provided by the access execution system plug-in local access service module 1280. These API interfaces allow the plug-in service program 0317,1213,1282 to query 1308 the relevant data integration program 1285 and data adapter 1287, allow the plug-in service program 1282 installed in the access execution system 0314 to connect to the local storage using the data adapter 1287 as necessary to extract data, and perform data model conversion through the data integration program 1285.

In the program properties template 1310 part the developer can define and fill out program properties for the plug-in service 0317 in writing: such as API name 1312, execution locale 1314, data model 1316, and other attributes 1318. The developer may fill in the API name for the plug-in service 0317,1213,1282 into the template for the API name 1312, define and fill in the fields in the API name 1312 to be used as API instructions, and the external application 1299 issues an instruction to the calling system 0313 to trigger the execution of the associated plug-in service 1213,1282. It must be stated that the API name of the plug-in service 0317,1213,1282 may be the name of the API plug-in service 0317,1213,1282 package, etc., and does not have to be filled in the framework property template 1310.

In the program property template 1310 part, there may also be a definition template for the execution place 1314 to be filled in the plug-in service 1213,1282, instructing the calling system 0313 and the access execution system 0314 to execute the programs written in the calling system program template 1320 and the access execution system program template 1330, for example, if the definition of the plug-in service 1213,1282 in the execution place 1314 is "GDDS", when the calling system 0313 receives an API instruction, the calling system plug-in service execution engine 1214 will start the plug-in service 1213 to execute the programs filled in the GDDS execution template 1326. If a "DSS" is entered in the execution location 1314, the calling system plug-in service execution engine 1214 will launch the plug-in service 1213 to execute its program written in the GDDS pre-processing template 1324, make a request to the access execution system 0314-a, b, c, the access execution system plug-in service execution engine 1262 launches the plug-in service 1282 to execute its program written in the DSS execution template 1332, forming a response, and send the response to the calling system 0313. Upon receipt of the access execution system response, the system plug-in service execution engine 1214 is invoked to launch the plug-in service 1213 to execute its programs that are populated in the GDDS post-processing template 1325. It should be noted that the function programming templates 1322, 1324, 1325, 1326, 1332 in the call system program template 1320 and the access execution system program template 1330 are merely examples, and the add-in service program 0317,1213,1282 fills out the program into the function programming templates 1322, 1324, 1325, 1326, 1332 as needed, and not all the function programming templates 1322, 1324, 1325, 1326, 1332 need to be filled out the program.

In the program properties template 1310 section, the data model definition template 1316 is where the developer enters the data model definition required by the plug-in service 0317,1213,1282. The data model definition contains a data structure format and some code that represents a particular meaning, e.g., the data model may define that "1" represents "male" and "2" represents "female". If the definition of the data model is not the same as the locally stored data model, a system administrator of the local access execution system may create 1304 a data integration program 1285 using a data integration framework 1284, converting the data model of the local data into the data model defined in the data model field 1316. By accessing the executive system data integration programming service module 1283, a local system administrator inputs data conversion logic into a user defined function implementation (udf_imp) template 1342 portion of the data integration framework 1284 to create 1304 an integration program 1285. The data integration program 1285 is also given a name, which is entered into the integrated name template 1340 of the framework 1284. With the access execution system plug-in local access service module 1280, a system administrator associates the data integration program 1285 with the plug-in service 0317,1213,1282 using the access execution system plug-in local access service module 1280. As shown in the access execution system 1330 section, the access execution system plug-in local access service module 1280 provides the plug-in service 0317,1213,1282 with the access execution system integration program API 1334 so that the plug-in service 1282 can query 1308 its associated data integration program 1285 when executing in the access execution system 0314. It should be noted that the above description is only one embodiment, for example, the data model need not be entered in the data model definition template 1316, and the data model definition may be entered in the access execution system program template 1330 in a function programming template manner. In addition, the name of the data integration program 1285 does not necessarily need to be filled in with the integrated name 1340 template, for example, the name of the data integration program 1285 may be its software name.

There may be more attributes 1318 in addition to the specific attributes 1312, 1314, 1316 shown in fig. 13. For example, instead of the fixed program labels 1322,1324,1325,1326,1332 of the calling system program template 1320 and the access execution system program template 1330, the calling system and access execution system program names may be dynamically created by a developer and specified as additional attributes in the attributes 1318. As shown in fig. 13, if a plug-in service 0317,1213,1282 were to access a data storage system 0320a, b, c attached to an access execution system 0317,1213,1282, a system administrator local to the access execution system may create 1306 a data adapter 1287 using a data adapter configuration framework 1288, which is required to create a local data adapter 1287 for the plug-in service 0317,1213, 1282.

The data adapter configuration framework 1288 in the embodiment shown in FIG. 13 allows a system administrator to enter an adapter name 1350 field for the adapter, select and reference data adapter micro-servlets 1286-a, b, c, and enter address and access information fields for the data storage system in configuration templates 1356. When the data adapter micro service 1286-a, b, c is selected, the address of the micro service is entered into the micro service field 1352. A system administrator associates plug-in service 0317,1213,1282 with data adapter 1287 using access execution system plug-in local access service module 1280. Plug-in local access service module 1280 provides plug-in service 0317,1213,1282 with access execution system adapter API 1336 so that when plug-in service 1282 executes in access execution system 0314, access execution system adapter 1336 may be used, which queries 1308 relevant data adapter 1287 through access execution system plug-in local access service module 1280. The execution of the plug-in service is shown in fig. 21 a.

It should be noted that the frames 0318, 1284 and 1288 shown in fig. 13 are just one example. In another embodiment, a more complex framework may be created to implement more complex execution control, for example, execution control attributes may be added, such as configuration execution mode, timing and scheduling of certain executions, etc., for example, the execution mode may specify that once an external application publishes the APIs of a plug-in service program, an access execution system set must first execute certain programs among their programs, then a call system set must execute certain programs, then another set of access execution systems or call systems execute another set of programs, and so on.

In another example, the execution mode may be defined as follows: when an external application issues the API of a plug-in service, some programs in some calling and accessing execution systems are executed first, then another set of programs in some calling and accessing execution systems are executed at a specific time based on a scheduling configuration, and so on. In a multi-tier network, the plug-in service programming framework permits such execution control definitions: firstly, executing some programs in a lowest layer access execution system; executing a program of the middle layer system based on the execution result; based on the result again, the procedure of the upper network layer system is executed. Furthermore, in a different embodiment, all program names in the calling system program template 1320 and the access execution system program template 1330 may be dynamically defined by a developer.

In the embodiments of the present disclosure, the plug-in service framework is essentially a few programming templates used in the software development process, and a developer can download the programming templates locally and program on a local computer by using the programming templates. Specifically, in writing a service plug-in using an access execution system execution template, a developer may use the access execution system integration program APIs 1334, 1336 to access the data integration program and data adapter associated therewith in the access execution system to which the currently written service plug-in is to be finally installed. Once the developer has completed programming and definition on the various programming templates, the code resulting from the programming will be uploaded into the calling system as a plug-in service package. Next, the plug-in service therein will be distributed and installed into all calling systems and access execution systems.

In each access execution system, a local system person or developer may write a data integration program 1285 for data conversion using a data integration program framework 1284. At each access execution system, a local system administrator may associate data integration program 1285 with installed plug-in service 1282. With this association with a particular plug-in service 1282, the access execution system data integration program API 1334 of the plug-in service 1282 may access the data integration program 1285 associated therewith.

In addition, in each access enforcement system, local system personnel may also create data adapters 1287 by selecting a data adaptation microservice. Specifically, the data adapter may be configured first, for example, an IP address, an Identification (ID), a password, and the like are configured in the local memory; the configured data adapter is then associated with the installed plug-in service 1282. With this association with a particular plug-in service 1282, the access execution system data adapter API 1336 of the plug-in service 1282 may access the data adapter 1287 associated therewith. For example, in embodiments of the present disclosure, each access execution system may provide a data adapter of an Oracle database, an SAP data adapter, a NAS data adapter, a Mongo database data adapter, a MySQL data adapter, and so forth.

Example twelve

As shown in FIG. 14, the present invention provides an embodiment of a multi-tier plug-in service programming framework. In this embodiment, the program properties template 1310, calling system program template 1320, and access execution system program template 1330 are similar in part to those in FIG. 13. In fig. 14, a new intermediate system node programming template 1420-a, b, c program is added between the calling system program template 1320 and the access execution system program template 1330, where the intermediate system node programming template may have one or more function programming templates, such as an intermediate system node preprocessing template 1424 (pre_process), an intermediate system node Post-processing template 1425 (post_process), and an intermediate system node execution template 1426 (Execute), etc., to allow a developer to input codes, input a Pre-execution command at the intermediate system node preprocessing template 1424-a, and input an execution intermediate result at the intermediate system node Post-processing template 1425-a. By analogy, this is just an example of how another presentation plug-in service programming framework may be adapted to perform in a multi-tier data network.

Example thirteen

As shown in FIG. 15, the present invention provides an embodiment of a method of querying or processing distributed data in a coordinated manner for accessing distributed multi-structure data in real time. The real time data network in this embodiment comprises a calling system 0313 and three access execution systems 0314-x, y, z. At each location x, y, z there is a regional data storage system 0320-x, y, z and connected to its local access execution system 0314-x, y, z, respectively. Plug-in service 0317-a is installed 1201 into the real-time data network, where plug-in services installed in calling system 0313 are labeled 1213-a, and plug-in services installed in access execution systems 0314-x, y, z are labeled 1282-a. In this embodiment, the plugin service program 0317-a,1213-a,1282-a accesses the local data storage system 0320-x, y, z, and the data model required in the program does not match the data model in all local data storage systems 0320-x, y, z, so there is one data integration program 1285-x, y, z and data adapter 1287-x, y, z in each access execution system 0314-x, y, z, and the plugin service program 1282-a is associated with the data integration program 1285-x, y, z and data adapter 1287-x, y, z by reference.

When the external application 1299 issues an API instruction for the plug-in service 1213-a, the request is forwarded to the three access execution systems 0314-x, y, z, assuming that the execution site 1314 of the plug-in service 1213-a is "DSS", where the plug-in service 1282-a is executed according to its implementation in the access execution system program template 1330. The complete access execution system program at the access execution system 0314-x location includes an installed plug-in service program 1282-a and a data integration program 1285-x, data being accessed through a data adapter 1287-x. The complete access execution procedure in the z-position of the access execution system 0314-y is similar to the procedure in the access execution system 0314-x. Once the access enforcement system 0314-x, y, z retrieves the requested data, a response is generated and sent back to the calling system 0313. Depending on its implementation in the calling system program template 1320, the plug-in service 1213-a may simply forward the response back to the external application 1299, or it may aggregate and process the response to the access execution system and send the final result back to the external application 1299. The figure illustrates real-time access to real-time distributed multi-structure data over a real-time data network.

Examples fourteen

As shown in FIG. 16, the present invention provides another embodiment of a method of querying or processing distributed data in a coordinated manner for processing distributed multi-structure data in real time. Fig. 16 is similar to fig. 15, except that a new plug-in service has been added 1201 to the real-time data network. The same plug-in service installed in the calling system 0313 is labeled 1213-b and the plug-in service in the access execution system 0314-x, y, z is labeled 1282-b. This newly added plug-in service 1213-b,1282-b is included in the calling system portion 1213-b of the program and the access execution system portion 1282-b of the program. Plug-in service 1282-b processes and analyzes the distributed data at each access execution system 0314-x, y, z, respectively, which requests the plug-in service 1282-a to retrieve the data, and then processes and analyzes the data. Results from all access execution systems 0314-x, y, z are returned to the calling system 0313 where the same plug-in service 1213-b aggregates and further processes the results for external applications 1299. The figure explains how real-time collaborative computing is achieved with real-time distributed data on a real-time data network.

Example fifteen

As shown in fig. 17, the present invention provides an embodiment of a communication procedure between the real-time data network management system 0311, the calling system 0313 and the access execution system 0314 based on the embodiment shown in fig. 10. Fig. 17 illustrates mainly the communication procedure between the calling system 0313 and the access execution system 0314 within one data access and processing network 0312, and thus does not include the communication between the data network management system 0311 and the plug-in service management system 0315.

As shown in fig. 17, when the call system 0313 is newly added, the data network administrator needs to obtain a call system account through the real-time data network management system 0311 first, and specify the topology location of the call system 0313 in the data access and processing network 0312. In the real-time data network management system 0311, the GDDS and DSS account management module 1020 is responsible for issuing and managing call system accounts, and the GDDS and DSS network topology management module 1030 is responsible for managing the locations of the call system 0313 and the access execution system 0314 in the data network topology of the data access and processing network 0312.

The data network administrator then provides the calling system account and its topology location to the calling system 0313, which uses this information to register 1712 in the real-time data network management system 0311. Upon verification of the call system account information, the real-time data network management system 0311 provides 1714 a security token to the newly added call system 0313 (ST 1).

Similarly, as shown in fig. 17, when the access execution system 0314 is newly added, the data network administrator needs to obtain an access execution system account through the real-time data network management system 0311 first, and specify the topology location of the access execution system 0314 in the data access and processing network 0312. In the real-time data network management system 0311, the GDDS and DSS account management module 1020 is responsible for publishing and managing access execution system accounts, and the GDDS and DSS network topology management module 1030 is responsible for managing the locations of the invocation system 0313 and the access execution system 0314 in the data network topology of the data access and processing network 0312. The data network administrator then provides the access enforcement system account and its topological location to the access enforcement system 0314, which uses this information to register with the real-time data network management system 0311. Upon verification of the access enforcement system account information, the real-time data network management system 0311 provides 1724 a security token to the newly added access enforcement system 0314 (ST 2). Then, based on the topology location provided by the GDDS and DSS network topology management module 1030, the access enforcement system 0314 contacts and connects to its associated upper layer invocation system 0313.

Once the calling system 0313 and the access execution system 0314 are connected to the data access and processing network 0312 in the real-time data network, both systems can start their secure communication. As shown in fig. 17, the access enforcement system 0314 may connect 1730 to its upper layer invocation system 0313 and provide the security token to the invocation system 0313 (ST 2). The invocation system 0313 may authenticate the access enforcement system 0314 and its security token to the real-time data network management system 0311 (ST 2). After the authentication is passed, the invoking system 0313 provides 1734 a unique access token (AT 1) to the access performing system 0314, and the access performing system 0314 can communicate with the invoking system 0313 using AT 1. The access enforcement system 0314 also provides 1736 a unique access token (AT 2) to the invocation system 0313 for the invocation system to communicate with the access enforcement system 0314.

The communication process shown in fig. 17 is one embodiment of a number of commonly used secure network communication protocols to illustrate how secure communications may be conducted between system components of a real-time data network.

Examples sixteen

Fig. 18a is an embodiment of a method for adding a calling system 0313 to a real-time data network to construct data access and process network communication in a prior embodiment provided by the present disclosure; fig. 18b is an embodiment of a method for adding an access enforcement system 0314 to a real-time data network to construct data access and handle network communications in a prior embodiment provided by the present disclosure. Fig. 18a and 18b refer to fig. 17, as it duplicates part of the illustration in fig. 17.

In the embodiment of the present disclosure, upon a request to add a new calling system 0313-x, the following steps shown in fig. 18a will be performed:

step 1810: a GDDS and DSS account management module 1020 in the real-time network management system 0311 adds an account that invokes the system 0313-x.

Step 1812: the invocation system 0313-x is located in the topology of the data access and processing network 0312 by the GDDS and DSS network topology management module 1030.

The above steps are also described in fig. 17, and reference is also made to the description in fig. 17.

Step 1814: registration of the calling system 0313-x with the real-time network management system 0311 is initiated.

Specifically, the steps include: the calling system account information is entered into the calling system 0313-x and registration of the calling system 0313-x with the GDDS and DSS account management module 1020 in the real-time network management system 0311 is initiated.

After the registration is completed, the invoking system 0313-x receives the security token (ST 1) and the topology of the data network from the real-time network management system 0311.

Step 1816: the GDDS and DSS network topology management module 1030 examines the location of the calling system 0313-x in the network topology to determine if the calling system 0313-x has an associated access execution system.

If the calling system 0313-x has no associated access execution system, it will end.

Step 1818: in response to the calling system 0313-x having an associated access execution system 0314-i, j, k, the GDDS and DSS network topology management module 1030 informs all associated access execution systems 0314-i, j, k to be associated with the calling system 0313-x.

Step 1820: each associated access enforcement system 0314-i, j, k is connected to a calling system 0313-x.

In an embodiment of the present disclosure, each associated access enforcement system 0314-i, j, k may first connect to the invocation system 0313-x using its previously obtained security token (ST 2) from the real-time network management system 0311. The calling system 0313-x then authenticates the access enforcement system 0314-i, j, k with the real-time network management system 0311. If authenticated, the invoking system 0313-x and the access enforcement system 0314-i, j, k exchange access tokens (AT 1 and AT 2) with each other.

In the embodiment of the present disclosure, when a new access execution system 0314-x is requested to be added, the following steps shown in fig. 18b will be performed:

step 1830: an account of the access enforcement system 0314-x is added in the GDDS and DSS account management module 1020 of the real-time network management system 0311.

Step 1832: the access performing system 0314-x is located in the topology of the data access and processing network 0312 by means of the GDDS and DSS network topology management module 1030.

Step 1834: the access execution system 0314-x is associated with one or more existing invocation systems 0313-e, f.

Step 1836: registration of the access enforcement system 0314-x with the real-time network management system 0311 is initiated.

Specifically, the steps include: the access enforcement system account information is entered into the access enforcement system 0314-x and registration of the access enforcement system 0314-x with the GDDS and DSS account management module 1020 of the real-time network management system 0311 is initiated.

After the registration is completed, the access enforcement system 0314-x may receive the security token (ST 2) and the topology of the data network from the real-time network management system 0311. The network topology may include, among other things, the relevant invocation systems 0313-e, f from the real-time network management system 0311.

Step 1838: the access execution system 0314-x establishes a connection with each associated calling system 0313-e, f.

In particular, the access enforcement system 0314-x may connect to each associated calling system 0313-e, f with its security token (ST 2). The system 0313-e is then invoked, f to authenticate the access enforcement system 0314-x to the real-time network management system 0311. Once authenticated, each calling system 0313-e, f and the access enforcement system 0314-x system exchange access tokens (AT 1 and AT 2) with each other.

Example seventeen

Fig. 19a shows an embodiment of a method of managing a data access and processing network 0312 in a real-time data network by deleting a calling system in a prior embodiment provided by the present disclosure; fig. 19b shows an embodiment of a method of managing a data access and processing network 0312 in a real-time data network by deleting an access enforcement system in a prior embodiment provided by the present disclosure.

The delete call system 0313-x may refer to fig. 19a, and mainly includes the following steps.

Step 1910: the delete call system 0313-x is started on the real-time data network management system 0311.

Step 1912: the GDDS and DSS network topology management module 1030 in the real-time data network management system 0311 checks if the calling system 0313-x has an associated access execution system.

If call system 0313-x has an associated access execution system, go to step 1916; if the calling system 0313-x does not have any associated access execution system, step 1914 is entered.

Step 1914: the calling system 0313-x is deleted from the real-time data network management system 0311.

The steps may specifically include: GDDS and DSS account management module 1020 deletes the account that called system 0313-x; and the GDDS and DSS network topology management module 1030 removes the calling system 0313-x from the network topology.

Step 1916: the real-time data network management system 0311 refuses to delete the calling system 0313-x.

The deletion access performing system 0314-x may refer to fig. 19b, and mainly comprises the following steps.

Step 1930: the deletion of the access enforcement system 0314-x is initiated at the real-time data network management system 0311.

Step 1932: the GDDS and DSS network topology management module 1030 in the real-time data network management system 0311 notifies the calling system 0313-e associated with the access enforcement system 0314-x that the f access enforcement system 0314-x is being deleted.

Step 1934: invoking system 0313-e, f removes access enforcement system 0314-x from its topology.

Step 1936: the access enforcement system 0314-x is deleted from the real-time data network management system 0311.

The steps may specifically include: GDDS and DSS account management module 1020 deletes access to the executive system 0314-x account; and the GDDS and DSS network topology management module 1030 removes the access enforcement system 0314-x from the data network topology.

Example eighteen

FIG. 20a shows an embodiment of installing a plug-in service in the current embodiment provided by an embodiment of the present disclosure; fig. 20a shows an embodiment of a method for integrating plug-in services into a real-time data network in the current embodiment provided by an embodiment of the present disclosure. The operations in the flowcharts of fig. 20a and 20b are also depicted in fig. 12 and 13, and fig. 20a and 20b are a simple example of one of many embodiments, alternate embodiments may have different system components, different modules, and different protocols to achieve the same result.

In embodiments of the present disclosure, a plug-in service may need to be created and uploaded prior to installation. Specifically, for the creation of plug-in service, plug-in service programming framework 0318 may be downloaded first through plug-in programming framework service module 1112 in data network plug-in service management system 0315; plug-in service programming framework 0318 is then used to create plug-in service programs 0317-a, b, c. In particular, with respect to how to create plug-in service 0317-a, b, c is depicted in FIG. 13, reference may be made to FIG. 13. For plug-in service upload, plug-in service 0317-a, b, c may be uploaded to the data network plug-in service management system 0315 by the program management module 1116. The program management module 1116 may also manage versions of plug-in service 0317-a, b, c.

After the plug-in service is created and uploaded, the installation of the plug-in service 0317-a, b, c may begin as shown in fig. 20 a.

Step 2010: the program management module 1116 of the data network plug-in service management system 0315 is instructed to install plug-in service 0317-a, b, c.

Step 2012: the program management module 1116 sends an install request to the calling system 0313 to install the plug-in service 0317-a, b, c.

Step 2014: upon receipt of the request, the calling system 0313 calls the plug-in service installation management module 1210 to install the plug-in service 0317-a, b, c into the pre-installed library 1212.

Step 2016: the plug-in service installation management module 1210 informs all associated access execution systems 0314-a, b, c to install plug-in service 0317-a, b, c.

The pre-installed plug-in services on the calling system 0313 are described as 1213-a, b, c, the plug-in services 0317-a, b, c and the installed plug-in services 1213-a, b, c may be identical, or the plug-in services 1213-a, b, c may be a subset of the plug-in services 0317-a, b, c associated with the calling system 0313.

Step 2018: the plug-in service installation management module 1260 of each access execution system 0314-a, b, c installs the plug-in service 0317-a, b, c into its own installer library 1264.

Plug-in services installed on the access execution system 0314-a, b, c are described as 1282-a, b, c. Wherein the plug-in service 0317-a, b, c and the installed plug-in service 1282-a, b, c may be identical, or the plug-in service 1282-a, b, c may be a subset of the plug-in service 0317-a, b, c associated with the access execution system 0314.

After installation of the plug-in service, the integration of the plug-in service plug-in services 1282-a, b, c into the real-time data network may begin as shown in fig. 20 b.

Step 2020: the access execution system receives a data integration program created by a system administrator through a data integration programming framework provided by the access execution system.

In embodiments of the present disclosure, at each access execution system, a local system administrator may create a data integration program 1285 using a data integration programming framework 1284 provided by the access execution system. In particular, a local system administrator may input data model conversion logic into user-defined implementation fields 1342 of data integration programming framework 1284 to create data integration program 1285. Wherein the data integration program 1285 may convert the native data model into a model required by the plug-in service 1213-a, b, c.

At step 2022: the access execution system associates the created data integration program with the plug-in service program according to the operation of the system administrator through the local access service module of the plug-in program of the access execution system.

That is, in embodiments of the present disclosure, after the data integration program 1285 is created, a local system administrator may further associate the data integration program 1285 with the plug-in service 1282 through the access execution system plug-in local access service module 1280. Once the association is established, the access execution system data integration program API 1334 in the plug-in service 1282 may access the access execution system data integration program 1285 to perform data model transformations.

At step 2024: the access execution system receives a data adapter created by a system administrator through a data adapter configuration framework provided by the access execution system.

In embodiments of the present disclosure, at each access execution system, a local system administrator may also create a data adapter 1287 using a data adapter configuration framework 1288 provided by the access execution system. In particular, a local system administrator may provide a data adapter name 1350, select and reference adapter micro services 1286-a, b, c to micro service program field 1352, and provide data storage addresses and access credentials to create a data adapter 1287.

At step 2026: the access execution system associates the created data adapter with the plug-in service according to the operation of the system administrator by accessing the local access service module of the execution system plug-in.

That is, in embodiments of the present disclosure, after the data adapter 1287 is created, a local system administrator may further associate the data adapter 1287 with the plug-in service 1282 through the access execution system plug-in local access service module 1280. Once the association is established, the access execution system data adapter API 1336 in the plug-in service 1282 may access the local data storage system through the access execution system data adapter 1287.

The integration of the plug-in service plug-in services 1282-a, b, c into a real-time data network is described in more detail below in conjunction with fig. 12 and 13. As shown in fig. 13, when the required data model filled in by the plug-in service 0317-a, b, c in its program or property 1316 is inconsistent with the data model in the local data storage system 0320, the access execution system needs to create a data integration program 1285-a, b, c, and input the data model conversion logic into the user-defined function template 1342 of the data integration programming framework 1284. By accessing the system plug-in local access service module 1280, in fig. 12 and 13, a system administrator of the access execution system associates plug-in service 1282-a, b, c with data integration 1285-a, b, c. As shown in fig. 12, a system administrator accessing the execution system also configures data adapters 1287-x, y, z for plug-in service programs 1282-a, b, c to access local data. As shown in FIG. 13, the configuration includes providing a data adapter name 1350 using the data adapter configuration framework 1288, selecting and referencing adapter micro services 1286 into the micro-service program field template 1352, and providing data storage addresses and access credentials into the storage system setup template 1356. As shown in FIG. 12, a local system administrator associates a plug-in service 1282-a, b, c with a data adapter 1287-x, y, z by accessing an executing system plug-in local access service module 1280.

Examples nineteenth

As shown in FIG. 21a, the present invention provides an embodiment of a process for executing plug-in services in a real-time data network. The embodiment shown in fig. 21a is a simple example of a number of embodiments, an alternative embodiment may have different system components, different modules and different protocols to achieve the same result.

The flow chart of this embodiment may refer to fig. 12 invoking system 0313 and access execution system 0314 modules and the various programming frameworks of fig. 13. The flow chart shown in fig. 21a illustrates the execution of a plug-in service using the examples in fig. 15 and 16.

Step 2110: an external application requests execution of the installed plug-in service.

Specifically, in embodiments of the present disclosure, an external application 1299 may request execution of an installed plug-in service 1213-b via the published API instructions of the plug-in service 1213-b.

Step 2120: and calling a plug-in service program of the system starting request.

Specifically, in embodiments of the present disclosure, upon receipt of the above-described instructions, invoking system plug-in service interface 1216 will validate the API. After verification passes, the trigger plug-in service execution engine 1214 starts the requested plug-in service 1213-b. Where the plug-in service execution engine 1214 first instructs the requested plug-in service 1213-b to execute the program in its GDDS verification template 1322 to verify the requestor.

Step 2130: the calling system verifies the identity of the application program.

In particular, in embodiments of the present disclosure, the plug-in service execution engine 1214 may examine the results of the verification of the programs in the GDDS verification template 1322. If the identity verification is not passed, ending the flow; if the authentication is successful, then execution continues to step 2124.

Step 2140: and determining the execution position of the plug-in service program.

Specifically, in embodiments of the present disclosure, plug-in service execution engine 1214 may examine the execution location of configured plug-in service 1213-b. The execution location may be executed on the calling system 0313 or on the access execution system 0314.

Step 2150: and executing the plug-in service program at the determined execution position and acquiring an execution result.

Specifically, in an embodiment of the present disclosure, the step 2150 may include:

in response to determining that the plug-in service 1213-b is configured to execute on the calling system 0313, the plug-in service execution engine 1214 executes the program in the GDDS execution template 1326 of the requested plug-in service 1213-b and obtains the execution result.

In response to determining that the plug-in service 1213-b is configured to execute on the access execution system 0314, the following steps are performed:

A, the plug-in service execution engine 1214 executes the programs in the GDDS pre-processing template 1324 of the requested plug-in service 1213-b.

B, sends instructions to the application-selected access execution system 0314-x, y, z to execute the plug-in service 1282-B associated with plug-in service 1213-B.

Note that the calling system's plug-in service 1213-b and the access execution system's plug-in service 1282-b come from the same service 0317-b.

C, each access execution system 0314-x, y, z, upon receiving an instruction, the access execution system plug-in service interface 1261 triggers the plug-in service execution engine 1262 to launch the requested plug-in service 1282-b.

The plug-in service execution engine 1262 instructs the requested plug-in service 1282-b to execute the program in its DSS execution template 1332.

Among the above programs, the plug-in service program 1282-b may request execution of another plug-in service program 1282-a from the plug-in service execution engine 1262. If either the add-in service 1282-b or the add-in service 1282-a needs to invoke the data of the local data storage system 0320, its program can query its associated data integration program 1285 and its associated data adapter 1287 through the access execution system add-in local access service module 1280 using the access execution system integration program query link 1334 and the access execution system adapter query link 1336, and access the access execution system area data storage system 0320 using its associated data adapter 1287. If there is one data integration program 1285, the plug-in service 1282-a executes the data integration program 1285 to implement data model conversion.

E, at each access execution system node 0314-x, y, z, the access execution system plug-in service program 1282-b generates and sends the result data back to the calling system plug-in service execution engine 1214 of the calling system 0313.

F, when the calling system 0313 receives results from the access execution system 0314-x, y, z, the calling system plug-in service execution engine 1214 instructs the plug-in service 1213-b to execute the program in its GDDS post-processing template 1325 and obtain the execution result.

In step 2160, execution results are returned to the requesting application 1299.

The above flow ends.

Fig. 21b shows a specific example of a process of executing a plug-in service in a real-time data network. As shown in fig. 21b, the above-mentioned process of executing the plug-in service may include:

at step 2110b, the external application 1299 sends an API instruction request by calling the system plug-in service API interface 1216.

In step 2120b, the calling system invokes its own plug-in service execution engine 1214 to load and execute the requested plug-in service 1213-b.

At step 2130b, it is confirmed whether the code is filled in the GDDS verification template 1322, and when the code is filled in the GDDS verification template 1322, the code in the GDDS verification template 1322 is executed to verify the received API instruction request. If the execution result of the code is that the verification is not passed, ending the flow; if the GDDS verification template 1322 is not filled with code or the result of the execution of the code is verified, step 2131b is performed.

In step 2131b, determining the execution location of the plugin service 1213-b, if it is a call to the system, then step 2132b is performed; if it is an access execution system, step 2133b is executed.

In step 2132b, in the calling system, the code in the calling system execution template 1326 is executed and the result is returned to the external application 1299, ending the flow.

In step 2133b, in the calling system, if code is filled in the calling system pre-processing template 1324, then executing the code in the calling system pre-processing template 1324 and then sending a request to the selected access execution system to execute the plug-in service 1282b therein; otherwise, the request is sent directly to the selected access execution system to execute the plug-in service 1282b therein.

It should be noted that, the plug-in service in the calling system and the plug-in service in the access execution system come from the same plug-in service package. Wherein invoking the plug-in service 1213-b in the system contains code that the developer creates by invoking the system program template 1320 when developing the plug-in service package 0317-b; and plug-in service 1282-b in the access execution system contains code created by the developer through access execution system program template 1330 when developing plug-in service package 0317-b.

Further, as previously described, invoking the system program template 1320 may include: invoking the system verification template, invoking the system preprocessing template, invoking the system execution template and invoking the system post-processing template. The calling system verification template, the calling system preprocessing template and the calling system post-processing template are all optional templates for a developer, namely the developer can create corresponding codes in the calling system verification template, the calling system preprocessing template and the calling system post-processing template, and can also not fill the codes in the templates.

In step 2134b, after receiving the request via plug-in service API interface 1261, each access execution system initiates its own plug-in service execution engine to load the requested plug-in service 1282-b and execute the code therein in the access execution system execution template 1332.

In embodiments of the present disclosure, code in the access execution system execution template 1332 may request execution of another plug-in service 1282 a.

In embodiments of the present disclosure, code in the access execution system execution template 1332 described above may request that the access execution system plug-in local access module 1280 launch a data integration program 1285 and data adapter 1287 associated therewith. The code in the access execution system execution template 1332 extracts data from the local data storage system 0320 via the data adapter 1287 and may further execute the data integration program 1285 to complete data model conversion.

In embodiments of the present disclosure, the code in the access execution system execution template 1332 may generate response data, at which point the access execution system feeds back the response data to the invocation system.

In step 2135b, in the calling system, if the code is filled in the calling system post-processing template 1325, executing the code in the calling system post-processing template 1325, returning the finally collected processing result to the external application program 1299, and ending the flow; otherwise, the response data is directly returned to the external application 1299, and the flow is ended.

The above flow ends.

Fig. 21c shows a specific example of a process of executing a plug-in service in a real-time data network. In this example, an intermediate processing node is also included between the calling system and the access execution system. As shown in fig. 21c, the above-mentioned process of executing the plug-in service may include:

at step 2110c, the external application 1299 sends an API instruction request by calling the system plug-in service API interface 1216.

In step 2120c, the calling system invokes its own plug-in service execution engine 1214 to load and execute the requested plug-in service 1213-b.

At step 2130c, it is confirmed whether the code is filled in the GDDS verification template 1322, and when the code is filled in the GDDS verification template 1322, the code in the GDDS verification template 1322 is executed to verify the received API instruction request. If the execution result of the code is that the verification is not passed, ending the flow; if the GDDS verification template 1322 is not filled with code or the result of the execution of the code is verified, step 2131c is performed.

In step 2131c, determining the execution location of the plugin service 1213-b, if it is a call to the system, then step 2132c is performed; if it is an intermediate processing node or an access execution system, step 2133c is executed.

In step 2132c, in the calling system, the code in the calling system execution template 1326 is executed and the result is returned to the external application 1299, ending the flow.

In step 2133c, in the calling system, if code is filled in the calling system pre-processing template 1324, executing the code in the calling system pre-processing template 1324, sending a request to the intermediate processing node; otherwise, the request is sent directly to the intermediate processing node.

At step 2134c, determining the execution location of the plugin service 1213-b, if it is an intermediate processing node, then step 2135c is performed; if it is an access execution system, then step 2136c is executed.

In step 2135c, the intermediate processing node executes the code in the intermediate processing node execution template 1426 and returns the processing results to the calling system, then step 2139c is performed.

In step 2136c, if code is filled in the intermediate processing node pre-processing template 1424, the intermediate processing node executes the code in the intermediate processing node pre-processing template 1424, sending a request to the selected access execution system. Otherwise, the intermediate processing node directly sends a request to the selected access execution system.

In step 2137c, after receiving the request via plug-in service API interface 1261, each access execution system initiates its own plug-in service execution engine to load the requested plug-in service 1282-b and execute the code therein in the access execution system execution template 1332.

In embodiments of the present disclosure, the code in the access execution system execution template 1332 may generate response data, at which point the access execution system feeds back the response data to the intermediate processing node.

In step 2138c, in the intermediate processing node, if the code in the intermediate processing node post-processing template 1425 is executed, then the code in the intermediate processing node post-processing template 1425 is executed and the aggregated processing results are returned to the calling system; otherwise, the response data is directly returned to the calling system.

In step 2139c, in the calling system, if the code in the system post-processing template 1325 is called, the code in the system post-processing template 1325 is executed and the final assembled processing result is returned to the external application 1299, and then the flow is ended; otherwise, the response data is directly returned to the external application 1299, and the flow is ended.

The above flow ends.

The flowcharts in fig. 20a, 20b and 21a, 21b and 21c illustrate how the real-time data network programming framework (see fig. 13 and 14) is used to build an installable plug-in service, and how the modules of the invocation system 0313 and the access execution system 0314 work cooperatively (see fig. 12) to access (see fig. 15) and process (see fig. 16) real-time distributed multi-structure data in a coordinated manner. It should be noted that the above procedure may be somewhat different if the embodiment of the plug-in service framework (refer to fig. 13) is different or the organization of the system modules of the calling system 0313 and the access execution system 0314 is different. In addition, the developer need not fill out program code in all function templates of the plug-in service framework (refer to fig. 13), such as the GDDS verification template 1322, the GDDS execution template 1326, the GDDS preprocessing template 1324, the GDDS post-processing template 1325, and the DSS execution template 1332, when writing the plug-in service 0317. If these functions do not contain program code, then these functions do not need to be performed.

The application of the above-described real-time data network will be described in connection with two specific examples.

Example 1: application of real-time data network in large-scale mechanism

Fig. 22 illustrates an example of an application of a real-time data network in a large organization according to some embodiments of the present disclosure.

It is assumed that the large-scale mechanism described above includes four-stage mechanisms, that is, a first-stage mechanism 2210, a second-stage mechanism 2220, a third-stage mechanism 2230, and a fourth-stage mechanism 2240. Among them, the first-stage mechanism 2210 is the highest-stage mechanism, and the fourth-stage mechanism 2240 is the lowest-stage mechanism. One first stage mechanism 2211, three second stage mechanisms 2221-a, b, c, six third stage mechanisms 2231-a, b, c, d, e, f, and 12 fourth stage mechanisms 2241-a, b, …, k, l are exemplarily shown in fig. 22.

In practical applications, as shown in fig. 22, the upper level organization should be able to acquire data and files from the lower level organization. In addition, the first level organization should also have access to the data and files of the various levels of organization.

In fig. 22, each organization itself has a store 2212, 2222-a, b, c,2232-a, …, f,2242-a, …, l of structured data, semi-structured data, and unstructured data. It should be noted that, although a case where each organization manages its own data is described in fig. 22, in practical applications, in some areas, there may be a case where data and files of a lower-level organization may be centrally managed by an upper-level organization to which it belongs. In addition, there may be situations where unstructured data is separately hosted and managed by a third party provider.

Fig. 22 also shows that an access enforcement system 0314 is deployed at each organization. For example, access enforcement system 2218 is deployed at a first level of organization; the second level mechanism is provided with an access execution system 2228-a, b, c; the third level mechanism is deployed with access execution systems 2238-a, b, c, d, e, f; and an access execution system 2248-a, b, …, k, l is deployed at the fourth level of organization. The data 2212, 2222-a, b, c,2232-a, b, …, e, f,2242-a, b, …, k, l stored in these institutions will be connected to the access execution systems 2218, 2228-a, b, c,2238-a, b, c, d, e, f,2248-a, b, …, k, l deployed at the local institutions, respectively. Data adapters (1287-x, y, z as shown in fig. 12) are also provided in these access enforcement systems 2218, 2228-a, b, c,2238-a, b, c, d, e, f,2248-a, b, …, k, l to ensure that the access enforcement systems can access data stored locally. For example, data 2212 stored at the first level organization may be connected to access enforcement system 2218 through a data adapter deployed in access enforcement system 2218.

In the above example, the first level organization 2211 may also be deployed with a calling system 2216; the second level mechanisms 2221-a, b, c may also be deployed with call systems 2226-a, b, c; a call system 2236-a may also be deployed at the third level of organization 2231-a. As previously described, the invocation system allows external applications to query and process data stored by each organization in a distributed manner through the access execution system. The data collected from the access execution system may also be further processed in the calling system before being passed to an external application. In this example, some second level mechanisms, such as second level mechanisms 2231-b, c, d, e, f, may not deploy the calling system. After the staff of these institutions obtain the proper security authorization, the staff can access the data through the calling system of the third-level institution to which they belong.

Fig. 22 shows an example of deploying a centralized plug-in management system 2214 alone at a first level organization 2211. Through the plug-in management system 2214 described above, a system administrator of the first-level organization 2211 can upload and issue plug-in service programs to call systems and access execution systems of all organizations or selected organizations around the country. In addition, at each of the second level mechanisms 2221-a, b, c and third level mechanisms 2231-a, its calling system 2226-a, b, c,2236-a may be tied together with its plug-in management system. The purpose of this deployment is to allow these second level mechanisms 2221-a, b, c and third level mechanisms 2231-a to create their own plug-in services and publish and install into their own invocation systems and their associated access execution systems, respectively.

In this example, a real-time data network management system 2213 is also deployed in the first-level organization 2211, for managing all call systems and access execution systems deployed around the country.

Fig. 23 shows a network configuration of a calling system and an access execution system in the embodiment of the present disclosure. For convenience of description, the network configuration of the real-time data network of the entire large organization is shown in fig. 23 by taking only a few organizations as representative.

In this example, all access enforcement systems deployed nationwide have registered and connected to the invocation system 2216 of the first level organization 2211, as required by the first level organization 2211, so that workers can access the real-time data and files of each level organization themselves as needed. In fig. 23, the access execution systems 2248-a, b, …, k, l deployed at the fourth level organization may be connected to the invocation system 2216 described above; the access execution systems 2238-a, b, …, e, f deployed at the third level organization may be connected to the invocation system 2216 described above; the access execution systems 2228-a, b, c deployed at the second level organization may be connected to the invocation system 2216 described above; an access enforcement system 2218 deployed at the first level organization may also be connected to the invocation system 2216 described above. An external application, such as the first level organization's work system 2310, is an application that a worker uses to access all of the organization's data and files. The application 2310 may send an API request to the calling system 2216 described above to trigger execution of the plug-in service installed in the calling system 2216 and all access execution systems.

The staff of each level of institutions also typically need to access data and files of the lower level institutions within and managed by the level of institutions. Thus, all its own and its subordinate's access execution systems also need to register and connect to the calling system at their own. For example, the access performing systems 2248-a, b, c, d deployed at the fourth level mechanisms 2241-a, b, c, d, the access performing systems 2238-a, b deployed at the third level mechanisms 2231-a, b, and the access performing systems 2228-a deployed at the second level mechanisms should register and connect to the calling systems 2226-a of their affiliated second level mechanisms 2221-a. Likewise, the access execution systems 2248-e, f, g, h deployed at the fourth level mechanisms 2241-e, f, g, h, the access execution systems 2238-c, d deployed at the third level mechanisms 2231-c, d, and the access execution systems 2228-b deployed at the second level mechanisms should also register and connect to the calling systems 2226-b of their affiliated second level mechanisms 2221-b. The access execution systems 2248-i, j, k, l deployed at the fourth level mechanisms 2241-i, j, k, l, the access execution systems 2238-e, f deployed at the third level mechanisms 2231-e, f, and the access execution systems 2228-c deployed at the second level mechanisms should register and connect to the calling systems 2226-c of their affiliated second level mechanisms 2221-c.

In the example shown in FIGS. 22 and 23, third level mechanism 2231-a is also deployed with a calling system 2236-a. At this point, the access execution system 2238-a deployed at the third level organization 2231-a itself and its administered fourth level organization 2241-a, b access execution system 2248-a, b should register and connect to the invocation system 2236-a deployed at the third level organization 2231-a.

The invocation system 2216 deployed at the first level organization 2211 allows personnel to read data and files through all access execution systems deployed throughout the network using local applications, such as office system 2310. The calling system 2226-a, b, c deployed at the second level organization 2221-a, b, c allows the staff member to use the native application 2320-a, b, c to read data and files through all access execution systems deployed within their administrative scope.

Since the third level organization 2231-a is deployed with its own invocation system 2236-a, the invocation system 2236-a allows its native applications 2330-a to read its own data and files through the access execution systems 2238-a,2248-a, b and the data and files of the fourth level organization 2241-a, b within its management scope. In this example, the other third level mechanisms 2231-b, c, d, e, f do not deploy their own invocation systems, and their applications 2330-b, c, d, e, f can read the data and files of the fourth level mechanisms 2241-c, d, e, f, g, h, i, j, k, l within their management range through their invocation systems 2226-a, b, c deployed by their upper level second level mechanisms 2221-a, b, c. As shown in FIG. 22, the lower level mechanisms of the third level mechanisms 2231-b are fourth level mechanisms 2241-c and fourth level mechanisms 2241-d; the lower level mechanisms of the third level mechanisms 2231-c are fourth level mechanisms 2241-e and fourth level mechanisms 2241-f; the lower level mechanisms of the third level mechanisms 2231-d are fourth level mechanisms 2241-g and fourth level mechanisms 2241-h; the lower mechanisms of the third-stage mechanisms 2231-e are fourth-stage mechanisms 2241-i and 2241-j; while the lower level mechanisms of third level mechanisms 2231-f are fourth level mechanisms 2241-k and 2241-l. The data for these fourth level mechanisms is coupled to the call systems 2226-a, b, c of the second level mechanisms.

In fig. 23, the access enforcement system 2218 of the first level organization may register and connect to only one invocation system 2216, while other access enforcement systems may register and connect to multiple invocation systems. For example, the access execution systems 2248-a, b deployed at the fourth level mechanisms 2241-a, b may each register with and connect to the invocation system 2216, the invocation system 2226-a, and the invocation system 2236-a.

Fig. 24 illustrates another example of deployment of a real-time data network in a large institution in accordance with further embodiments of the present disclosure. Fig. 24 clearly illustrates a situation where the real-time data network management system 0311 manages five sub-networks 2400, 2410, 2412, 2414, 2420. Wherein each subnetwork is associated with a calling system 2216, 2226-c,2226-b,2226-a,2236-a, respectively. Fig. 24 also clearly shows that the access execution systems 2248-a, b deployed at the fourth level mechanisms 2241-a, b have registered and connected to the invocation system 2216 in the subnetwork 2400, the invocation system 2226-a in the subnetwork 2414 and the invocation system 2236-a in the subnetwork 2420.

Fig. 25 shows an example of a real-time data network 2400 for a first level organization and a list of plug-in services 2500 created by first level organization 2210. Plug-in service 2500 is labeled 0317-a, b, c in the previous figures. As shown in fig. 12, once the plug-in service 0317-a, b, c is uploaded and published onto the network, the program installed on the calling system 0313 will be marked 1213-a, b, c; and the programs installed at the access enforcement system 0314 are marked 1282-a, b, c.

Fig. 25 shows three implementation examples of plug-in service programs, namely a first plug-in service program 2510 (getallllivecases), a second plug-in service program 2512 (ReadFile), and a third plug-in service program 2514 (extraactallllivecasedocdata). The API names of these plug-in services are defined under their API names. As previously described, plug-in service 0317 is implemented by populating program code into plug-in service programming framework 0138, which is essentially a programming template. The example of a plug-in service program shown in FIG. 25 above shows only a subset of the plug-in service programming framework 0138 shown in FIG. 13.

As also described above, the calling system 1320 portion of the programming framework 0318 is a template for filling out code that needs to be executed in the calling system 0313. The access execution system program 1330 is used in part to fill in code that needs to be executed in the access execution system 0314. In fig. 25, the program code executed by the calling system 0313 among the program codes of the above-described example implementation is in the calling system program. The program code executed by the access execution system 0314 among the program code implemented in the above example is in the access execution system program. For simplicity of illustration, fig. 25 does not show the data integration program and the data adapter of each access execution system.

The first add-in service 2510 shows code that when the office system 2310 sends a get data API request (GetAllLiveCases API request) to the calling system 2216, the calling system 2216 will execute the developer thereof to fill in the GDDS pre-processing template 1324 of the first add-in service 2510 to send the get data API request to the access execution system of the specified institution. These access execution systems will then execute the code in DSS execution template 1332 in first plug-in service 2510 as described above.

As shown by the code in the DSS execution template 1332 of the first plug-in service 2510 described above, if there is a data integration program 1285 as shown in fig. 12, the code uses the data integration program 1285 to modify the database query statement that is used to map the data model of the local data store to the data model required by the first plug-in service 2510 described above, otherwise standard database query statements need to be used. Code in the DSS execution template 1332 of the first plug-in service 2510 described above will query the local database using the data adapter 1287 with standard or modified database query statements. The results of the query will be sent back to the calling system 2216. The invocation system 2216 will then execute the code in the GDDS post-processing template 1325 of the first plug-in service 2510 described above, formatting the query results and feeding back to the office system application 2310 in response.

To read a particular file, the office system 2310 may issue a read file API request to the calling system 2216 (ReadFile API request) with the URL of the file as a parameter of the read file API request. It should be noted that, before issuing the above-mentioned request for reading the file API, the office system 2310 may send other API requests to obtain the URL of the file. Further, the above-mentioned read file API request may further include in its parameters a data range of the file to be retrieved. Upon receiving the read file API request, the calling system 2216 will load the second plug-in service 2512 and execute the code in the GDDS pre-processing template 1324 in the second plug-in service 2512 as described above. Code in the GDDS preprocessing template 1324 of the second plug-in service 2512 described above may use the URLs in the read file API requests to identify the access execution systems associated with the institutions where the files are located and forward the read file API requests to the responding access execution systems. Upon receiving the request to read the file API, the access execution system will execute the code in the DSS execution template 1332 of the second add-in service program 2512 to perform link verification, retrieve the file contents within the specified range, and send the retrieved data as a response to the invocation system 2216. Finally, the invocation system 2216 forwards the response to the application office system 2310.

As shown in these examples above, the staff of the first level organization may access the latest real-time data from any organization by itself through an application such as the work system 2310. Most importantly, the above-described approach does not require the collection of large amounts of files and data from each level of organization to a centralized location.

Implementation of the third plug-in service 2514 demonstrates another important function of the present invention. The first level organization may need to extract meaningful data from unstructured data in the document for use. For example, a first level organization may wish to create an index of files to facilitate a staff member to perform a search for file content; analyzing the quality of the file content; and analyzing and identifying content of interest, etc. Since it is not practical to aggregate all files of all institutions into one central location to extract information, a plug-in service, such as third plug-in service 2514, may be created and uploaded to calling system 2216. Then, it is issued and installed to all access execution systems.

In this way, data can be extracted from the document in a distributed manner at the network edge using distributed computing resources in the access enforcement system. Upon receipt of the extract file data API request (ExtractAllLiveCaseDocData API request), the calling system 2216 will load the third plug-in service 2514 and execute code in the GDDS pre-processing template 1324 of the third plug-in service 2514 to send the extract file data API request to the access execution systems of all or selected institutions. Upon receipt of the extract file data API request, the access execution system executes code in the DSS execution template 1332 of the third plug-in service program 2514. The above implementation can make direct requests to other plug-in service programs to extract document information and document content, then use OCR logic to convert the content to text and extract data according to application requirements. The extracted data may then be fed back to the invocation system 2216 as a response and eventually forwarded to the application 2310 sending the request.

Distributed data retrieval and data processing logic can be implemented very easily using service plug-in programming framework 0318, through first plug-in service 2510, second plug-in service 2512, and third plug-in service 2514 as described above. Moreover, these distributed data retrieval and data processing logic may be installed on all systems in the real-time data network as desired.

The application scenario shows that the requirement of large-scale institutions for automatically accessing and processing distributed data can be met by using a real-time data network, and a large amount of system resources are not required to be concentrated in a central position.

Example 2: application of real-time data network in Internet of things

Fig. 26 illustrates an example of an internet of things application implemented using a real-time data network in accordance with some embodiments of the present disclosure. In this example, a four-tier network structure consisting of a calling system 0313, an intermediate processing node 0912, an access execution system 0314, and devices/sensors is applied. The four-layer network comprises: device layer 2640, edge computation layer 2630, region computation layer 2620, and central computation layer 2610.

The device layer 2640 is located at the edge of the network, where devices and sensors a … p for detecting microclimate are deployed.

The edge computation layer 2630 is comprised of the internet of things gateways 2632-a, …, g for computing data streams received from devices and sensors in the device layer 2640. The access enforcement systems 2634-a, …, g would be embedded in the internet of things gateways 2632-a, …, g, respectively. As shown in fig. 26, sensor a and sensor b are connected to an internet of things gateway 2632-a; sensor c and sensor d are connected to the internet of things gateway 2632-b; the sensors e, f, g, h are connected to the internet of things gateway 2632-c; sensor i and sensor j are connected to the internet of things gateway 2632-d; the sensors k, l, m are connected to the internet of things gateway 2632-e; sensor n and sensor o are connected to the internet of things gateway 2632-f; finally, sensor p is connected to an Internet of things gateway 2632-g. When the internet of things gateway 2632-a, …, g processes, analyzes, and generates meaningful data, the processed data will be forwarded to the regional calculation layer 2620.

The region calculation layer 2620 is composed of internet of things servers 2622-a, b, c and is used for calculating data generated by the internet of things gateways 2632-a, …, g. The real-time data network intermediate processing nodes 2624-a, b, c would be embedded in the internet of things servers 2622-a, b, c, respectively, described above. As shown in fig. 26, the internet of things gateway 2632-a, b, c is connected to the internet of things server 2622-a; the Internet of things gateway 2632-d, e is connected to the Internet of things server 2622-b; finally, the Internet of things gateway 2632-f, g connects to the Internet of things server 2622-c. The data generated by the internet of things servers 2622-a, b, c will be forwarded to the central computing layer 2610.

The central computing layer 2610 is located at the top of the network. Although fig. 26 shows only one centralized internet of things server 2613, there may actually be a plurality of such servers. As shown in fig. 26, all of the regional internet of things servers 2622-a, b, c are connected to the centralized internet of things server 2613. The invocation system 2614 will be embedded in the centralized internet of things server 2613.

In addition, as shown in FIG. 26, the embedded calling system 2614, the intermediate processing nodes 2624-a, b, c, and the access execution systems 2634-a, …, g may be managed by a centralized real-time data network management system 2611. The real-time data network is further deployed with a plug-in management system 2612 for managing all plug-in service 0317-a, b, c uploaded to the real-time data network by developers, which are then published and installed into the calling system 2614, the intermediate processing nodes 2624-a, b, c and the access execution systems 2634-a, …, g via the calling system 2614.

Fig. 26 shows an example of a fourth plug-in service 2650 (flood_alert). The fourth plug-in service 2650 described above is created by a developer using the multi-tier service plug-in programming framework 1410 shown in FIG. 14. As previously described, the program code written by the developer into the calling system program 1320 section will be executed by the embedded execution calling system 2614. Program code written by a developer into the programming template 1420-a portion of the intermediate system node will be executed by the intermediate processing nodes 2624-a, b, c. Program code written by the developer into the access execution system program 1330 portion will be executed by the access execution systems 2634-a, … g.

The fourth plug-in service 2650 described above allows a user to start or stop his analysis service. As shown by the code in the GDDS preprocessing template 1324 and the intermediate system node preprocessing template 1424-a of the fourth add-in service 2650, upon receipt of the flood alert API command (start/stop, … parameters), the calling system 2614 propagates the flood alert API command down to all intermediate processing nodes 2624-a, b, c. The intermediate processing nodes 2624-a, b, c load and execute code in the intermediate system node pre-processing templates 1424-a of the fourth add-in service 2650, which flood alert API commands will be forwarded further to the access execution systems 2634-a, …, g in all internet of things gateways 2632-a, …, g. Upon receipt of the flood alert API command described above, the access execution system 2634-a, …, g loads and executes code in the DSS execution template 1332 of the fourth plug-in service 2650. In response to the above command, the fourth plug-in service 2650 will start or stop the computing process at the access execution system 2634-a, …, g. After the fourth plug-in service sequence 2650 at the access execution system 2634-a, …, g is initiated, the code continuously processes the data streams from the devices and sensors 2640-a, …, p in a distributed manner to extract meaningful data and send the extracted information to the relevant intermediate processing nodes 2624-a, b, c. In addition, the developer may program code in DSS execution template 1332 in fourth plug-in service 2650 to detect an abnormal event or threshold in the data stream that it receives. When an abnormal event or threshold is detected, an alert may be sent to some external emergency management application. When the embedded intermediate processing nodes 2624-a, b, c receive responses from their associated access execution systems 2634-a, … g, the code in the intermediate processing node post-processing template 1425-a in the fourth plug-in service 2650 may aggregate and further analyze the data from the multiple access execution systems and then send the analysis results to the calling system 2614. At the same time, intermediate processing nodes 2624-a, b, c may also send alarms to some external emergency management applications. When the calling system 2614 receives results from the intermediate processing nodes 2624-a, b, c, the code in the GDDS post-processing template 1325 of the fourth add-in service 2650 will further analyze the data and generate decision support information for external emergency management applications. The code in the GDDS post-processing template 1325 of the fourth plug-in service 2650 described above may also generate the necessary reports.

The fourth plug-in service 2650 described above is merely one example intended to demonstrate that distributed data processing and analysis programs can be easily created, published, installed, and initiated on a large-scale internet of things over a real-time data network. Without the real-time data network described above, the internet of things data processing and analysis software is typically static. Typically, all software updates on the nodes of the internet of things must be done manually. Through the real-time data network, not only can software be rapidly released and installed, but also a plurality of plug-in service programs can be simultaneously released and executed in the network. In addition, the software update may also be performed automatically. For example, after viewing the data results, the developer may wish to add more logic in the fourth plug-in service 2650 described above, or the developer may wish to add more emergency management applications as their alert targets, through the real-time data network, software updates may be completed very quickly and published on the internet of things network.

The sequence of the above embodiments is only for convenience of description, and does not represent the advantages and disadvantages of the embodiments.

It can be seen from the above embodiments that the embodiments of the present disclosure provide a real-time data network system and a method for querying and processing distributed data, which relate to the technical field of distributed data, and can solve the problem of real-time querying and processing real-time data of a large-scale distributed multi-structure.

one or more invocation systems; and

one or more access enforcement systems; wherein,

In some embodiments of the present disclosure, the distributed data stored by the data storage system comprises: structured data, semi-structured data, and unstructured data, or any combination thereof.

an intermediate layer access execution system;

a middle layer calling system; and

a middle layer data storage system; wherein,

In some embodiments of the present disclosure, the above system further comprises: a plug-in service programming framework for providing a programming template for developing a plug-in service package for a developer; wherein the programming template allows the developer to define an application program interface API of each plug-in service in the plug-in service package, define execution sites of each plug-in service in the plug-in service package, and create code of each plug-in service executed in the calling system and/or the access execution system; wherein the API is programming interface instructions used by the external application.

In some embodiments of the disclosure, the execution place includes one of the calling system and the access execution system.

In some embodiments of the disclosure, the execution place comprises one of the calling system, an intermediate processing node, and the access execution system.

In some embodiments of the present disclosure, the programming template comprises: calling a system verification template, calling a system preprocessing template, calling a system execution template, calling a system post-processing template and accessing an execution system execution template.

In some embodiments of the present disclosure, the programming template further comprises: the intermediate processing node pre-processes the template, the intermediate processing node execution template and the intermediate processing node post-processes the template.

In some embodiments of the present disclosure, the plug-in service programming framework further provides the developer with an access execution system API by using which the developer can launch and execute its associated data integration program and data adapter in each of the access execution systems in a plug-in service.

In some embodiments of the present disclosure, the above method further comprises: providing a programming template for developing a plug-in service package for a developer; wherein the programming template allows the developer to define an application program interface API of each plug-in service program in the plug-in service program package, define execution locations of each plug-in service program in the plug-in service program package, and write codes of each plug-in service program executed in the calling system and/or the access execution system; wherein the API is programming interface instructions used by the external application.

In some embodiments of the present disclosure, the programming template comprises: invoking a system verification template; invoking a system preprocessing template; calling a system execution template; and calling a system post-processing template and accessing an execution system execution template.

In some embodiments of the disclosure, the accessing and processing the distributed data stored by the data storage system by each calling system through one or more access execution systems connected with the calling system comprises:

in response to determining that the execution place is a calling system, the calling system executes codes in the calling system execution template and returns a first execution result to an external application program;

in response to determining that the execution place is an access execution system, the calling system executes codes in the calling system preprocessing template, and sends processed access and instructions for processing distributed data to the access execution system; the access execution system executes codes in the access execution system execution template and feeds back a second execution result to the calling system; and the calling system receives the second execution result from the access execution system, executes codes in the post-processing template of the calling system, and returns a third execution result to an external application program.

In some embodiments of the present disclosure, the programming template further comprises: preprocessing a template by an intermediate processing node; the intermediate processing node executes the template and the intermediate processing node post-processes the template.

in response to determining that the execution place is a calling system, the calling system executes codes in the calling system execution template and returns a fourth execution result to an external application program;

in response to determining that the execution site is an intermediate processing node, the calling system executes code in the calling system preprocessing template, and sends processed instructions for accessing and processing distributed data to the intermediate processing node; the intermediate processing node executes codes in the intermediate processing node execution template and feeds back a fifth execution result to the calling system; the calling system receives the fifth execution result from the intermediate processing node, executes codes in the post-processing template of the calling system, and returns a sixth execution result to an external application program;

in response to determining that the execution site is an access execution system, the calling system executes code in the calling system preprocessing template, and sends processed access and instructions for processing distributed data to an intermediate processing node; the intermediate processing node executes codes in the intermediate processing node preprocessing template and sends processed access and instructions for processing distributed data to an access execution system; the access execution system executes codes in the access execution system execution template and feeds back a seventh execution result to the intermediate processing node; the intermediate processing node receives the seventh execution result from the access execution system, executes codes in an intermediate processing node post-processing template, and feeds back the eighth execution result to the calling system; and the calling system receives the eighth execution result from the intermediate processing node, executes codes in the calling system post-processing template, and returns the ninth execution result to an external application program.

In some embodiments of the present disclosure, the above method further comprises: in response to determining that the code is filled in the calling system verification template, the calling system executes the code in the calling system verification template before executing other plug-in services.

In some embodiments of the present disclosure, the calling system executing code in the calling system preprocessing template comprises: responding to the condition that codes in the calling system preprocessing template are empty, and sending the received instructions for accessing and processing the distributed data to an access execution system by the calling system; responding to the fact that codes in the calling system preprocessing template are not empty, the calling system firstly executes the codes in the calling system preprocessing template, and sends processed access and instructions for processing distributed data to an access execution system; the executing the code in the call system post-processing template comprises: responding to the condition that codes in the post-processing template of the calling system are empty, and returning the received execution result to the external application program by the calling system; and responding to the fact that the codes in the post-processing template of the calling system are not empty, executing the codes in the post-processing template of the calling system by the calling system, and returning the processed execution result to the external application program.

In some embodiments of the present disclosure, the intermediate processing node executing code in the intermediate processing node preprocessing template comprises: responsive to the intermediate processing node preprocessing the code in the template being null, the intermediate processing node transmitting the received instructions to access and process the distributed data to an access execution system; responding to the fact that codes in the intermediate processing node preprocessing template are not empty, the intermediate processing node firstly executes the codes in the intermediate processing node preprocessing template, and sends processed access and instructions for processing distributed data to an access execution system; the executing code in the intermediate processing node post-processing template comprises: responding to the condition that codes in the intermediate processing node post-processing templates are empty, and returning the received execution results to the calling system by the intermediate processing node; and responding to the fact that the codes in the intermediate processing node post-processing template are not null, executing the codes in the intermediate processing node post-processing template by the intermediate processing node, and returning the processed execution result to the calling system.

Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A real-time data network system, comprising:

one or more invocation systems; and

one or more access enforcement systems; wherein,

2. The system of claim 1, wherein each calling system installs a first plug-in service and a second plug-in service; wherein,

the calling system is used for:

receiving a first application programming interface instruction sent by the external application program through the interface for accessing and processing the distributed data;

determining an execution place of the first application programming interface instruction;

in response to determining that the execution site is an access execution system, initiating a first plug-in service associated with the first application programming interface instruction to cause the calling system to send second application programming interface instructions to one or more access execution systems associated therewith and access distributed data stored by the data storage system through the one or more access execution systems; and

And in response to determining that the execution place is a calling system, starting a second plug-in service program related to the first application programming interface instruction, acquiring data according to data processing logic set by the second plug-in service program, processing the acquired data, and feeding the processed data back to the external application program through the interface for accessing and processing the distributed data.

3. The system of claim 2, wherein each access execution system is installed with a third plug-in service; wherein,

and the access execution system receives a second application programming interface instruction sent by the calling system, and starts a third plug-in service program related to the second application programming interface instruction so that the access execution system reads the distributed data from the data storage system, and processes the distributed data according to data processing logic set by the third plug-in service program to obtain first response data.

4. A system according to claim 3, wherein each calling system is installed with a fourth plug-in service;

the calling system is further used for receiving the first response data from the access execution system, starting a fourth plug-in service program related to the first application programming interface instruction, processing the first response data to obtain second response data, and feeding back the second response data to the external application program through the interface for accessing and processing the distributed data.

5. The system of claim 3, wherein the access enforcement system is configured with a data adapter; the access execution system matches the distributed data through the data adapter and reads the distributed data from the data storage system.

6. The system of claim 5, wherein the access execution system is further installed with a data integration program; the access execution system converts the data model of the distributed data into the data model required by the third plug-in service by starting the data integration program.

7. The system of claim 2, wherein each access execution system installs a third plug-in service and a fifth plug-in service; wherein,

the access execution system receives a second application programming interface instruction sent by the calling system and starts a fifth plug-in service program related to the second application programming interface instruction; the fifth plug-in service program calls the third plug-in service program so that the access execution system reads the distributed data from the data storage system and processes the distributed data according to the data processing logic set by the third plug-in service program to obtain intermediate data; and processing the intermediate data according to the data processing logic set by the fifth plug-in service program to obtain first response data.

8. The system of claim 1, wherein the distributed data stored by the data storage system comprises: structured data, semi-structured data, and unstructured data, or any combination thereof.

9. The system of claim 1, further comprising: an intermediate processing node and an intermediate storage system; wherein,

the intermediate processing node is connected with a calling system and one or more access execution systems, and is used for receiving the instructions for accessing and processing the distributed data from the calling system, sending the instructions for accessing and processing the distributed data to the one or more access execution systems, receiving response data from the one or more access execution systems, processing the response data and sending the response data to the calling system;

the intermediate storage system is used for storing response data processed by the intermediate processing node.

10. The system of claim 1, further comprising:

an intermediate layer access execution system;

a middle layer calling system; and

a middle layer data storage system; wherein,

11. The system of any of claims 2-10, further comprising: the data network plug-in service management system is used for managing all plug-in service programs installed in the calling system and the access execution system; wherein,

the data network plug-in service management system comprises:

a plug-in service programming framework for providing a programming framework for developing plug-in service packages for developers; and

and the program management module is used for managing and releasing the plug-in service program package developed by the developer so that the calling system and the access execution system install the plug-in service program package developed by the developer.

12. The system of claim 2, wherein the access enforcement system comprises:

a data integration programming framework for providing a programming framework for developing a data integration program for a developer; and

a data adapter configuration framework for providing a developer with an interface to enter an adapter name field, select and reference a data adapter micro-service, and enter address and access information fields for a data storage system.

13. The system of claim 1, further comprising: the data network management system is used for managing each system in the real-time data network; wherein,

the data network management system includes:

the calling system and the access execution system account management module are used for creating or deleting the calling system and the access execution system; and

and the calling system and access execution system network topology management module is used for adding or deleting the calling system and the access execution system in the real-time data network system and managing the network connection relationship between the calling system and the access execution system.

14. The system of claim 1, further comprising: a plug-in service programming framework for providing a programming template for developing a plug-in service package for a developer; wherein the programming template allows the developer to define an application program interface API of each plug-in service in the plug-in service package, define execution sites of each plug-in service in the plug-in service package, and create code of each plug-in service executed in the calling system and/or the access execution system; wherein the API is programming interface instructions used by the external application.

15. The system of claim 14, wherein the execution place comprises one of the invocation system and the access execution system.

16. The system of claim 14, wherein the execution place comprises one of the calling system, an intermediate processing node, and the access execution system.

17. The system of claim 14, wherein the programming template comprises: calling a system verification template, calling a system preprocessing template, calling a system execution template, calling a system post-processing template and accessing an execution system execution template.

18. The system of claim 14, wherein the programming template further comprises: the intermediate processing node pre-processes the template, the intermediate processing node execution template and the intermediate processing node post-processes the template.

19. The system of claim 14, wherein the plug-in service programming framework further provides access execution system APIs for the developer, using which the developer can launch and execute their associated data integration programs and data adapters in a plug-in service in each of the access execution systems.

20. A method of querying and processing distributed data applied to a real-time data network system, wherein the real-time data network system comprises: one or more invocation systems; and one or more access enforcement systems; wherein the method of querying and processing distributed data comprises:

each calling system receives an access instruction from an external application program and an instruction for processing distributed data respectively; and

and each calling system accesses and processes the distributed data stored by the data storage system through one or more access execution systems connected with the calling system.

21. The method of claim 20, wherein,

the instructions for each calling system to receive access from an external application and process distributed data include: the calling system receives a first application programming interface instruction sent by the external application program through an interface for accessing and processing the distributed data; and

each calling system accesses and processes the distributed data stored by the data storage system through one or more access execution systems connected with the calling system, wherein the distributed data comprises:

the calling system determines the execution place of the first application programming interface instruction;

In response to determining that the execution site is an access execution system, the calling system initiates a first plug-in service associated with the first application programming interface instruction, sends second application programming interface instructions to one or more access execution systems associated with the first plug-in service, and accesses distributed data stored by the data storage system through the one or more access execution systems;

22. The method of claim 21, wherein the accessing, by the one or more access enforcement systems, the distributed data stored by the data storage system comprises:

the access execution system receives a second application programming interface instruction sent by the calling system;

starting a third plug-in service program related to the second application programming interface instruction;

Reading the distributed data from the data storage system; and

and processing the distributed data according to the data processing logic set by the third plug-in service program to obtain first response data.

23. The method of claim 22, wherein the accessing, by the one or more access enforcement systems, the distributed data stored by the data storage system further comprises: the calling system receives the first response data from the access execution system, starts a fourth plug-in service program related to the first application programming interface instruction, processes the first response data according to data processing logic set by the fourth plug-in service program to obtain second response data, and feeds the second response data back to the external application program through the interface for accessing and processing the distributed data.

24. The method of claim 22, wherein the accessing, by the one or more access enforcement systems, the distributed data stored by the data storage system further comprises: and the third plug-in service program of the one or more access execution systems converts the data model of the distributed data into the data model required by the third plug-in service program by starting the data integration program.

25. The method of claim 21, wherein the accessing, by the one or more access enforcement systems, the distributed data stored by the data storage system comprises: the access execution system receives a second application programming interface instruction sent by the calling system; starting a fifth plug-in service related to the second application programming interface instruction; the fifth plug-in service program calls a third plug-in service program; reading the distributed data from the data storage system; processing the distributed data according to the data processing logic set by the third plug-in service program to obtain intermediate data; and processing the intermediate data according to the data processing logic set by the fifth plug-in service program to obtain first response data.

26. The method of claim 20, further comprising:

acquiring an account of a calling system and the topological position of the calling system in a real-time data network system;

registering a calling system in a real-time data network management system according to the calling system account and the topological position of the calling system in the real-time data network system; and

Receiving a first security token corresponding to the calling system from the real-time data network management system;

determining whether the calling system has an associated access execution system according to the topological position of the calling system in a real-time data network system; and

in response to determining that there are associated access execution systems, notifying all of the associated access execution systems to be associated with the calling system, and establishing a communication connection between the access execution system and the calling system.

27. The method of claim 20, further comprising:

acquiring an account of an access execution system and a topological position of the access execution system in a data network system;

registering an access execution system in a real-time data network management system according to the account of the access execution system and the topological position of the access execution system in the data network system;

receiving a second security token corresponding to the access execution system from the real-time data network management system; and

and associating the access execution system to a calling system associated with the access execution system based on the topological position of the access execution system in the data network system, and establishing communication connection between the access execution system and the calling system.

28. The method of claim 26 or 27, wherein establishing a communication connection between the access execution system and a calling system comprises:

the access execution system provides the second security token to the calling system;

the calling system provides a first access token to the access execution system;

the access execution system provides a second access token to the upper layer calling system; wherein the first access token and the second access token are used for communication between the calling system and the access execution system.

29. The method of claim 20, further comprising:

sending an installation request to a calling system to install the plug-in service package;

the calling system installs all or part of plug-in service programs in the plug-in service program package into a pre-installed program library;

the calling system informs all the associated access execution systems to install the plug-in service package; and

each access execution system installs all or part of plug-in service programs in the plug-in service program package into an installation program library of the plug-in service program package;

the access execution systems each configure a data adapter for the plug-in service to access distributed data stored by a data storage system connected to the access execution system and associate the plug-in service with the data adapter.

30. The method of claim 29, further comprising:

each access execution system provides a programming framework of a data integration program and acquires the data integration program from the programming framework of the data integration program; the data integration program is used for converting the data model of the distributed data into a data model required by the third plug-in service program;

the third plug-in service is associated with the data integration program.

31. The method of claim 20, further comprising: providing a programming template for developing a plug-in service package for a developer; wherein the programming template allows the developer to define an application program interface API of each plug-in service program in the plug-in service program package, define execution locations of each plug-in service program in the plug-in service program package, and write codes of each plug-in service program executed in the calling system and/or the access execution system; wherein the API is programming interface instructions used by the external application.

32. The method of claim 31, wherein the programming template comprises: invoking a system verification template; invoking a system preprocessing template; calling a system execution template; and calling a system post-processing template and accessing an execution system execution template.

33. The method of claim 32, wherein each calling system performs system access through one or more accesses connected thereto and processes distributed data stored by the data storage system comprises:

34. The method of claim 32, wherein the programming template further comprises: preprocessing a template by an intermediate processing node; the intermediate processing node executes the template and the intermediate processing node post-processes the template.

35. The method of claim 34, wherein each calling system performs system access through one or more accesses connected thereto and processes distributed data stored by the data storage system comprises:

36. The method as in claim 32, further comprising: in response to determining that the code is filled in the calling system verification template, the calling system executes the code in the calling system verification template before executing other plug-in services.

37. The method of claim 33 or 35, wherein,

the calling system executing the code in the calling system preprocessing template comprises the following steps: responding to the condition that codes in the calling system preprocessing template are empty, and sending the received instructions for accessing and processing the distributed data to an access execution system by the calling system; responding to the fact that codes in the calling system preprocessing template are not empty, the calling system firstly executes the codes in the calling system preprocessing template, and sends processed access and instructions for processing distributed data to an access execution system;

the executing the code in the call system post-processing template comprises: responding to the condition that codes in the post-processing template of the calling system are empty, and returning the received execution result to the external application program by the calling system; and responding to the fact that the codes in the post-processing template of the calling system are not empty, executing the codes in the post-processing template of the calling system by the calling system, and returning the processed execution result to the external application program.

38. The method of claim 35, wherein,

the intermediate processing node executing code in the intermediate processing node preprocessing template includes: responsive to the intermediate processing node preprocessing the code in the template being null, the intermediate processing node transmitting the received instructions to access and process the distributed data to an access execution system; responding to the fact that codes in the intermediate processing node preprocessing template are not empty, the intermediate processing node firstly executes the codes in the intermediate processing node preprocessing template, and sends processed access and instructions for processing distributed data to an access execution system;

the executing code in the intermediate processing node post-processing template comprises: responding to the condition that codes in the intermediate processing node post-processing templates are empty, and returning the received execution results to the calling system by the intermediate processing node; and responding to the fact that the codes in the intermediate processing node post-processing template are not null, executing the codes in the intermediate processing node post-processing template by the intermediate processing node, and returning the processed execution result to the calling system.