CN112800289A - Metadata query method and system - Google Patents

Abstract

The invention provides a metadata query method and a metadata query system, and belongs to the technical field of cloud computing. The method comprises the following steps: when the cluster name in the metadata query request is matched with the local autonomous metadata cluster, acquiring a query result from the local autonomous metadata cluster according to metadata query information in the metadata query request; otherwise, determining a target metadata agent end in the pre-constructed metadata query network according to the cluster name; sending the metadata query request to a target metadata agent end so that the target metadata agent end obtains a query result from a corresponding non-local autonomous metadata cluster according to metadata query information; and receiving a query result returned by the target metadata agent. The invention avoids the problem of multiple copies of metadata on the basis of maintaining the efficiency of metadata autonomy and fault isolation in the information system, and simplifies the technical difficulty and the management cost of strategic decision support of the cross-information system of large enterprises.

Description

Metadata query method and system

Technical Field

The invention relates to the technical field of cloud computing, in particular to a metadata query method and a metadata query system.

Background

In engineering research of information systems, it has been found that data such as configuration information of software and access information (e.g., IP address, port, etc.) of software services have the following commonality and are classified into one type by system designers, which is referred to as metadata (meta data).

1) The universality is that no matter the scale of the information system, the information is ubiquitous and the difference is only divided by the scale;

2) the information is persistent, and the information cannot be lost for the operation of an information system in the program operation process;

3) the reading and writing are less, and the information of the type is changed less, so that new values do not need to be written frequently, and the new values do not need to be added or deleted frequently.

Metadata interacts with the information system in which it is used very frequently, and is often stored in the information system in which it is used by design for the purpose of fault isolation and execution efficiency. But also, therefore, such distributed storage presents two challenges within a large enterprise that owns a large number of information systems.

One aspect is the problem of multiple copies of metadata across system information. Besides internal autonomous operation, a modern information system almost inevitably interacts with other information systems, so that the metadata of the system contains external information such as IP addresses, port numbers, user names and the like of other information systems. In this process, the external information of the other information system is "copied", one being the original copy in the other information system and the other being the copied copy placed in the present information system. Once the original copy is changed, the copied copy of the information system is not updated in time, which results in failure of interaction between the two information systems, which is a problem of multiple copies. The more information systems an information system needs to interact with, the higher the risk of this problem.

Another aspect is the increased difficulty of enterprise management. Information systems in a large enterprise with high informatization degree are often respectively responsible for a part of business of the enterprise and even a part of basic capability, and daily operation of the enterprise needs interaction and cooperation among different information systems. The method for maintaining the metadata of each information system increases the operation complexity of strategic analysis such as cross-information system drawing business process and the like, is not beneficial to strategic decision support, and increases the design workload in aspects of communication planning and the like when a new information system is built; the centralized storage of all metadata causes overheating of the centrally stored system/infrastructure, which presents a great challenge to performance capacity design, and when the system fails, all information systems are detected, and the design principle of fault isolation among the information systems is violated.

Disclosure of Invention

The embodiment of the invention mainly aims to provide a metadata query method and a metadata query system, which can avoid the problem of multiple copies of metadata on the basis of maintaining the efficiency of metadata autonomy and fault isolation in an information system, and simplify the technical difficulty and the management cost of strategic decision support of a large-scale enterprise across information systems.

In order to achieve the above object, an embodiment of the present invention provides a metadata query method, including:

receiving a metadata query request;

judging whether the cluster name in the metadata query request is matched with a local autonomous metadata cluster or not;

when the cluster name is matched with the local autonomous metadata cluster, acquiring a query result from the local autonomous metadata cluster according to metadata query information in the metadata query request;

when the cluster name is not matched with the local autonomous metadata cluster, determining a target metadata proxy end in a pre-constructed metadata query network according to the cluster name; the target metadata agent end is a metadata agent end corresponding to the non-local autonomous metadata cluster matched with the cluster name;

sending the metadata query request to a target metadata agent end so that the target metadata agent end obtains a query result from a corresponding non-local autonomous metadata cluster according to metadata query information;

and receiving a query result returned by the target metadata agent.

An embodiment of the present invention further provides a metadata query system, including:

a receiving unit for receiving a metadata query request;

the judging unit is used for judging whether the cluster name in the metadata query request is matched with the local autonomous metadata cluster;

the query result unit is used for acquiring a query result from the local autonomous metadata cluster according to metadata query information in the metadata query request when the cluster name is matched with the local autonomous metadata cluster;

the target metadata proxy end unit is used for determining a target metadata proxy end in a pre-constructed metadata query network according to the cluster name when the cluster name is not matched with the local autonomous metadata cluster; the target metadata agent end is a metadata agent end corresponding to a non-local autonomous metadata cluster matched with the cluster name;

the sending unit is used for sending the metadata query request to the target metadata proxy end so that the target metadata proxy end can obtain a query result from the corresponding non-local autonomous metadata cluster according to the metadata query information;

the query result unit is further configured to: and receiving a query result returned by the target metadata agent.

The embodiment of the present invention further provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor implements the steps of the metadata query method when executing the computer program.

Embodiments of the present invention further provide a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the steps of the metadata query method.

According to the metadata query method and system, when the cluster name is matched with the local autonomous metadata cluster, a query result is obtained from the local autonomous metadata cluster according to the metadata query information in the metadata query request, otherwise, a target metadata proxy end in a metadata query network which is constructed in advance is determined according to the cluster name, the metadata query request is sent to the target metadata proxy end, and finally the query result returned by the target metadata proxy end is received.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained based on these drawings without creative efforts.

FIG. 1 is a flow diagram of a method for metadata query in an embodiment of the invention;

FIG. 2 is a flow chart of a method for metadata query in another embodiment of the present invention;

FIG. 3 is a flow diagram of building a metadata query network in one embodiment of the invention;

FIG. 4 is a schematic diagram of a metadata query network;

FIG. 5 is a flow diagram of building a metadata query network in another embodiment of the invention;

FIG. 6 is a schematic diagram of an initial metadata query network in one embodiment of the invention;

FIG. 7 is a diagram illustrating a metadata query network after a metadata broker is added in accordance with an embodiment of the present invention;

FIG. 8 is a diagram illustrating a metadata broker broadcasting the broker information in accordance with an embodiment of the present invention;

fig. 9 is a diagram illustrating the metadata broker updating a local reachability list in accordance with an embodiment of the present invention;

FIG. 10 is a diagram illustrating a metadata broker forwarding broadcast broker information in accordance with an embodiment of the present invention;

fig. 11 is a diagram illustrating the metadata broker further forwarding the broadcast broker information according to an embodiment of the present invention;

fig. 12 is a diagram illustrating that the metadata broker further forwards the broadcast broker information according to an embodiment of the present invention;

FIG. 13 is a diagram illustrating metadata broker maintenance completion in accordance with an embodiment of the present invention;

FIG. 14 is a schematic diagram of query results obtained from local autonomous metadata clusters in an embodiment of the present invention;

FIG. 15 is a schematic diagram of query results obtained from locally reachable autonomous metadata clusters in an embodiment of the invention;

FIG. 16 is a schematic diagram of query results obtained from globally reachable autonomous metadata clusters in an embodiment of the present invention;

FIG. 17 is a block diagram showing the structure of a metadata query system according to an embodiment of the present invention;

fig. 18 is a block diagram of a computer device in the embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

As will be appreciated by one skilled in the art, embodiments of the present invention may be embodied as a system, apparatus, device, method, or computer program product. Accordingly, the present disclosure may be embodied in the form of: entirely hardware, entirely software (including firmware, resident software, micro-code, etc.), or a combination of hardware and software.

In view of the problems of multiple copies of metadata of cross-system information and difficulty in enterprise management in the prior art, embodiments of the present invention provide a method and a system for constructing a metadata query network, a method and a system for querying metadata, which avoid the problems of multiple copies of metadata on the basis of maintaining efficiency and fault isolation of metadata autonomy in an information system, simplify technical difficulty and management cost of cross-information system strategic decision support for large-scale enterprises, and meet requirements of metadata autonomy and strategic view of enterprise management in the information system. The present invention will be described in detail below with reference to the accompanying drawings.

FIG. 1 is a flow chart of a metadata query method in an embodiment of the invention. Fig. 2 is a flowchart of a metadata query method according to another embodiment of the present invention.

In the process of metadata query, the metadata proxy end has a temporary role score according to the difference of the upstream and downstream of the process. The metadata agent end directly receiving the metadata inquiry request sent by the user is an initial metadata agent end; the metadata agent end of the inquired metadata in the autonomous metadata cluster in charge of the metadata agent end is a target metadata agent end; the metadata agent end for transmitting the query result between the initial metadata agent end and the target metadata agent end is a relay metadata agent end. The metadata agent terminals with three different roles in the invention do not need additional configuration and share the same set of execution logic, so as to ensure that the design is simple enough and enhance the robustness of the metadata agent terminals.

The execution subject of the metadata query method shown in fig. 1-2 may be the initial metadata agent. As shown in fig. 1-2, the metadata query method includes:

s101: a metadata query request is received.

S102: and judging whether the cluster name in the metadata query request is matched with the local autonomous metadata cluster.

S103: and when the cluster name is matched with the local autonomous metadata cluster, acquiring a query result from the local autonomous metadata cluster according to the metadata query information in the metadata query request.

S104: and when the cluster name is not matched with the local autonomous metadata cluster, determining a target metadata proxy end in the pre-constructed metadata query network according to the cluster name.

The target metadata proxy end is a metadata proxy end corresponding to the non-local autonomous metadata cluster matched with the cluster name.

In one embodiment, determining a target metadata proxy in a pre-constructed metadata query network according to a cluster name includes: and judging whether the cluster name is matched with the non-local autonomous metadata cluster in the local reachability list. And when the cluster name is matched with the non-local autonomous metadata cluster in the local reachability list, determining that the metadata proxy end corresponding to the non-local autonomous metadata cluster is the target metadata proxy end. The target metadata proxy is located in the local reachability list and is directly connected to the starting metadata proxy serving as an execution subject.

When the cluster name does not match a non-local autonomous metadata cluster in the local reachability list, determining whether the cluster name matches a non-local autonomous metadata cluster in the global reachability list. And when the cluster name is matched with the non-local autonomous metadata cluster in the global reachability list, determining that the metadata proxy end corresponding to the non-local autonomous metadata cluster is the target metadata proxy end. The target metadata proxy end is located in the global reachability list and is indirectly connected with the initial metadata proxy end serving as an execution main body through the relay metadata proxy end.

In one embodiment, the method further comprises: and when the target metadata proxy end is positioned in the global reachability list, determining a relay metadata proxy end in the metadata query network according to the target metadata proxy end.

S105: and sending the metadata query request to a target metadata agent end so that the target metadata agent end obtains a query result from the corresponding non-local autonomous metadata cluster according to the metadata query information.

In one embodiment, sending the metadata query request to the target metadata broker includes:

and sending the metadata query request to the target metadata proxy end through the relay metadata proxy end.

In specific implementation, when the target metadata proxy end is located in the global reachability list, the starting metadata proxy end serving as the execution subject selects a path with the shortest time consumption from paths which can reach the target metadata proxy end, and sends the metadata query request to the corresponding relay metadata proxy end or the target metadata proxy end according to the path. When the metadata query request is sent to the relay metadata proxy end, the relay metadata proxy end can select a path with the shortest time consumption again from paths which can reach the target metadata proxy end, and the metadata query request is sent to the corresponding relay metadata proxy end or the target metadata proxy end according to the path. And the target metadata agent end acquires a query result from the non-local autonomous metadata cluster according to the metadata query information, and returns the query result to the initial metadata agent end through the relay metadata agent end.

S106: and receiving a query result returned by the target metadata agent.

And when the cluster name is not matched with the non-local autonomous metadata cluster in the global reachability list, returning query failure information to the user.

The metadata query request comprises query time set by a user. When the cluster name is matched with the non-local autonomous metadata cluster, the query time is updated according to the time of the metadata query request in the metadata proxy end (the query time is subtracted by the time of the data query request in the metadata proxy end), and the updated query time is sent to the corresponding metadata proxy end. When the updated query time is less than or equal to zero, the query time is insufficient, and at the moment, the metadata agent end where the query request or the query result is located returns query timeout information to the user.

As can be seen from the process shown in fig. 1, in the metadata query method according to the embodiment of the present invention, when the cluster name matches the local autonomous metadata cluster, a query result is obtained from the local autonomous metadata cluster according to the metadata query information in the metadata query request, otherwise, a target metadata proxy in a metadata query network that is pre-constructed is determined according to the cluster name, the metadata query request is sent to the target metadata proxy, and finally, a query result returned by the target metadata proxy is received.

FIG. 3 is a flow diagram of building a metadata query network in an embodiment of the invention. Fig. 4 is a schematic diagram of a metadata query network. FIG. 5 is a flow diagram of building a metadata query network in another embodiment of the invention. As shown in fig. 3-5, the metadata proxy in the metadata query network internally uses an independent structure to perform an ad hoc process of the metadata query network, and the building of the metadata query network includes:

s201: and acquiring the contact information of the designated metadata agent end in the initial metadata query network.

The designated metadata agent contact information comprises positioning information of the metadata agent in the system, such as an IP address, a port, an autonomous metadata cluster in charge and the like. The initial metadata query network (existing system) joins the metadata query network before the metadata query network for the original data agent. If the contact information of the appointed metadata agent end is not received, the metadata agent end is used as the first node of the metadata inquiry network to be directly started successfully.

S202: and sending a registration request to the appointed metadata agent end corresponding to the appointed metadata agent end contact information, and establishing the inter-agent connection with the appointed metadata agent end.

In specific implementation, the connection between the agents can be established to the designated metadata agent one by one, and the contact information of the metadata agent can be registered to the other side. And when all the registration requests fail, the starting fails and the operation is quitted.

S203: and receiving the non-local system topology from the designated metadata agent end, and creating the local system topology according to the non-local system topology.

The system topology comprises a local system topology and a non-local system topology, and the contact information of the metadata proxy end and the known metadata proxy end in the same system (metadata query network) is recorded by taking the located metadata proxy end as a unit. The local system topology and the non-local system topology each include a local reachability list and a global reachability list. The local reachability list comprises other metadata proxy ends which can directly communicate with the metadata proxy end where the local reachability list is located in the current metadata query network and contact information of the other metadata proxy ends, and the global reachability list comprises other metadata proxy ends which can indirectly communicate with the metadata proxy end where the local reachability list is located in the current metadata query network and contact information of the other metadata proxy ends. And the metadata agent end stores the system topology in a memory of the metadata agent end and volatilizes along with the disappearance of the progress of the metadata agent end.

S204: and determining a local non-designated metadata proxy end in the initial metadata query network according to the local system topology.

The local non-designated metadata proxy end is a local reachability list located in the local system topology and is not the metadata proxy end of the designated metadata proxy end.

S205: and sending the local system topology to the local non-specified metadata agent end, and establishing inter-agent connection with the local non-specified metadata agent end according to a sending result so as to construct a metadata query network.

As shown in fig. 4, the metadata agent as the execution subject is deployed in the autonomous metadata cluster of its agent nearby, provides an external interface for looking up any one of the autonomous metadata clusters to the information system having the autonomous metadata cluster, and obtains and returns the requested metadata when receiving a request for querying the autonomous metadata cluster; the metadata proxy does not store metadata, and can sleep for 50+ delta milliseconds, and the more the number of the metadata proxy written in the local system topology is, the higher the probability of the delta value increase is. Wherein 50 milliseconds is a constant value; delta is a random value with unit of millisecond, and the specific determination process of the delta value is as follows: randomly selecting an integer from 0 to N-1 to multiply by 50, wherein N is the number of metadata agents written in the topology of the local system.

The autonomous metadata cluster is used for storing metadata of the information system and providing query and write-in functions. One autonomous metadata cluster serves only one information system.

An inter-proxy connection is a logical communication channel established between adjacent metadata proxies and combinations thereof. The inter-proxy connection is composed of one or more underlying non-blocking communication sessions, and network communication efficiency is improved by using multi-link aggregation and single-link service technologies.

In one embodiment, S205 includes: and taking out the contact information in the local system topology one by one, sending the local system topology to the local unspecified metadata proxy end, establishing inter-proxy connection with the local unspecified metadata proxy end to construct a metadata query network when the sending result is that the sending is successful, and writing the local unspecified metadata proxy end into a local reachability list.

The metadata agent side in the metadata query network is used for maintaining the topology of a local system or processing a registration request from an external metadata agent side at regular time, and the following tasks are processed in parallel during the period:

the method comprises the steps of firstly, taking out contact information in local system topology one by one according to a principle of preferential local reachability, sending the local system topology to a metadata proxy end which is directly connected with the metadata proxy end in a metadata inquiry network, and writing the metadata proxy end into a local reachability list stored by the metadata proxy end when the sending is successful. Each metadata agent end in the metadata query network only sends local system topology to the metadata agent end connected with the metadata agent end, and each metadata agent end also sends agent information of the metadata agent end to other metadata agent ends in the metadata query network. The agent information includes a system (autonomous metadata cluster) for which the metadata agent is responsible, a distance between itself and an agent information receiver, and a relay record.

And secondly, updating the local system topology according to the non-local system topology of the metadata agent end in the metadata query network.

For example, the non-native system topology is compared to the native system topology:

1. and when the metadata agent end which is not in the local system topology exists in the non-local system topology, writing the metadata agent end and the contact information thereof into the local system topology.

2. And when the metadata agent end which is not in the non-local system topology exists in the local system topology, removing the metadata agent end from the global reachability list of the local system topology.

3. And when the metadata proxy end exists in the local reachability list of the non-local system topology and the metadata proxy end is not in the global reachability list of the local system topology, writing the metadata proxy end into the global reachability list of the local system topology.

4. And when the metadata proxy end does not exist in the local reachability list of the non-local system topology and is in the global reachability list of the local system topology, removing the metadata proxy end from the global reachability list of the local system topology.

In one embodiment, after executing S205, the method further includes: receiving a registration request from an external metadata agent end, updating a local reachability list according to metadata agent end contact information in the registration request, writing the external metadata agent end into the local reachability list, and informing the external metadata agent end of successful registration. Wherein, the external metadata proxy end is positioned outside the current metadata inquiry network.

The specific process of the embodiment of the invention is as follows:

1. and acquiring the contact information of the designated metadata agent end in the initial metadata query network.

2. And sending a registration request to the appointed metadata agent end corresponding to the appointed metadata agent end contact information, and establishing the inter-agent connection with the appointed metadata agent end.

3. And receiving the non-local system topology from the designated metadata agent end, and creating the local system topology according to the non-local system topology.

4. And determining a local non-designated metadata proxy end in the initial metadata query network according to the local system topology.

5. And sending the local system topology to a local unspecified metadata proxy end, establishing inter-proxy connection with the local unspecified metadata proxy end to construct a metadata query network when the sending result is that the sending is successful, and writing the local unspecified metadata proxy end into a local reachability list.

6. And taking out the contact information in the local system topology one by one, sending the local system topology to other metadata proxy terminals in the metadata query network, and writing the metadata proxy terminals into local reachability lists stored by the metadata proxy terminals when the sending is successful.

7. And updating the local system topology according to the non-local system topology from the metadata agent end in the metadata query network.

8. And receiving a registration request from an external metadata agent end, and writing the external metadata agent end into a local reachability list according to the metadata agent end contact information in the registration request.

9. And receiving a metadata query request sent by a user, and judging whether the cluster name in the metadata query request is matched with the local autonomous metadata cluster.

10. And when the cluster name is matched with the local autonomous metadata cluster, acquiring a query result from the local autonomous metadata cluster according to the metadata query information in the metadata query request.

11. And when the cluster name is not matched with the local autonomous metadata cluster, determining a target metadata proxy end in the metadata query network according to the cluster name.

12. And sending the metadata query request to a target metadata agent end so that the target metadata agent end obtains a query result from the corresponding non-local autonomous metadata cluster according to the metadata query information.

13. And receiving a query result returned by the target metadata agent.

14. And returning the query result to the user.

FIG. 6 is a diagram of an initial metadata query network in an embodiment of the invention. Fig. 7 is a schematic diagram of a metadata query network after adding a metadata broker in an embodiment of the present invention. Taking fig. 6-7 as an example, the process of adding the metadata agent to the metadata query network is as follows:

as shown in fig. 6, the initial metadata query network consists of A, B, C three metadata brokers. The system of the metadata agent A agent (autonomous metadata cluster) is A ', the system of the metadata agent B agent is B ', and the system of the metadata agent C agent is C '. Wherein the metadata agents A and C can not directly communicate because of network problems, but B and A, or B and C can communicate. The metadata broker that wishes to join the initial metadata query network is D.

As shown in fig. 7, when the user starts a new metadata broker D, the assignment D synchronizes to C. Therefore, D is successfully started after directly copying the system topology of C; and the metadata agent end C is added with D to a local reachability list in the system topology of the metadata agent end C.

Fig. 8 is a diagram illustrating the metadata broker broadcasting the broker information according to an embodiment of the present invention. Fig. 9 is a diagram illustrating that the metadata broker updates the local reachability list in an embodiment of the present invention. Fig. 10 is a diagram illustrating forwarding of broadcast agent information by a metadata agent in an embodiment of the present invention. Fig. 11 is a diagram illustrating that the metadata broker further forwards the broadcast broker information according to an embodiment of the present invention. Fig. 12 is a diagram illustrating that the metadata broker further forwards the broadcast broker information according to an embodiment of the present invention. Fig. 13 is a schematic diagram of the metadata broker end maintenance completion according to an embodiment of the present invention. Taking fig. 8-fig. 13 as an example, the process of maintaining the system topology by the metadata broker is as follows:

as shown in fig. 8, when the metadata agent D starts to maintain its own system topology, it generates an agent information describing a metadata query of which system it is responsible for, and then tries to directly transmit the agent information to the agents in the local reachability list (C, B) and the global reachability list (a) of the local system topology.

As shown in fig. 9, since the metadata proxy D cannot be directly connected to the metadata proxy a, the metadata proxy D removes information of a from its local reachability list; the metadata agent end D can be directly connected with the metadata agent end B and the metadata agent end C, so that the information of B and C is added into a local reachability list of the metadata agent end D. The metadata agent end B and the metadata agent end C can be directly connected with the metadata agent end D, so that the agent information broadcasted by the metadata agent end D is respectively received, and the B and the C respectively add D into the local reachability list of the B and the C. In addition, the metadata proxy B and the metadata proxy C add one to the "distance from the recipient (D)" in the received proxy information of D, and add their own name to the "relay record" last.

As shown in fig. 10, the metadata agent B and the metadata agent C calculate to which metadata agent the agent information of D is to be sent. Taking the metadata proxy C as an example, the metadata proxy C takes out its local reachability list to obtain B and D, and excludes the metadata proxy that has the local reachability list in the "relay record" (D and C exist inside) from (D is excluded), so that the metadata proxy C only needs to send the proxy information of D to the metadata proxy B; the metadata agent B is also a similar process, and therefore, the metadata agent B sends the agent information of D to the metadata agent a and the metadata agent C.

As shown in fig. 11, three pieces of proxy information D are generated in fig. 11, and their broadcasting order is as follows:

1.D→B→A

(1) updating the system topology of the pathway proxy: and the metadata agent end A receives the agent information of the D, because the agent information is not directly obtained, the agent information of the D is added into the global reachability list of the A, the next hop of the D is marked as the agent which is closest to the metadata agent end A in the relay record, and the shortest distance between the metadata agent end A and the receiver is 2.

(2) Forwarding and broadcasting: since all metadata proxies (B) in the local reachability list of the metadata proxy a appear in the "relay record", the metadata proxy a does not forward the proxy information to any metadata proxy.

2.D→B→C

(1) Updating the system topology of the pathway proxy: the metadata agent terminal C receives the broadcast information of D, the agent information is not directly obtained, but the local reachability list of C already has the information of D, so the metadata agent terminal C does not update the local reachability list or the global reachability list of the metadata agent terminal C.

(2) Forwarding and broadcasting: since all the proxies (B and D) in the local reachability list of the metadata proxy C are present in the "relay record", the metadata proxy C does not forward the proxy information to any metadata proxy.

3.D→C→B

(1) Updating the system topology of the pathway proxy: the metadata agent terminal B receives the broadcast information of the D, the agent information is not directly obtained, but the local reachability list of the B already contains the information of the D, so the metadata agent terminal B does not update the local reachability list or the global reachability list of the metadata agent terminal B.

(2) Forwarding and broadcasting: since C and D appear in the "relay record" in all the proxies (A, C and D) in the local reachability list of the metadata proxy B, the metadata proxy B will only forward the proxy information to the metadata proxy a.

As shown in fig. 12, in terms of updating the system topology of the routing agent, the metadata agent a receives the agent information of D, and because the agent information is not directly obtained, the metadata agent a attempts to add the information of D to the global reachability list of the agent. However, since the global reachability list of a already contains the information of D and the shortest distance (2) to the recipient is smaller than the "distance to the recipient" (3) in the proxy information received this time, the metadata proxy a does not update the global reachability list.

In the aspect of forwarding broadcast, since all metadata proxies (B) in the local reachability list of the metadata proxy a appear in the "relay record", the metadata proxy a does not forward the proxy information to any metadata proxy.

The above-described process of the metadata broker broadcasting the broker information occurs at both the metadata brokers A, B, C and D, and may even occur simultaneously. After all the metadata brokers have completed a round of maintenance as described above, the system topology of all the metadata brokers is updated to the latest.

FIG. 14 is a schematic diagram of obtaining query results from a local autonomous metadata cluster in an embodiment of the present invention. FIG. 15 is a diagram illustrating query results obtained from locally reachable autonomous metadata clusters in an embodiment of the invention. FIG. 16 is a diagram illustrating query results obtained from globally reachable autonomous metadata clusters in an embodiment of the present invention. Taking fig. 14-16 as an example, the process of obtaining the query result is as follows:

as shown in fig. 14, the metadata to be accessed by the user is just in the system D' managed by the metadata broker D, and D directly queries the data and returns the query result to the user.

As shown in fig. 15, the metadata to be accessed by the user is not in the system D' managed by the metadata broker D, but the metadata broker C that finds itself in the local reachability list is directly responsible for the metadata. Therefore, D forwards the request to C, C gives D after inquiring the data, and D returns the inquiry result to the user.

As shown in fig. 16, the metadata to be accessed by the user is not in the system D' managed by the metadata broker D, and there is no metadata broker directly responsible for the metadata in the local reachability list of the metadata broker D, but the metadata broker a in the global reachability list of the metadata broker D is directly responsible for the metadata. Therefore, D forwards the request to B according to the 'next hop' of A in the global reachability list, B carries out query judgment, B → A logic forwards the request and retrieves data from A's system A', B gives the data to D, and D returns the query result to the user.

To sum up, the metadata query method of the embodiment of the present invention obtains a query result from a local autonomous metadata cluster according to metadata query information in a metadata query request when the cluster name matches the local autonomous metadata cluster, otherwise determines a target metadata proxy end in a metadata query network constructed in advance according to the cluster name, sends the metadata query request to the target metadata proxy end, and finally receives the query result returned by the target metadata proxy end, thereby avoiding the multiple-copy problem of metadata on the basis of maintaining the efficiency of metadata autonomy and fault isolation in an information system, and simplifying the technical difficulty and management cost of strategic decision support of a large-scale enterprise across information systems.

In addition, the invention also has the following advantages:

1) elastic expansion: a network architecture formed between the metadata agent ends has self-organization capability, nodes are added and deleted without manually modifying the topology, and a new topology can be automatically generated in the self-organization process.

2) And (3) robustness: the metadata agent end has the advantages that the functions are completely equal to each other and decentralized, strong synchronous information is avoided, and the vulnerability of a centralized distributed architecture can be avoided.

3) Availability: the metadata agent end supports two modes of direct query and agent query at the same time, and can automatically adapt to new network conditions to try to acquire results when a network connection problem occurs.

4) Low invasion: each information system (autonomous metadata cluster) does not need to change an internal metadata access mode, and metadata access across the information systems can be implemented by gradual access and periodic implementation, so that the service of each information system is prevented from being interfered.

5) Correctness: the metadata agent end does not store actual data, and the problem of multiple copies of metadata does not exist, so that the query accuracy is guaranteed.

6) The cost is low: the metadata agent end is designed in a stateless mode, and is not particularly suitable for a high configuration environment to operate, so that extra cost investment is reduced.

7) The efficiency is high: the connection between the agents uses multi-link aggregation and single-link administration technology, so that reliable non-blocking communication which cannot be achieved by a common communication protocol can be realized, and the occupation of network resources is reduced.

Based on the same inventive concept, the embodiment of the invention also provides a metadata query system, and as the principle of solving the problems of the system is similar to that of the metadata query method, the implementation of the system can refer to the implementation of the method, and repeated parts are not described again.

Fig. 17 is a block diagram showing the structure of a metadata query system in the embodiment of the present invention. As shown in fig. 17, the metadata query system includes:

a receiving unit for receiving a metadata query request;

the judging unit is used for judging whether the cluster name in the metadata query request is matched with the local autonomous metadata cluster;

the query result unit is used for acquiring a query result from the local autonomous metadata cluster according to metadata query information in the metadata query request when the cluster name is matched with the local autonomous metadata cluster;

the target metadata proxy end unit is used for determining a target metadata proxy end in a pre-constructed metadata query network according to the cluster name when the cluster name is not matched with the local autonomous metadata cluster; the target metadata agent end is a metadata agent end corresponding to a non-local autonomous metadata cluster matched with the cluster name;

the sending unit is used for sending the metadata query request to the target metadata proxy end so that the target metadata proxy end can obtain a query result from the corresponding non-local autonomous metadata cluster according to the metadata query information;

the query result unit is further configured to: and receiving a query result returned by the target metadata agent.

In one embodiment, the method further comprises the following steps:

the relay metadata proxy end unit is used for determining a relay metadata proxy end in the metadata inquiry network according to the target metadata proxy end when the target metadata proxy end is positioned in the global reachability list;

the sending unit is specifically configured to:

and sending the metadata query request to the target metadata proxy end through the relay metadata proxy end.

In one embodiment, the method further comprises the following steps: a metadata query network construction unit for:

acquiring contact information of a designated metadata agent end in an initial metadata query network;

sending a registration request to a designated metadata agent end corresponding to the designated metadata agent end contact information, and establishing inter-agent connection with the designated metadata agent end;

receiving a non-local system topology from a designated metadata agent end, and creating a local system topology according to the non-local system topology;

determining a local non-designated metadata proxy end in an initial metadata query network according to the local system topology;

and sending the local system topology to the local non-specified metadata agent end, and establishing inter-agent connection with the local non-specified metadata agent end according to a sending result so as to construct a metadata query network.

In one embodiment, the local system topology includes a local reachability list;

the metadata query network construction unit is specifically configured to:

and when the sending result is that the sending is successful, establishing inter-proxy connection with the local non-specified metadata proxy end to construct a metadata query network, and writing the local non-specified metadata proxy end into a local reachability list.

In one embodiment, the method further comprises the following steps:

and the updating unit is used for updating the local system topology according to the non-local system topology from the metadata proxy end in the metadata query network.

In one embodiment, the method further comprises the following steps:

a registration request receiving unit for receiving a registration request from an external metadata agent;

the update unit is further configured to:

and updating the local reachability list according to the metadata agent side contact information in the registration request.

To sum up, when the cluster name is matched with the local autonomous metadata cluster, the metadata query system of the embodiment of the invention obtains a query result from the local autonomous metadata cluster according to the metadata query information in the metadata query request, otherwise, determines a target metadata proxy end in a metadata query network constructed in advance according to the cluster name, sends the metadata query request to the target metadata proxy end, and finally receives the query result returned by the target metadata proxy end, thereby avoiding the multi-copy problem of the metadata on the basis of keeping the efficiency and fault isolation of the metadata autonomy in the information system, and simplifying the technical difficulty and management cost of strategic decision support of a large-scale enterprise cross-information system.

The embodiment of the present invention further provides a specific implementation manner of a computer device, which can implement all the steps in the metadata query method in the foregoing embodiment. Fig. 18 is a block diagram of a computer device in an embodiment of the present invention, and referring to fig. 18, the computer device specifically includes the following:

a processor (processor)1801 and a memory (memory) 1802.

The processor 1801 is configured to call a computer program in the memory 1802, and when the processor executes the computer program, the processor implements all the steps in the metadata query method in the foregoing embodiment, for example, when the processor executes the computer program, the processor implements the following steps:

receiving a metadata query request;

judging whether the cluster name in the metadata query request is matched with a local autonomous metadata cluster or not;

when the cluster name is matched with the local autonomous metadata cluster, acquiring a query result from the local autonomous metadata cluster according to metadata query information in the metadata query request;

when the cluster name is not matched with the local autonomous metadata cluster, determining a target metadata proxy end in a pre-constructed metadata query network according to the cluster name; the target metadata agent end is a metadata agent end corresponding to the non-local autonomous metadata cluster matched with the cluster name;

sending the metadata query request to a target metadata agent end so that the target metadata agent end obtains a query result from a corresponding non-local autonomous metadata cluster according to metadata query information;

and receiving a query result returned by the target metadata agent.

To sum up, when the cluster name is matched with the local autonomous metadata cluster, the computer device of the embodiment of the invention obtains the query result from the local autonomous metadata cluster according to the metadata query information in the metadata query request, otherwise determines the target metadata proxy end in the metadata query network constructed in advance according to the cluster name, sends the metadata query request to the target metadata proxy end, and finally receives the query result returned by the target metadata proxy end, thereby avoiding the multiple copies of metadata on the basis of keeping the efficiency of metadata autonomy and fault isolation in the information system, and simplifying the technical difficulty and management cost of strategic decision support of a large-scale enterprise across the information system.

An embodiment of the present invention further provides a computer-readable storage medium capable of implementing all the steps in the metadata query method in the foregoing embodiment, where the computer-readable storage medium stores a computer program, and the computer program, when executed by a processor, implements all the steps in the metadata query method in the foregoing embodiment, for example, when the processor executes the computer program, the processor implements the following steps:

receiving a metadata query request;

judging whether the cluster name in the metadata query request is matched with a local autonomous metadata cluster or not;

when the cluster name is matched with the local autonomous metadata cluster, acquiring a query result from the local autonomous metadata cluster according to metadata query information in the metadata query request;

when the cluster name is not matched with the local autonomous metadata cluster, determining a target metadata proxy end in a pre-constructed metadata query network according to the cluster name; the target metadata agent end is a metadata agent end corresponding to the non-local autonomous metadata cluster matched with the cluster name;

sending the metadata query request to a target metadata agent end so that the target metadata agent end obtains a query result from a corresponding non-local autonomous metadata cluster according to metadata query information;

and receiving a query result returned by the target metadata agent.

To sum up, when the cluster name matches the local autonomous metadata cluster, the computer-readable storage medium of the embodiment of the present invention obtains a query result from the local autonomous metadata cluster according to the metadata query information in the metadata query request, otherwise determines a target metadata proxy end in a metadata query network constructed in advance according to the cluster name, sends the metadata query request to the target metadata proxy end, and finally receives the query result returned by the target metadata proxy end, thereby avoiding the multi-copy problem of metadata on the basis of maintaining the efficiency of metadata autonomy and fault isolation in the information system, and simplifying the technical difficulty and management cost of strategic decision support of a large-scale enterprise cross-information system.

The above-mentioned embodiments are intended to illustrate the objects, technical solutions and advantages of the present invention in further detail, and it should be understood that the above-mentioned embodiments are only exemplary embodiments of the present invention, and are not intended to limit the scope of the present invention, and any modifications, equivalent substitutions, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.

Those of skill in the art will further appreciate that the various illustrative logical blocks, units, and steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate the interchangeability of hardware and software, various illustrative components, elements, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design requirements of the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present embodiments.

The various illustrative logical blocks, or elements, or devices described in connection with the embodiments disclosed herein may be implemented or performed with a general purpose processor, a digital signal processor, an Application Specific Integrated Circuit (ASIC), a field programmable gate array or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general-purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, e.g., a digital signal processor and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a digital signal processor core, or any other similar configuration.

The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may be stored in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. For example, a storage medium may be coupled to the processor such the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium may be integral to the processor. The processor and the storage medium may reside in an ASIC, which may be located in a user terminal. In the alternative, the processor and the storage medium may reside in different components in a user terminal.

In one or more exemplary designs, the functions described above in connection with the embodiments of the invention may be implemented in hardware, software, firmware, or any combination of the three. If implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media that facilitate transfer of a computer program from one place to another. Storage media may be any available media that can be accessed by a general purpose or special purpose computer. For example, such computer-readable media can include, but is not limited to, RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to carry or store program code in the form of instructions or data structures and which can be read by a general-purpose or special-purpose computer, or a general-purpose or special-purpose processor. Additionally, any connection is properly termed a computer-readable medium, and, thus, is included if the software is transmitted from a website, server, or other remote source via a coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL), or wirelessly, e.g., infrared, radio, and microwave. Such discs (disk) and disks (disc) include compact disks, laser disks, optical disks, DVDs, floppy disks and blu-ray disks where disks usually reproduce data magnetically, while disks usually reproduce data optically with lasers. Combinations of the above may also be included in the computer-readable medium.

Claims (14)

1. A metadata query method, comprising:

receiving a metadata query request;

judging whether the cluster name in the metadata query request is matched with a local autonomous metadata cluster or not;

when the cluster name is matched with the local autonomous metadata cluster, acquiring a query result from the local autonomous metadata cluster according to metadata query information in the metadata query request;

when the cluster name is not matched with the local autonomous metadata cluster, determining a target metadata proxy end in a pre-constructed metadata query network according to the cluster name; the target metadata agent end is a metadata agent end corresponding to a non-local autonomous metadata cluster matched with the cluster name;

sending the metadata query request to the target metadata agent end so that the target metadata agent end obtains a query result from a corresponding non-local autonomous metadata cluster according to the metadata query information;

and receiving a query result returned by the target metadata agent terminal.

2. The metadata query method according to claim 1, further comprising:

when the target metadata proxy end is positioned in a global reachability list, determining a relay metadata proxy end in the metadata query network according to the target metadata proxy end;

sending the metadata query request to the target metadata broker comprises:

and sending the metadata query request to the target metadata proxy end through the relay metadata proxy end.

3. The metadata query method of claim 1, wherein constructing a metadata query network comprises:

acquiring contact information of a designated metadata agent end in an initial metadata query network;

sending a registration request to a designated metadata agent end corresponding to the designated metadata agent end contact information, and establishing inter-agent connection with the designated metadata agent end;

receiving a non-local system topology from the designated metadata agent end, and creating a local system topology according to the non-local system topology;

determining a local non-designated metadata proxy end in the initial metadata query network according to the local system topology;

and sending the local system topology to the local non-specified metadata agent end, and establishing inter-agent connection with the local non-specified metadata agent end according to a sending result to construct a metadata query network.

4. The metadata query method of claim 3, wherein the local system topology comprises a local reachability list;

establishing inter-proxy connection with the local non-specified metadata proxy terminal according to the sending result to construct a metadata query network comprises the following steps:

and when the sending result is that the sending is successful, establishing inter-proxy connection with the local non-specified metadata proxy end to construct a metadata query network, and writing the local non-specified metadata proxy end into the local reachability list.

5. The metadata query method according to claim 4, wherein after establishing an inter-proxy connection with the local non-specified metadata proxy according to the transmission result to construct a metadata query network, the method further comprises:

and updating the local system topology according to the non-local system topology from the metadata agent end in the metadata query network.

6. The metadata query method according to claim 4, further comprising:

receiving a registration request from an external metadata agent terminal;

and updating the local reachability list according to the metadata agent terminal contact information in the registration request.

7. A metadata query system, comprising:

a receiving unit for receiving a metadata query request;

the judging unit is used for judging whether the cluster name in the metadata query request is matched with a local autonomous metadata cluster or not;

the query result unit is used for acquiring a query result from the local autonomous metadata cluster according to metadata query information in the metadata query request when the cluster name is matched with the local autonomous metadata cluster;

the target metadata proxy end unit is used for determining a target metadata proxy end in a pre-constructed metadata query network according to the cluster name when the cluster name is not matched with the local autonomous metadata cluster; the target metadata agent end is a metadata agent end corresponding to a non-local autonomous metadata cluster matched with the cluster name;

the sending unit is used for sending the metadata query request to the target metadata proxy end so that the target metadata proxy end obtains a query result from a corresponding non-local autonomous metadata cluster according to the metadata query information;

the query result unit is further configured to: and receiving a query result returned by the target metadata agent terminal.

8. The metadata query system of claim 7, further comprising:

a relay metadata proxy unit, configured to determine a relay metadata proxy in the metadata query network according to the target metadata proxy when the target metadata proxy is located in the global reachability list;

the sending unit is specifically configured to:

and sending the metadata query request to the target metadata proxy end through the relay metadata proxy end.

9. The metadata query system of claim 7, further comprising: a metadata query network construction unit for:

acquiring contact information of a designated metadata agent end in an initial metadata query network;

sending a registration request to a designated metadata agent end corresponding to the designated metadata agent end contact information, and establishing inter-agent connection with the designated metadata agent end;

receiving a non-local system topology from the designated metadata agent end, and creating a local system topology according to the non-local system topology;

determining a local non-designated metadata proxy end in the initial metadata query network according to the local system topology;

and sending the local system topology to the local non-specified metadata agent end, and establishing inter-agent connection with the local non-specified metadata agent end according to a sending result to construct a metadata query network.

10. The metadata querying system according to claim 9, wherein the local system topology comprises a local reachability list;

the metadata query network construction unit is specifically configured to:

and when the sending result is that the sending is successful, establishing inter-proxy connection with the local non-specified metadata proxy end to construct a metadata query network, and writing the local non-specified metadata proxy end into the local reachability list.

11. The metadata query system of claim 10, further comprising:

and the updating unit is used for updating the local system topology according to the non-local system topology from the metadata agent end in the metadata query network.

12. The metadata query system of claim 10, further comprising:

a registration request receiving unit for receiving a registration request from an external metadata agent;

the update unit is further configured to:

and updating the local reachability list according to the metadata agent terminal contact information in the registration request.

13. A computer device comprising a memory, a processor and a computer program stored on the memory and running on the processor, characterized in that the processor implements the steps of the metadata query method of any one of claims 1 to 6 when executing the computer program.

14. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the metadata query method of any one of claims 1 to 6.

Images