EP2035916A2 - Verfahren und system für föderierten ressourcenentdeckungsdienst in verteilten systemen - Google Patents
Verfahren und system für föderierten ressourcenentdeckungsdienst in verteilten systemenInfo
- Publication number
- EP2035916A2 EP2035916A2 EP07798187A EP07798187A EP2035916A2 EP 2035916 A2 EP2035916 A2 EP 2035916A2 EP 07798187 A EP07798187 A EP 07798187A EP 07798187 A EP07798187 A EP 07798187A EP 2035916 A2 EP2035916 A2 EP 2035916A2
- Authority
- EP
- European Patent Office
- Prior art keywords
- resource
- attribute data
- servers
- hierarchy
- server
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
- 238000000034 method Methods 0.000 title claims abstract description 45
- 238000013507 mapping Methods 0.000 claims description 10
- 230000008569 process Effects 0.000 claims description 9
- 230000004931 aggregating effect Effects 0.000 claims description 5
- 230000003362 replicative effect Effects 0.000 claims description 3
- 238000012545 processing Methods 0.000 abstract description 14
- 238000013468 resource allocation Methods 0.000 abstract description 4
- 230000002776 aggregation Effects 0.000 description 24
- 238000004220 aggregation Methods 0.000 description 24
- 238000004891 communication Methods 0.000 description 20
- 230000008520 organization Effects 0.000 description 18
- 230000015654 memory Effects 0.000 description 13
- 230000006870 function Effects 0.000 description 12
- 230000010076 replication Effects 0.000 description 11
- 230000004044 response Effects 0.000 description 11
- 238000007726 management method Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 4
- 230000003068 static effect Effects 0.000 description 4
- 238000013459 approach Methods 0.000 description 3
- 230000003247 decreasing effect Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 238000012804 iterative process Methods 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000000737 periodic effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 238000007792 addition Methods 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000013455 disruptive technology Methods 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
- 238000011144 upstream manufacturing Methods 0.000 description 1
- 230000002747 voluntary effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2471—Distributed queries
Definitions
- Contemporary applications and services will run on multiple computing systems that belong to different organizations and fall under different administrative authorities.
- resources within all the computing systems need to be discovered and allocated for these applications.
- These resources include computing hardware resources, e.g. central processing unit (CPU) resources, storage capacity, e.g. hard drive size, memory size in physical machines and data collectors or data sensors that collect various types of data such as atmospheric or meteorological data and video or audio data.
- the available resources can be both static and dynamic resources. For example, the memory size or network adaptor speed of a given machine is usually fixed, but the available memory or bandwidth changes frequently over time.
- a repository of available resources from various computing systems belonging to all organizations involved needs to be created and maintained. Creation and maintenance of this repository includes discovering resources that are available for allocation. Resource discovery can locate remote resources subject to the specified requirements of a given resource demand and is widely used in many distributed computing systems for a variety of applications. For example, in grid computing, machines or nodes that possess the required CPU and memory resources to run an application are discovered or identified, and then the application is deployed on those identified machines.
- SWORD scalable wide-area resource discovery tool
- DHT distributed hash table
- Peer-to-peer networks are arranged to allow high autonomy of individual nodes, making it quite difficult to facilitate, especially centralized, control and management in the system.
- the resource discovery tool in SWORD requires that each individual autonomous system export its complete resource records to the peer-to-peer network, possibly any arbitrary node. This requires that all autonomous systems have high degrees of trust among each other, which can become a problem in reality. Individual autonomous systems may not be willing to expose their original records to an arbitrary node in the outside world.
- a plurality of dedicated servers are arranged in a hierarchy such as a tree hierarchy.
- This hierarchy is used to answer user queries about resources.
- each server is a point or node within this hierarchy and is used to aggregate data from its children in a bottom-up fashion in the hierarchy.
- Each child is also a server within the hierarchy.
- One or more servers disposed in each domain or organization containing discoverable resources is in communication with one of the D-servers.
- Each individual organizational server is referred to as an O-server.
- each O- server selects one of the D-servers that are positioned as a leaf node in the hierarchy as its parent.
- the O-server aggregates its resource records into a summary and periodically forwards the summarized data to its selected parent D-server.
- each D-server Upon receipt of this summarized data from one or more O-servers, each D-server further aggregates the summarized data, for example using a coarser granularity, and forwards the additionally aggregated data further up though the hierarchy to its parents and grandparents. Aggregation and forwarding, with potentially increasing granularity, is repeated at each level of the hierarchy until the root server of the hierarchy receives and aggregates summarized data from all its descendents, i.e. all O-servers and D-servers.
- Aggregation from the leaf nodes to the root nodes provides each parent node within the hierarchy with an aggregated view of the subtree rooted at that parent, and replication overlay provides each node with a telescopic view of other portions of the hierarchy.
- the aggregate summarized data are used to identify children servers to the root server within the hierarchy that contain data or information about the location and other attributes of the desired resources. These children servers are identified and the query is forwarded to the children. Each child similarly forwards the query further down through the hierarchy until the query reaches the individual O-servers.
- the O-servers respond to the query by searching their databases and returning the search results to the user that proposed the query. Taking queries at the root server requires each query to be passed through the entire hierarchical structure, a time and resource consuming process. Query response times are shortened, resources preserved and bottlenecks avoided by replicating the summaries among various servers within the hierarchy.
- each D-server stores its own summarized data and the summarized data from its siblings, and its ancestors' siblings.
- a given D-server searches the replicated summaries and forwards the query directly to the appropriate D-servers without routing the query through the root D-server.
- the resource attribute data are sufficient to facilitate identification and allocation of the associated resources disposed within the plurality of distributed computing systems.
- the resource attribute data are obtained in a plurality of native attribute formats, and each native attribute format is determined by and associated with one of the distributed computing systems.
- the communicated resource attribute data are mapped from the plurality of native attribute formats to a common attribute format provided by at least one of the plurality of dedicated servers.
- a plurality of resource discovery servers are used to obtain the resource attribute data, and each resource discovery server is associated with one of the plurality of distributed computing systems.
- the common attribute format is communicated to each one of the resource discovery servers, and at least one of the dedicated servers or one of the resource discovery servers are used to map the communicated resource attribute data from the plurality of native formats to a common attribute format provided by the dedicated servers. This mapping is communicated to one of the dedicated servers in conjunction with the resource attribute data.
- a sufficient amount of resource attribute data is maintained at any given dedicated server within the hierarchy of dedicated servers to process resource queries and to route those queries through the hierarchy of dedicated servers to a distributed computing system comprising a suitable resource for the query.
- the resource attribute data are aggregated at each dedicated server to which the resource attribute data are communicated from the plurality of computing systems. These aggregated data are communicated through one or more intermediate dedicated servers to a root dedicated server of the hierarchy. In addition to aggregating data at the dedicated servers that initially receive the resource attribute data, the resource attribute data can be further aggregated at each one of the intermediate dedicated servers. [0013] In one embodiment, in order to maintain a sufficient amount of resource attribute data at each dedicated server, the resource attribute data and aggregated data maintained at each dedicated server are replicated within the hierarchy at one or more additional dedicated servers within the hierarchy.
- communicating the resource attribute data involves communicating the resource attribute data from one of the plurality of computing systems to one of a plurality of leaf node dedicated servers within the tree hierarchy.
- maintaining the resource attribute data includes periodically updating the resource attribute data maintained in the dedicated servers. Old or expired data are removed form the hierarchy by identifying a time to live value for each resource attribute datum communicated to the dedicated servers, and deleting resource attribute data from the dedicated servers for which updates are not received in compliance with the identified time to live values.
- FIG. 1 is a schematic representation of an embodiment of a dedicated server hierarchy and distributed computing system for use with the present invention
- Fig. 2 is a schematic representation of an embodiment of a dedicated server hierarchy for use with the present invention.
- FIG. 3 is a schematic representation illustrating an embodiment of query processing in accordance with the present invention.
- a system 10 for use in accordance with the present invention includes a plurality of dedicated servers 12 (D-servers) arranged in accordance with a pre-determined hierarchy.
- the dedicated servers are in communication with each other through this hierarchy and can exchange files or data.
- Suitable servers are known and available in the art and include any type of processor or processing unit capable of processing commands, storing data and communicating with other servers.
- each server is a software program that runs on any suitable machine or processor. Therefore, servers or server functionality can be provided by any machine or domain that is willing to host a server.
- Suitable sources for servers include, but are not limited to resource owners or other volunteering entities. These entities can be referred to as server providers.
- the entity that provided a given D-server chooses another server, for example either from the same or a different entity, as a parent.
- Each server usually has at least one parent.
- servers can organize into a hierarchy. Exactly which server to choose, may involve negotiation between two entities and is up to each entity to determine. Therefore, decisions regarding server arrangements within the hierarchy are made by the various domains or organizations that volunteer or provide these servers and are not made by a centralized entity or third party.
- a given server does not choose a parent from among its own descendants, thus avoiding loops.
- a descendant list is maintained at each server.
- parent servers are chosen in an order or arrangement that leads to a balanced hierarchy, improving efficiency. For example, a maximum depth for each server's sub-branches is maintained, and only those servers having greater maximum depths are selected as parents.
- the system also includes a plurality of resource discovery servers associated with specific domains or organizations and referred to as O-servers 22.
- Each O-server can be associated with a separate domain or two or more O-servers can be associated with a common domain. Suitable types of O-servers are the same as for the D-servers. In one embodiment, all of the O-servers are separate and distinct from the D-servers. Alternatively, at least one O-server and one D-server are a common server. In general, the O-servers are arranged as a widely distributed system of servers. Each O-server is in communication with at least one of the D- servers across one or more networks.
- Each O-server can be a single server or a distributed service within the given domain or organization.
- Each O-server is capable of discovering resources within its organization, of updating stored resource data, of summarizing resource data in accordance with a desired level of granularity, of communicating resource data to other servers and of receiving resource data from other servers.
- the resource data can be communicated in raw format or in aggregated summary format.
- the resources contained within a given organization are organized in accordance with the particular or proprietary format of the organization, and the data or attributes describing these resources in that format are obtained and maintained within the O- server.
- the particular or native format for the attributes used to describe the resources includes the conventions used to name a particular type of resource.
- Suitable attribute types include, but are not limited to, integer, double, string, timestamp, categorical, blob and combinations thereof. Variations in attribute formats arise from the use of different names to describe the exact same resource, e.g. memory vs. RAM, and the use of different resources having the same basic functionality. For example, one organization may use RAM as its primary storage source and another organization may use flash memory as its primary storage source. Both provide the same basic functionality to their respective organization but would have different attributes to describe the resources. Suitable resources include processing and communication resources that can be allocated to applications and to sensors or other data sources, e.g. video cameras and other surveillance equipment, that provide data that can be utilized by various organizations.
- each O-server Since each O-server is in communication with and capable of exchanging data with one or more of the distributed dedicated servers, D-servers, contained within the hierarchical structure, each O-server exports the resource attribute data stored on that O-server in accordance with the particular format or protocol of its organization to one of the D-servers.
- This D-server is the parent D-server to the O-server. Therefore, the various organizations do not have to be formatted in accordance with a universal format.
- FIG. 2 an exemplary embodiment of a tree hierarchy of D-servers 26 for use in resource discovery and allocation in accordance with the present invention is illustrated in Fig. 2.
- This tree hierarchy resource discovery system includes a plurality of the individual D- servers including the root D-server 14, the intermediate node D-servers 16 and the leaf node D- servers 20.
- Each leaf node D-server 20 contains the resource attribute data communicated to it from one or more O-servers.
- These resource attribute data include quantitative data 26 such as ranges of numbers and qualitative data 28 such as lists or sets of components.
- aggregation functions In order to improve the storage of resource attribute data at any given D-server, summaries or aggregations of the resource attribute data associated with the resources and communicated to that D-server are created using aggregation functions.
- aggregation functions generate summarized data that are of coarser granularity than the original data. This is usually achieved by reducing the amount of data or information associated with a given attribute.
- aggregation includes decreasing the amount of data associated with a given attribute. Therefore, for a given attribute, an aggregation function is applied over a set of resources having data associated with that attribute, and aggregation involves aggregation with respect to a given attribute.
- Suitable aggregation functions include any type of aggregation that reduces data volume while still supporting resource queries. These aggregation functions include, but are not limited to, numeric, either integer or real, single bucket or multi-bucket (histogram) and categorical, i.e. set union.
- a single aggregation function can be applied to all attributes, or a separate or distinct aggregation function can be used for each attribute. For example, for an attribute, e.g. available bandwidth of network adaptors, that is described using a numerical value, the aggregation result is produced by computing the upper and lower bounds of the attribute for the given set of resources. This aggregation result is communicated to a given D-server.
- Each aggregated result for that attribute has upper and lower bounds, and aggregated results can be further aggregated.
- all of the acceptable ranges of numerical values for all of the aggregated results will fall within a range defined by the smallest lower bound associated with all of the aggregated results and the largest upper bound associated with all of the aggregated results. Therefore, the aggregated attribute contains the smallest overall lower bound and the largest overall upper bound.
- a separate attribute for each resource is not maintained at the D-server. Only the aggregate attribute containing the overall upper and lower bounds. This aggregate attribute is communicated upwards through the hierarchy of D-servers and ultimately the root D-server.
- the D-server in the hierarchy that receives the aggregated attribute obtains a grainier or broader view of the resources below it in terms of that attribute. That D-server may not know the exact value associated with an attribute of any resource, but the D-server will know that any resource associated with its children nodes has for that attribute values that fall within the upper and lower bounds of its aggregated attribute.
- a given D-server for example one of the intermediate D-servers, receives the aggregated attribute data from its children, that D-server further aggregates the data along each dimension. Again, for a quantitative or numerical attribute, the D-server takes the smallest of all lower bounds and greatest of all upper bounds, thus obtaining the new lower and upper bounds. This further aggregated attribute is communicated to the parent of that D-server.
- a set or list is used to represent the members or categories included in the attribute.
- the attribute can be an identification of the types of communication protocols that a given resource can support, and the attribute is represented by a set containing an identification of the supported communication protocols.
- Aggregation includes taking a combination or union of all the attribute sets associated with the resources and received from an O-server or a child D-server. Attribute sets representing the union of all sets are communicated up through the hierarchy, being further aggregated or combined at each level. As shown in Fig.
- an aggregation function is suitable for use with location attributes, e.g.
- the summarized data of a given D-server' s siblings, that D- server's ancestors' siblings, and the D-server' s ancestors if they themselves have data are periodically sent top-down in the hierarchy from that D-server to a recipient D-server and cached by the receiving D-server. For example as illustrated in the expanded list 30 in Fig.
- a given leaf node D-server Di contains its own aggregated resource attribute data and the replicated aggregated resource data of leaf node D-server D 2 , which is its sibling, intermediate D-server C 1 , it parent, intermediate D-server C 2 , its parent's sibling, intermediate D-server B 1 , its grandparent, intermediate D-servers B 2 and B3, its grandparent's siblings, and root D-server A. Therefore, sufficient information about all of the resource allocation data submitted to the hierarchy of D-servers is maintained at a single leaf node within the hierarchy such that resource allocation queries can be received and processed at that leaf node without having to communicate that query to the root D-server for processing. That is, the leaf nodes have a sufficiently detailed picture of all available or identified resources.
- each server forwards its summarized and aggregated data up through the hierarchy and propagates the aggregated data of servers located upstream in the hierarchy down through the hierarchy towards the leaf nodes.
- D-server node Bi receives the aggregated attribute data of D-servers B 2 and B3 from root D-server A and forwards this aggregated attribute data to both D-servers Ci and C 2 .
- D-servers Ci and C 2 upon receipt of the aggregated attribute data continue its propagation down the hierarchy towards the leaf level servers.
- each D-server contains the summary or aggregate copy of at least a portion and potentially the entire set of global resources, speeding up resource discovery and query times.
- the amount of detail that a given D-server has about any particular resource is proportional to the distance, e.g. the number of intermediate node D-servers and O-servers, between that D-server and the resource. For shorter distances between the D-server and the resource, a higher degree of specificity or detail is maintained in that D-server about the resource. As the distance grows, the D-server will still maintain information about the resource, but at much coarser granularity.
- D-server node Di knows the locations of camera records stored at D-server node D 2 at a city block level of specificity; however, D-server node Di only knows the locations of camera records for D-server nodes B 2 and B 3 at a state level of specificity. Reducing the level of specificity and increasing the granularity with distance ensures that the total storage and message overhead does not become exceedingly large.
- the storage complexity is about k log n at each server.
- Replication also allows queries to be sent to any D-server within the hierarchy and not just the root D-server.
- FIG. 3 an exemplary embodiment illustrating the processing of resource queries using a given hierarchy of D-servers is shown.
- a client 34 submits a query 36 to one of the D-servers al in the hierarchy.
- This D-server al searches the summarized or aggregate data stored at that D-server, including is own summarized data and any replicated data that has been communicated to it. Based upon this search, the D-server al determines that its sibling D-server cl and its uncle D-server b2 contain information about the resources required by the query.
- D-server al forwards the query 38, 40 to it sibling and uncle D-servers cl and b2.
- each D-server within the hierarchy maintains periodic "heart-beat" messages along each communication link with its children servers and parent server. These heart-beat messages test the availability of the communication link and the server. If a given heart-beat message indicates that a server or communication link, for example associated with the parent D-server, is not accessible, the D-server chooses an uncle D-server as its new parent.
- This choice can be made randomly or in accordance with one or more pre-determined selection criteria. If the parent D-server is the root D-server of the hierarchy system, one of the children is elected as the new root. Suitable methods for the selection of a new root D-server include, but are not limited to, selecting the child D-server having the smallest IP address, considering the current load on each child D-server and combinations thereof. In general, eliminating the need to submit or forward every query to a single root D-server alleviates the associated bottleneck and distributes the computation load throughout the hierarchy, reducing query response times.
- the summarized data are soft-state.
- each D-server maintains a time-to-live (TTL) value for each dimension of the summarized data.
- TTL time-to-live
- the data are refreshed periodically as new data are received from children or parents. As long as refreshed data are received in compliance with the prescribed TTL values, the summarized data is maintained as current and reliable.
- the summary data associated with these are removed from a given D-server when those data expire in accordance with the TTL values. Therefore, outdated data, changes to the hierarchy topology and changes in the available resources do not interfere with or corrupt resource discovery in response to the queries.
- routines that are integrated into the D-server or O-server or operate separate from either server and that run automatically in the background out of the direct control of a system administrator, e.g., a wrapper daemon, are used to forward data updates from the O-servers to the D-servers.
- the routines periodically query a given O-server, extract the desired summarized data and forward the extracted data to the parent D-server.
- the routine can also reformat the results returned by a search of the O- server into a common format understood by the client.
- the hierarchy of D-servers is not limited to using any particular attribute format to monitor and administer the allocation of resources that are contained in a variety of different organizations or domains.
- attribute format for resource identification that is imposed on the distributed computing systems.
- Each computing systems determines and utilizes its own attribute format. Therefore, the schemas, i.e. the list of attributes, used for each type of resource, do not need to be exactly the same within different organizations.
- the D-servers at the leaf level can map the attributes about a given type of resource received from a variety of O-servers to one or more common attributes used among the D-servers in the hierarchy, even if the various received attributes are different.
- one or more of the D-servers communicate to the O-servers a pre-determined common attribute format that is to be used for the resource attribute data in the hierarchy.
- the communicated common attributes contain a sufficient amount of information to allow each O-server to map the resource data attributes for its resources to the common attributes. This map is communication to the D-servers along with the resource attribute data. Therefore, each autonomous organization has an increased amount of freedom in choosing its native resource discovery system, and the federated hierarchy system can incorporate a wider variety of legacy autonomous systems. No external attribute format or hierarchy is imposed on the individual computing systems.
- resource attribute data associated with a plurality of resources disposed in a plurality of distributed computing systems is communicated from those distributed computing systems to one or more of a plurality of dedicated servers arranged in accordance with a pre-defined hierarchy.
- the resource attribute data contain a sufficient amount of data about the resources contained in the distributed computing systems to facilitate identification and allocation of the associated resources disposed within the plurality of distributed computing systems.
- the resources are identified and allocated in response to resource demand queries from clients, for example applications running on one of the servers in the computing systems or within the hierarchy.
- Suitable computing systems including any arrangement or networked arrangement of computing resources including, but not limited to, an information technology (IT) computing system.
- the resource data in order to communicate the resource attribute data to the D- servers, the resource data is actively obtained from each computing system using the D-servers.
- the O-servers disposed within the plurality of computing systems obtain the resource attribute data for the resources contained within the computing system in which the O- server is located.
- each computing system contains two or more O-servers that are used to obtained resource attribute data.
- each computing contains one O- server for resource attribute data acquisition and communication.
- the resource attribute data can be communicated to any D-server within the hierarchy.
- the resource attribute data are communicated to one of the leaf node D-servers contained within the hierarchy.
- a suitable leaf node D-server is identified for each O-server, and the O-servers communicate the resource attribute data to the associated leaf node D-servers.
- Each O-server or resource discovery server obtains the resource attribute data for the resources contained within its computing system in the native attribute format for that computing system.
- a plurality of native attribute formats is possible, and each format can be unique. These native formats vary the way resources are accounted for and described including, but not limited to, the name assigned to a particular resource and the types of resources utilized for a given function within the computing system.
- Each native attribute format for each computing system is determined by and associated with one of the distributed computing systems. A generic or universal attribute format is not imposed upon the various computing systems.
- the communicated resource attribute data is mapped, at the D-server level, from the plurality of native attribute formats to a common attribute format.
- This common attribute format is provided by at least one of the plurality of dedicated servers. Mapping between the common format and the native formats can be conducted at the D-server level or at the O-server level.
- the common attribute format is communicated to each one of the O-servers, and the O-servers in communicating the resource attribute data to the D-servers map the communicated resource attribute data from the plurality of native formats to a common attribute format provided by the D-servers. Therefore, both the mapping information in conjunction with the resource attribute data is communicated to the D-servers.
- resource queries can be submitted to and processed by any D-server, and these D-servers can route those queries through the hierarchy of dedicated servers to the appropriate distributed computing system or to the appropriate O-server within the computing system that contains a suitable resource for the query.
- Maintenance of the desired amount of resource attribute data at each D-server is accomplished using aggregation and replication.
- the resource attribute data are aggregated at each D-server to which the resource attribute data are communicated from the plurality of computing systems.
- These aggregated resource attribute data are communicated through one or more intermediate D- servers to a root D-server of the hierarchy.
- the resource attribute data are further aggregated at each one of the intermediate dedicated servers as the data are communicated upward through the hierarchy.
- the resource attribute data maintained at each dedicated server within the hierarchy are replicated at one or more additional dedicated servers within the hierarchy, for example siblings and ancestors within the hierarchy. Since the status and availability of any given resource, i.e. the resource attribute data associated with that resource changes over time, the resource attribute data maintained in the D-servers are periodically updated. For example, a time to live value is identified for each resource attribute datum communicated to the dedicated servers. This is the prescribed conditions or period of time in which an update has to be received or that datum expires. Resource attribute data are deleted from the dedicated servers when updates are not received in compliance with the identified time to live values.
- a request i.e. a resource request
- Suitable queries include structured query language (SQL) select queries. Since each node contains aggregated and replicated data, the query can be forwarded to other node D- servers within the hierarchy and can even be forwarded across relatively long jumps between servers and not just to adjacent, i.e. parent-child, D-servers.
- the recipient D-server in response to the query, can return matching records to the query source, i.e. a user, or can forward the query to other D-servers within the hierarchy.
- the query is forwarded to one or multiple servers that can return resource records in response to the query.
- one or more D-servers map queries from the native format of the query to the common query format used within the hierarchy. Preferably this mapping or translation is handled at the D- server level.
- the query formats are communicated to the D-servers for example from O-servers or other query sources. The D-server creates the necessary mapping relationships so that in response to the receipt of a given query in native format, that query can be mapped to the common query format utilized by the hierarchy for processing by the hierarchy of D-servers.
- a resolver module can act on behalf of the client to send queries and receive results.
- the resolver can relieve the client from the burden of interacting with the D- server hierarchy and the O-servers. Since the client might be an external application that does not know the native formats of results returned from the O-servers, which may vary from one organization to another, the resolver can do appropriate formatting and return to the client results in a universal format.
- client queries are handled in as an iterative process, i.e. each server redirects the client to query other servers.
- client queries are not handled in a recursive manner where servers forward the query and relay the results back to the client.
- Using the iterative process allows the client to stop the search when needed. For example, the client may not need an exhaustive search in response to its query and can stop the query process when the record of the first qualifying resource is received or when a sufficient number or records are received.
- the client may not need an exhaustive search in response to its query and can stop the query process when the record of the first qualifying resource is received or when a sufficient number or records are received.
- Using the iterative mode shifts the complexity and responsibility to the client, and the client is at a better position to make stop decisions.
- Systems and methods in accordance with the present invention can be used in a variety of applications with a variety of arrangements of distributed organizations. Suitable applications include, but are not limited to, internet-based stream processing systems and resource rich sensor and actuator networks. In these systems, the present invention unifies previously isolated and potentially application specific networks and resources, while supporting applications that run across multiple networks.
- exemplary methods and systems in accordance with the present invention can be used in conjunction with enterprise directory services to provide unified access to directory services developed and deployed separately and to grid computing systems by unifying multiple grids that each potentially employs a different directory service.
- Methods and systems in accordance with exemplary embodiments of the present invention can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment containing both hardware and software elements.
- exemplary methods and systems can take the form of a computer program product accessible from a computer-usable or computer- readable medium providing program code for use by or in connection with a computer, logical processing unit or any instruction execution system.
- a computer-usable or computer-readable medium can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.
- Suitable computer-usable or computer readable mediums include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems (or apparatuses or devices) or propagation mediums.
- Examples of a computer-readable medium include a semiconductor or solid state memory, magnetic tape, a removable computer diskette, a random access memory (RAM), a read-only memory (ROM), a rigid magnetic disk and an optical disk.
- Current examples of optical disks include compact disk - read only memory (CD-ROM), compact disk - read/write (CD-R/W) and DVD.
- Suitable data processing systems for storing and/or executing program code include, but are not limited to, at least one processor coupled directly or indirectly to memory elements through a system bus.
- the memory elements include local memory employed during actual execution of the program code, bulk storage, and cache memories, which provide temporary storage of at least some program code in order to reduce the number of times code must be retrieved from bulk storage during execution.
- I/O devices including but not limited to keyboards, displays and pointing devices, can be coupled to the system either directly or through intervening I/O controllers.
- I/O controllers can be coupled to the system either directly or through intervening I/O controllers.
- Exemplary embodiments of the methods and systems in accordance with the present invention also include network adapters coupled to the system to enable the data processing system to become coupled to other data processing systems or remote printers or storage devices through intervening private or public networks. Suitable currently available types of network adapters include, but are not limited to, modems, cable modems, DSL modems, Ethernet cards and combinations thereof.
- the present invention is directed to a machine-readable or computer-readable medium containing a machine-executable or computer-executable code that when read by a machine or computer causes the machine or computer to perform a method for federated resource discovery in distributed information technology systems in accordance with exemplary embodiments of the present invention and to the computer-executable code itself.
- the machine -readable or computer-readable code can be any type of code or language capable of being read and executed by the machine or computer and can be expressed in any suitable language or syntax known and available in the art including machine languages, assembler languages, higher level languages, object oriented languages and scripting languages.
- the computer-executable code can be stored on any suitable storage medium or database, including databases disposed within, in communication with and accessible by computer networks utilized by systems in accordance with the present invention and can be executed on any suitable hardware platform as are known and available in the art including the control systems used to control the presentations of the present invention.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computational Linguistics (AREA)
- Probability & Statistics with Applications (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Fuzzy Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Computer And Data Communications (AREA)
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US11/471,981 US7849069B2 (en) | 2006-06-21 | 2006-06-21 | Method and system for federated resource discovery service in distributed systems |
PCT/US2007/070544 WO2007149701A2 (en) | 2006-06-21 | 2007-06-06 | A method and system for federated resource discovery service in distributed systems |
Publications (2)
Publication Number | Publication Date |
---|---|
EP2035916A2 true EP2035916A2 (de) | 2009-03-18 |
EP2035916A4 EP2035916A4 (de) | 2012-06-20 |
Family
ID=38834228
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP07798187A Withdrawn EP2035916A4 (de) | 2006-06-21 | 2007-06-06 | Verfahren und system für föderierten ressourcenentdeckungsdienst in verteilten systemen |
Country Status (5)
Country | Link |
---|---|
US (1) | US7849069B2 (de) |
EP (1) | EP2035916A4 (de) |
JP (1) | JP5043937B2 (de) |
CN (1) | CN101454747B (de) |
WO (1) | WO2007149701A2 (de) |
Families Citing this family (44)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8185423B2 (en) * | 2005-12-22 | 2012-05-22 | Canon Kabushiki Kaisha | Just-in time workflow |
US20080052397A1 (en) * | 2006-08-24 | 2008-02-28 | Ramanathan Venkataraman | Future locking of resources |
US7720841B2 (en) * | 2006-10-04 | 2010-05-18 | International Business Machines Corporation | Model-based self-optimizing distributed information management |
US9754273B2 (en) * | 2006-12-19 | 2017-09-05 | Microsoft Technology Licensing, Llc | Enterprise resource tracking of knowledge |
US8555397B2 (en) * | 2006-12-29 | 2013-10-08 | Sap Ag | Consumer-controlled data access to shared RFID data |
US20080157930A1 (en) * | 2006-12-29 | 2008-07-03 | Steve Winkler | Object name service for RFID tags |
US8639825B2 (en) * | 2006-12-29 | 2014-01-28 | Sap Ag | Enterprise-based access to shared RFID data |
US8555398B2 (en) * | 2006-12-29 | 2013-10-08 | Sap Ag | Role-based access to shared RFID data |
US8041808B1 (en) | 2007-03-30 | 2011-10-18 | United Services Automobile Association | Managing the performance of an electronic device |
US8359347B2 (en) * | 2007-04-10 | 2013-01-22 | International Business Machines Corporation | Method and apparatus for cooperative data stream processing |
US8892624B2 (en) * | 2007-04-10 | 2014-11-18 | International Business Machines Corporation | Method for the interoperation of virtual organizations |
US8417762B2 (en) * | 2007-04-10 | 2013-04-09 | International Business Machines Corporation | Mechanism for execution of multi-site jobs in a data stream processing system |
US8688850B2 (en) * | 2007-04-10 | 2014-04-01 | International Business Machines Corporation | Method for inter-site data stream transfer in cooperative data stream processing |
US20080256549A1 (en) * | 2007-04-10 | 2008-10-16 | International Business Machines Corporation | System and Method of Planning for Cooperative Information Processing |
US20080319957A1 (en) * | 2007-06-19 | 2008-12-25 | Microsoft Corporation | Extensible command trees for entity data model platform |
US8589490B2 (en) * | 2008-01-16 | 2013-11-19 | Janos Tapolcai | System, method, and computer program for solving mixed integer programs with peer-to-peer applications |
JP5243804B2 (ja) * | 2008-01-21 | 2013-07-24 | インターナショナル・ビジネス・マシーンズ・コーポレーション | 構成要素を管理するためのコンピュータ・システム、並びにその方法及びコンピュータ・プログラム |
US8805875B1 (en) * | 2008-10-04 | 2014-08-12 | Reflex Systems Llc | Systems and methods for information retrieval |
US8713426B2 (en) * | 2009-06-25 | 2014-04-29 | Oracle International Corporation | Technique for skipping irrelevant portions of documents during streaming XPath evaluation |
US9081873B1 (en) | 2009-10-05 | 2015-07-14 | Stratacloud, Inc. | Method and system for information retrieval in response to a query |
US9165086B2 (en) | 2010-01-20 | 2015-10-20 | Oracle International Corporation | Hybrid binary XML storage model for efficient XML processing |
EP2668600A4 (de) * | 2011-01-27 | 2014-08-06 | Computenext Inc | Dynamische organisation von cloud-computing-ressourcen für erleichterte erkennung |
US20160105399A1 (en) * | 2012-04-05 | 2016-04-14 | Peter Rung | Systems and Methods for Cloaking Communications |
US8965849B1 (en) * | 2012-08-06 | 2015-02-24 | Amazon Technologies, Inc. | Static sorted index replication |
US9703807B2 (en) | 2012-12-10 | 2017-07-11 | Pixia Corp. | Method and system for wide area motion imagery discovery using KML |
CN103324713B (zh) * | 2013-06-19 | 2017-04-12 | 北京奇安信科技有限公司 | 多级服务器中的数据处理方法、装置和数据处理系统 |
CN104636397B (zh) * | 2013-11-15 | 2018-04-20 | 阿里巴巴集团控股有限公司 | 用于分布式计算的资源分配方法、计算加速方法以及装置 |
US20150294248A1 (en) * | 2014-04-15 | 2015-10-15 | Kevin Andrew DEIBLER | Position-centric personnel assessment apparatus and method |
US9531749B2 (en) | 2014-08-07 | 2016-12-27 | International Business Machines Corporation | Prevention of query overloading in a server application |
US11755559B1 (en) | 2014-10-09 | 2023-09-12 | Splunk Inc. | Automatic entity control in a machine data driven service monitoring system |
US11200130B2 (en) * | 2015-09-18 | 2021-12-14 | Splunk Inc. | Automatic entity control in a machine data driven service monitoring system |
US10115071B1 (en) | 2015-01-08 | 2018-10-30 | Manhattan Associates, Inc. | Distributed workload management |
US10558702B2 (en) * | 2016-04-06 | 2020-02-11 | Baidu Usa Llc | Unified storage system for online image searching and offline image analytics |
CN109644147A (zh) * | 2016-06-10 | 2019-04-16 | 瑞典爱立信有限公司 | 用于在实时数据收集中使用的分级数据收集器和相关技术 |
US11093518B1 (en) | 2017-09-23 | 2021-08-17 | Splunk Inc. | Information technology networked entity monitoring with dynamic metric and threshold selection |
US11106442B1 (en) | 2017-09-23 | 2021-08-31 | Splunk Inc. | Information technology networked entity monitoring with metric selection prior to deployment |
US11159397B2 (en) | 2017-09-25 | 2021-10-26 | Splunk Inc. | Lower-tier application deployment for higher-tier system data monitoring |
US10416661B2 (en) | 2017-11-30 | 2019-09-17 | Abb Schweiz Ag | Apparatuses, systems and methods of secure cloud-based monitoring of industrial plants |
US10955539B2 (en) * | 2018-10-12 | 2021-03-23 | Goodrich Corporation | Layered tactical information system and method |
JP6573044B1 (ja) * | 2019-03-22 | 2019-09-11 | 富士ゼロックス株式会社 | データ管理システム |
US20210012025A1 (en) * | 2019-07-10 | 2021-01-14 | At&T Intellectual Property I, L.P. | System and method for session-aware datastore for the edge |
CN112256199B (zh) * | 2020-10-21 | 2023-07-07 | 张立旭 | 一种提高存储资源利用率的分布式数据存储方法及系统 |
US11676072B1 (en) | 2021-01-29 | 2023-06-13 | Splunk Inc. | Interface for incorporating user feedback into training of clustering model |
CN113312359B (zh) * | 2021-06-28 | 2023-04-07 | 北京筑云慧建软件技术有限公司 | 一种分布式作业进度计算方法、装置和存储介质 |
Family Cites Families (20)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5689700A (en) * | 1993-12-29 | 1997-11-18 | Microsoft Corporation | Unification of directory service with file system services |
US5893107A (en) * | 1996-07-01 | 1999-04-06 | Microsoft Corporation | Method and system for uniformly accessing multiple directory services |
US6647393B1 (en) * | 1996-11-22 | 2003-11-11 | Mangosoft Corporation | Dynamic directory service |
US6151624A (en) * | 1998-02-03 | 2000-11-21 | Realnames Corporation | Navigating network resources based on metadata |
US6185567B1 (en) * | 1998-05-29 | 2001-02-06 | The Trustees Of The University Of Pennsylvania | Authenticated access to internet based research and data services |
US6549932B1 (en) * | 1998-06-03 | 2003-04-15 | International Business Machines Corporation | System, method and computer program product for discovery in a distributed computing environment |
US6256634B1 (en) * | 1998-06-30 | 2001-07-03 | Microsoft Corporation | Method and system for purging tombstones for deleted data items in a replicated database |
US6199077B1 (en) * | 1998-12-08 | 2001-03-06 | Yodlee.Com, Inc. | Server-side web summary generation and presentation |
US6517587B2 (en) * | 1998-12-08 | 2003-02-11 | Yodlee.Com, Inc. | Networked architecture for enabling automated gathering of information from Web servers |
US6587874B1 (en) * | 1999-06-29 | 2003-07-01 | Cisco Technology, Inc. | Directory assisted autoinstall of network devices |
US6792416B2 (en) * | 1999-09-21 | 2004-09-14 | International Business Machines Corporation | Managing results of federated searches across heterogeneous datastores with a federated result set cursor object |
EP1328890A2 (de) * | 2000-10-20 | 2003-07-23 | Siemens Aktiengesellschaft | System und verfahren zum verwalten von softwareapplikationen, insbesondere mes-applikationen |
US6976060B2 (en) * | 2000-12-05 | 2005-12-13 | Agami Sytems, Inc. | Symmetric shared file storage system |
US7523454B2 (en) * | 2001-03-06 | 2009-04-21 | Hewlett-Packard Development Company, L.P. | Apparatus and method for routing a transaction to a partitioned server |
US8666983B2 (en) * | 2003-06-13 | 2014-03-04 | Microsoft Corporation | Architecture for generating responses to search engine queries |
FI20040196A0 (fi) * | 2004-02-09 | 2004-02-09 | Nokia Corp | Signaalin havaitseminen käyttäen sphere decoding - tekniikkaa |
US7487142B2 (en) * | 2004-06-10 | 2009-02-03 | International Business Machines Corporation | Methods and apparatus for specifying and processing descriptive queries for data sources |
US7441243B2 (en) * | 2004-06-10 | 2008-10-21 | International Business Machines Corporation | Exception handling in the processing of proposal requests in a grid computing environment |
JP4494901B2 (ja) * | 2004-08-06 | 2010-06-30 | 日本電信電話株式会社 | リソース検索方法およびリソース検索システム |
US7613703B2 (en) * | 2004-09-30 | 2009-11-03 | Microsoft Corporation | Organizing resources into collections to facilitate more efficient and reliable resource access |
-
2006
- 2006-06-21 US US11/471,981 patent/US7849069B2/en not_active Expired - Fee Related
-
2007
- 2007-06-06 JP JP2009516627A patent/JP5043937B2/ja not_active Expired - Fee Related
- 2007-06-06 WO PCT/US2007/070544 patent/WO2007149701A2/en active Application Filing
- 2007-06-06 EP EP07798187A patent/EP2035916A4/de not_active Withdrawn
- 2007-06-06 CN CN2007800190752A patent/CN101454747B/zh not_active Expired - Fee Related
Non-Patent Citations (2)
Title |
---|
No further relevant documents disclosed * |
See also references of WO2007149701A2 * |
Also Published As
Publication number | Publication date |
---|---|
JP5043937B2 (ja) | 2012-10-10 |
CN101454747B (zh) | 2011-04-13 |
CN101454747A (zh) | 2009-06-10 |
WO2007149701A3 (en) | 2008-11-06 |
WO2007149701A2 (en) | 2007-12-27 |
US20070299804A1 (en) | 2007-12-27 |
JP2009541856A (ja) | 2009-11-26 |
EP2035916A4 (de) | 2012-06-20 |
US7849069B2 (en) | 2010-12-07 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US7849069B2 (en) | Method and system for federated resource discovery service in distributed systems | |
US8037023B2 (en) | System and apparatus for optimally trading off the replication overhead and consistency level in distributed applications | |
US11461356B2 (en) | Large scale unstructured database systems | |
US8166074B2 (en) | Index data structure for a peer-to-peer network | |
JP4406609B2 (ja) | 単一のインターフェイスからのデータの多重階層を管理するための手法 | |
RU2409846C2 (ru) | Организация ресурсов в коллекции, способствующая более эффективному и надежному доступу к ресурсам | |
US7546284B1 (en) | Virtual message persistence service | |
CN102239476B (zh) | 用于存储集群的共享名称空间 | |
US8108338B2 (en) | Method and system for model-based replication of data | |
US9244958B1 (en) | Detecting and reconciling system resource metadata anomolies in a distributed storage system | |
US6363375B1 (en) | Classification tree based information retrieval scheme | |
JPS63201743A (ja) | データ処理システム・ネットワークのためのキャッシング方法 | |
US8296420B2 (en) | Method and apparatus for constructing a DHT-based global namespace | |
US10712964B2 (en) | Pre-forking replicas for efficient scaling of a distributed data storage system | |
JP2012531688A (ja) | メタデータに従ってファイルシステムのファイルにアクセスする方法、およびその方法を実装する装置 | |
Schwartz et al. | Experience with a semantically cognizant internet white pages directory tool | |
CN116756096B (zh) | 元数据处理方法、装置、电子设备及存储介质 | |
JP2007156700A (ja) | 情報検索方法、情報登録方法およびネットワークサービス情報検索システム | |
Aktas | Information federation in grid information services | |
Malensek | On the evaluation of exact-match and range queries over multidimensional data in distributed hash tables | |
Gopalakrishnan | System support for keyword-based search in structured Peer-to-Peer systems | |
Vernica | Publish/Subscribe in Peer-to-Peer Networks | |
Jagannathan et al. | Plethora: An Efficient Wide-Area Storage System |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
17P | Request for examination filed |
Effective date: 20090114 |
|
AK | Designated contracting states |
Kind code of ref document: A2 Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU LV MC MT NL PL PT RO SE SI SK TR |
|
AX | Request for extension of the european patent |
Extension state: AL BA HR MK RS |
|
DAX | Request for extension of the european patent (deleted) | ||
A4 | Supplementary search report drawn up and despatched |
Effective date: 20120518 |
|
RIC1 | Information provided on ipc code assigned before grant |
Ipc: G06F 17/30 20060101AFI20120511BHEP |
|
17Q | First examination report despatched |
Effective date: 20140521 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE APPLICATION IS DEEMED TO BE WITHDRAWN |
|
18D | Application deemed to be withdrawn |
Effective date: 20141001 |