CN101741614B - Equivalent type node manager and equivalent type node management method - Google Patents
Equivalent type node manager and equivalent type node management method Download PDFInfo
- Publication number
- CN101741614B CN101741614B CN2009102385031A CN200910238503A CN101741614B CN 101741614 B CN101741614 B CN 101741614B CN 2009102385031 A CN2009102385031 A CN 2009102385031A CN 200910238503 A CN200910238503 A CN 200910238503A CN 101741614 B CN101741614 B CN 101741614B
- Authority
- CN
- China
- Prior art keywords
- resource
- node
- service
- data
- manager
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Abstract
The invention discloses an equivalent type node manager used for realizing organization and management of grid resource in grid computing environment. The equivalent type node manager comprises a virtual node resource aggregator, a temporary data resource aggregating container, a grid node element service library and a virtual node portal configurator; the virtual node resource aggregator is used for describing, organizing and managing the permanent data resource in the grid computer environment, and leading the overall resources to be updated synchronously at each node via a synchronization mechanism; the temporary data resource aggregating container is used for managing new data resource (intermediate result data) produced by the service tier, wherein the new data resource is converted via the synchronization mechanism into the permanent data resource managed by the virtual node resource aggregator; the grid node element service library is used for structuring the service tier, managing, serving and allocating the on-line hardware resource; and the virtual node portal configurator is used for providing a service entry for a user to connect the user and the service tier. The invention deploys the equivalent type node manager in the equivalent calculation manner, thus realizing synchronous update of the overall resource and providing a mechanism for ceterless architecture; and in the grid computing environment, the whole system can not break down even though plural nodes break down.
Description
Technical field
The present invention relates to the computer network resources administrative skill, relate in particular to the administrative skill of gridding resource.
Background technology
Resource is the main research object in the grid, and the purpose of grid is exactly the resource in the managing gridding, and offers the user.What resource management was paid close attention to is not the Core Feature (being what resource can do for the user) of resource and service, but how control mesh resource and service provide the sequence of operations of available capability to user, application or service at other interior entities, promptly how resource are described and organize, how to find how the function of resource and resource is carried out.
Grid is distributed in a plurality of territories in wide scope, in large scale, and the resource kind is many and isomerism is stronger, has very many users, and has shared and privately owned resource in the grid simultaneously.These characteristics of grid have caused grid resource more more complicated than the resource management of previous systems such as cluster.
Resource organizations's mode is the version of various gridding resources in the grid computing environment, to grid resource efficient important influence.The organizational form of resource mainly contains two kinds in the current grid computing environment: the resource pool model (centralized) and the overall situation-local two-layer resource model.
Resource pool is a kind of simple resource model, and this organizational form is to write down the information of all resources in the computing environment on a central server, and these information are carried out centralized management.When being the computational problem Resources allocation, from the Resources list (resource pool) of server, select idling-resource to calculate simply.Description to resource in the resource pool is unordered, resource is not carried out structured organization effectively, thereby can compare blindly for the computational problem distributes calculation resources time.Representational software has Parabon, Entropia, Condor-G, Charlott etc.This centralized resource management pattern can not solve the autonomy of each node, says nothing of the collaborative of resource between each node and has shared.
Adopting the overall situation-local two-layer resource model to come the exemplary software of organizational resources is Globus.Globus Toolkit conventional tool bag by the subordinate Globus project team member of Global Grid forum (GGF) joint development, be acknowledged as the current normative reference of setting up the grid system and the exploitation grid software fact, this software mainly is made up of 4 parts: Resource Broker, collaborative distributor, resource information serviced component and resource allocation manager.The employed local resource management tool of Globus has nothing in common with each other, and it is all inequality that the administrative mechanism of each local resource management tool reaches the interface that the outside is provided.Local resource management tool commonly used at present has Condor, NQE, CODINE, EASY, LSF, PBS, LoadLeveler.In order to allow a calculation task move by while distributed earth in whole grid environment, Globus is necessary for each local resource management tool provides a kind of GRAM to realize, the extensibility that this has just limited the Globus resource model will cause using the computational resource of other local resource management tools to provide service as calculation task as the part of grid environment.
Want the managing gridding resource, at first need to describe resource is unified.Usually adopt UDDI (UniversalDescription Discovery and Integration, unified description, discovery and integrated agreement) or similar registration tool at present.Its problem that mainly exists is: needs carry out data acquisition at the registering unit of resource organizations specially, cause existing data, can not directly use as metadatabase, not only can cause the inconsistency of data description, also can waste the plenty of time simultaneously and gather again; Major part adopts centralized management, does not meet the characteristics of grid; Shortage is to hardware resource, software resource, the incorporate description of data resource and tissue.
Secondly, resource management should be distributed (the node structure is acentric), consider autonomy, coordination and the sharing of node, hardware resource, software resource and data resource should have local and notion of overall importance, local resource is by site autonomy, and global resource is by resource-sharing in the grid and collaborative.The problem of the grid resource instrument that adopts usually is at present: great majority adopt centralized management, are a kind of management that center type is arranged, and can't embody autonomy; Do not have the description of local resource, can't embody autonomy; Do not work in coordination with and synchronization mechanism, global resource information can not be duplicated by multiple spot, if node failure occurs, the part resource can't be found and use.
At last, in grid computing or service process, support long affair mechanism.The user except mutual and feedback procedure are arranged, also has the reusability problem of results of intermediate calculations in one of request is calculated or served.Promptly going up the output of a link, is again the input of next ring, and can provide a plurality of users to reuse.Prior art problems is at present: generally do not support long affair mechanism, work is carried out in a request (calculating container) under an interim environment, and after calculating was finished, environment was cancelled, and can't keep breakpoint and recover the calculating of breakpoint environment again; Concerning ordinary affair affair computing mechanism, the reusability of resource is more outstanding, if all intermediate object program is placed in the permanent resources Register, can cause the generation of source manager a large amount of " rubbish " resource; Do not consider the relation of permanent resources and temporary resource.
Summary of the invention
The technical problem to be solved in the present invention provides a kind of connection point manager (CP manager) method and connection point manager (CP manager) device, can carry out incorporate description and tissue to the various resource organizations that disperse in the grid to hardware resource, software resource, data resource; Adopt distributed (the node structure is acentric) resource organizations's mode, consider autonomy, coordination and the sharing of node, hardware resource, software resource and data resource are categorized as local and resource of overall importance, local resource is by site autonomy, and global resource is by resource-sharing in the grid and collaborative, and sharing with collaborative needs the synchronization mechanism support; In grid computing or service process, support long affair mechanism.
In order to address the above problem, the invention provides a kind of to equation connection point manager (CP manager) device, be used for realizing the Organization And Management of grid computing environment gridding resource, comprise virtual node resource clustering device, mesh node unit service library, virtual node portal configurator and temporary data resource aggregation container, wherein:
Described virtual node resource clustering device is used for that permanent data resource to gridding computer environment is described, Organization And Management, and by synchronization mechanism global resource is upgraded synchronously at each node;
Described temporary data resource aggregation container, be used to manage the new data resource that produces by service layer, this new data is middle achievement data, and to make this new data resource conversion by synchronization mechanism be the permanent data resource that described virtual node resource clustering device is managed;
The first service library of described mesh node is used to constitute service layer, management service and distribution on-line hardware resource;
Described virtual node portal configurator is used for providing the service entrance to the user, and user and service layer are coupled together.
Further, described virtual node resource clustering device comprises that system metadata catalogue listing, apply metadata table, copy metadata catalog table, system's unit's agent list and resource unify the catalogue Sync Manager, wherein:
Described system metadata catalogue listing is used to store the information about this body structure of grid;
Described apply metadata table is used to store content and structure about the data relevant with concrete application, obtains the conditional information of data;
Described copy metadata catalog table is used to store the information about the data trnascription of apply metadata;
Unit of described system agent list is used to store the information about first service structure;
Described resource is unified the catalogue Sync Manager and is used to use the synchronization mechanism based on message queue to equation to upgrade described system metadata catalogue listing, apply metadata table, copy metadata catalog table and unit of system agent list synchronously.
Further, described temporary described time property data resource aggregation container comprises node metadata table, node spatial data catalogue listing and temporary data resource catalog manager, wherein:
Described node metadata table is expanded on described apply metadata table;
Described node spatial data catalogue listing is expanded on described copy metadata catalog table;
Described temporary data resource catalog manager is used for management and upgrades described node metadata table and node spatial data catalogue listing.
Further, the first service library of described mesh node comprises unit's service and first calculating storehouse, workflow interpreter, global resource scheduler and intelligent service-Engine, wherein:
The service of described unit is calculated the storehouse with unit, is used for computational resource monitoring and the service of scheduling unit; Wherein unit's service comprises: the service of the basic unit of grid is basic GIS function service; The first calculation services of specialty is geology field specialty calculation services; The unit service of node territory, the service that provides for autonomy;
Described workflow interpreter is used for the overall situation of workflow service is resolved by standard agreement, and wherein said workflow service is by the service of unit's service with first calculation combination;
Described global resource scheduler is used for by calling the service of scheduling unit the node resource being monitored and being dispatched;
Described intelligent service-Engine is used for realizing service call according to asynchronous with equally loaded, life cycle, service and authority binding and resource entity operation.
Further, described virtual node portal configurator comprises that the user registers with role assignments table, role and right assignment table, domain logic allocation list, authority and resource binding table, overall user domain Sync Manager and node portal service and migrates device.
It is a kind of to equation connection point manager (CP manager) method that the present invention also provides, and comprises the steps:
Step 1 uploads to local metadatabase by the metadata management instrument with local metadata;
Step 2 is newly-increased data record message, and condition of information is set for being untreated in the message queue table;
Step 3, resource are unified the catalogue Sync Manager and read untreated message from the message queue tables;
Step 4 is obtained data according to message content from corresponding tables of data;
Step 5, according to the action type of data, the metadata service on the invocation target node;
Step 6 by the metadata service on the destination node, realizes the synchronous of operation;
Step 7 is obtained the operating result sign of success or failure the metadata service on destination node;
Step 8 is revised the success of the corresponding message in the message queue table on the node of source or the state of failure.
Further, in the described step 1, also comprise the operation of the data in the local metadatabase being deleted and/or upgrading by the metadata management instrument.
Further, in the described step 6, described operation comprises interpolation, modification or deletion action.
Further, in the described step 8, the method for the success of the corresponding message on the node of modification source in the message queue table or the state of failure is: as success, be 255 with the message assignment; As failure, be value less than 255 with the message assignment.
The present invention disposes equation connection point manager (CP manager) device with P2P (equity is calculated) pattern, established the platform that (middleware) based on this built, be acentric, by the directory service synchronization mechanism, not only realize synchronous renewal to the global resource in the equation connection point manager (CP manager) device, for no center framework provides mechanism, and in this grid environment, can not make the whole system collapse even the collapse of a plurality of nodes occurs yet.The present invention is by providing the framework of resource organizations for the informant, defined based on organizing stored resource to describe rule, data resource, software resource, hardware resource describe be organized into as a whole, support at first that particularly national geography information word data standard list structure is a metadata standard, or international OGC (Open GISConsortium, association of open geographic information system) or self-defined standard, the user can be under the situation that does not change former metadata structure, the issue of realization data resource and integrated.The present invention has set up a resource association and the passage that is communicated with between node, make the resource of dispersion become as a whole, by redefining the combination of resource, the node service is transferred to initiatively from passive.
Description of drawings
Fig. 1 is to equation resource polymerizer part relation figure.
Fig. 2 is that resource is unified catalogue Sync Manager architecture.
Fig. 3 is that resource is unified catalogue Sync Manager workflow diagram.
Embodiment
The invention will be further described below in conjunction with the drawings and specific embodiments, can be implemented so that those skilled in the art can better understand the present invention also, but illustrated embodiment is not as a limitation of the invention.
Of the present invention to equation connection point manager (CP manager) device, it is the mass spatial information resource of compiling and share geographical last distribution, it is carried out integration organization and associated treatment, thereby have the spatial information infrastructure of on-demand service ability, can realize the dynamic management of gridding resource.The environmental requirement of service that this manager provides does not influence the management and the independence of each node this locality, do not change original operating system, procotol and service, guarantee the fail safe of user and Remote Node RN, by using the grid middleware platform, shielding bottom hardware and software information provide transparent calculating or service environment fully to the user.This manager couples together the computer resource that is dispersed in various places with network, organically resources such as calculating, data, software are bonded together with custom-designed middleware software, accept calculating and the service request that the various places user proposes with the Web interface, and it is assigned on the suitable node moves.Node is the cluster that server and server, server and PC constitute.To equation connection point manager (CP manager) device is distributed, can divide with physical distribution and also can divide by domain logic.
As shown in Figure 1, the general frame to equation connection point manager (CP manager) device comprises: virtual node resource clustering device, mesh node unit service library, virtual node portal configurator, temporary data resource aggregation container are formed.
Wherein, virtual node resource clustering device and temporary node resource clustering device and data Layer relevance are bigger, mainly be description, Organization And Management to the data resource, and global resource is upgraded synchronously at each node by synchronization mechanism, can support various GIS (Geographic Information System, GIS-Geographic Information System) data of software data form, SDE (spatial database engine), HDF (Hierarchical Data Format, Hierarchical Data Format) file format.Temporary node resource aggregation container mainly is that management produces new data resource by service layer, and makes the data resource of temporary node resource clustering device rise to the permanent data resource of virtual node resource clustering device by synchronization mechanism.In resource discovery procedure, virtual node resource clustering device and temporary data resource aggregation container tool equal authenticity, but need examine the lifting of the content of temporary data resource aggregation container for the content of permanent resources collector.
Mesh node unit service library is the main source that constitutes service layer, can manage the service of forming based on agreements such as web, WRSF, grid streams and distribute on-line hardware resource.Virtual node portal configurator (autonomy) is distributed door, also is Connection Service layer and user's bridge.The user is behind Portal portal entry website, can inquire about from the request end system metadata in the node of (or load minimum) recently to equation connection point manager (CP manager) device, and, promptly searching request is sent to this node after the discovery by present all available and load the is minimum nodes of node IP address detection.This global resource scheduler can be transferred the search work that all nodes participate in metadata and service, compare with centralized way of search, to improve search efficiency greatly, thereby realize the seamless integrated of the gridding of information search mode, rapid, transparence and all node information.
In order to strengthen door and operating efficiency, the source synchronous manager is the promoter of resource sharing platform source synchronous, disposes metadata, the information on services of all nodes in each node computing pool.Each virtual node all contains the source synchronous manager of global field, and realizes the data sync of global field.
Virtual node resource clustering device comprises: system metadata catalogue listing, apply metadata table, copy metadata catalog table, system's unit's agent list and resource are unified the catalogue Sync Manager.
The system metadata catalogue listing: mainly be information about this body structure of grid, as, network interconnection situation and address.Can be designed to following two tables:
(1) system metadata catalogue listing
(2) node routing table
Apply metadata table: about the content and structure of the data relevant, obtain the condition of data etc. with concrete application.
Copy metadata catalog table: about the information of the data trnascription of apply metadata, as the map information between: file (spatial data) and the concrete storage system.
Unit of system agent list: mainly be information about first service structure, as, service content and address.
Resource is unified the catalogue Sync Manager: use mechanism and driving to above-mentioned four Resource TOCs of synchronous renewal of equation.
Resource is unified the specific implementation of catalogue Sync Manager:
The operation principle that resource is unified the catalogue Sync Manager is based on the synchronization mechanism of message queue and by message-driven.At present, China has accumulated the digitlization space information resource of considerable scale, but because operation architecture, scattered each area, each unit of being distributed in.Along with the quickening that national information is built step, how the implementation space sharing of information resource becomes the important topic of pendulum in face of us.Traditional solution is that data are carried out centralized management, its advantage is centralized management and the maintenance of being convenient to shared data, thereby guarantee the consistency of data and reduce data redundancy, in case but the public database collapse, each node just can not be shared data each other.Thus, if each node is stored the different copies of same data, just can address the above problem preferably.Particularly, if adopt the firewall-penetrating characteristic of a kind of message queue synchronization mechanism, can realize the uploading of resource between different node, download, add based on the web service technology, operation such as modification, deletion.A message queue is all safeguarded in each node inside, and data synchronization drives by message, and it and conventional message formation have bigger difference as MSMQ (MicroSoft Message Queue, Microsoft's message queue).The realization principle of MSMQ is: the sender of message puts into a container (calling Message) to the information of oneself wanting to send, and then it is saved in the message queue (Message Queue) in a system share space; The message sink program in this locality or strange land is taken out the message of issuing it again and is handled from this formation.Above-mentioned message queue is only limited at local operation, need not to send message, and concrete data sync operation all is that the Web service that local node initiatively calls other nodes is finished, and this is a place with traditional message queue a great difference.In addition, though the message queue table structure in each node database is identical, independent fully each other.By voluntarily the design the metadata management instrument upload the data to metadatabase or to the data in the metadatabase upgrade with deletion action after, the message queue table will be the related data recording messages, and state is set to be untreated.Resource is unified the catalogue Sync Manager and is obtained untreated message by reading the message queue table, and message is carried out analyzing and processing, obtains key message and carries out data sync.Advantage at this message queue synchronization mechanism is on physical structure, the notion of weakened center and node has realized no center framework, has directly realized communicating by letter between node and the node.
The data structure of message queue specifically can be designed to three tabulations: global message queue table, node message send queue table (sending to other node) and the queue table of node message sink.
The global message queue table is used to preserve the information such as state when synchronous.
Node message send queue table is used to preserve the information such as state when synchronous.
| Title | Type | Length | Explanation |
| LogID | INT | 32 | Message id |
| SrcNodeID | VARchar | 16 | The node identification code is bound with following tabulation |
| MetadataList | VARchar | Elongated | Node metadata ID tabulation |
| SendFlag | Boolean | 1 | The message flag position, on behalf of message, 0 do not send, and 1 representative has sent success |
| DateTime | DateTime | DateTime | Message time of advent |
| Operator | VARchar | 64 | The operator |
| OpType | Boolean | 1 | Action type (0-inserts, and 1-upgrades, the 2-deletion) |
The queue table of node message sink is used to preserve the information such as state when synchronous.
| Title | Type | Length | Explanation |
| LogID | INT | 32 | Message id |
| SrcNodeID | VARchar | Elongated | The node identification code is bound with following tabulation |
| MetadataList | VARchar | Elongated | Node metadata ID tabulation |
| DownloadFlag | Boolean | 1 | The flag bit of whether downloading, 0 representative data are not downloaded from the center, and 1 representative is downloaded |
| DateTime | DateTime | DateTime | Message time of advent |
| OpType | Boolean | 1 | Action type (0-inserts, and 1-upgrades, the 2-deletion) |
Based on the architecture of message queue synchronization mechanism as shown in Figure 2.Under distributed environment, data synchronization is the data sharing between the node, it is a kind of acentric framework, each node is safeguarded the database in metadatabase, user information database, trend of work storehouse and the notice information storehouse of this node, and provide correspondingly metadata service and notice information service, safeguard the message queue of this node simultaneously, move the source synchronous manager of this node.The resource of local node is unified the catalogue Sync Manager, by the metadata service and the notice information service of calling this node and other nodes, realizes the uploading of resource, downloads, adds, modification, deletion etc., finally finish resource synchronously.
Synchronization policy based on the message queue synchronization mechanism: being present in synchronously between the node of resource, suppose to have N node, when the new resource of typing on some nodes, to unify the catalogue Sync Manager by resource and call metadata service of this node or the particular content that new resources are obtained in the notice information service, and then the service of calling other (N-1) individual nodes uploads to other (N-1) individual node with new resources, thus realized that resource uploads synchronously.The synchronization principles of deletion, renewal operation is similar.
Each step of source synchronous can be an off-line mode, if the network disrupted condition occurs between certain two node, when network connect to recover, above-mentioned synchronization mechanism still can read the message queue table and work on and can not produce omission.The message queue table is the core of synchronization policy, and the message queue table is safeguarded all operations record and mode of operation, thereby has guaranteed synchronous the carrying out smoothly of offline resources.Node manages and operates data by the metadata management instrument.
Each node is disposed identical metadata service, and resource is unified catalogue Sync Manager and metadatabase.Its concrete process is as follows:
1. upload this locality of data: metadata is imported in the middle of each corresponding table of local metadatabase by local metadata management instrument, and in the message queue table other node recording messages simultaneously, Status Flag is " being untreated ", i.e. ChargeFlag=0.
2. obtain routing iinformation: data synchronization is to be undertaken by the service of calling other nodes, so at first will obtain the routing iinformation of other nodes.By reading all information in the overall routing table in the local data base, it is left in the buffer memory, when needing the service of invocation target node, mate with routing iinformation in the buffer memory, obtain the IP address of destination node.
3. the design of data sync: resource is unified the catalogue Sync Manager by reading the message queue table, according to time sequencing obtain quota (such as, be set to each 10) untreated message.According to the value of obtaining the DesTab (sign of Data Receiving table) in the message, carry out the operation synchronously such as interpolation, deletion and renewal of corresponding data, the particular content of each operation is as follows:
Add operation: when action type is insertion, IDList (ID tabulation) according to this message, from corresponding tables of data, obtain data, be encapsulated among the DataSet, metadata service--the InsertDataInfo of invocation target node, data are inserted in the corresponding tables of data of destination node, return the whether successful sign of insertion this moment.If all insert successfully, just change corresponding condition of information ChargeFlag=255 in the local message queue table.If there is one to insert failure, (number of times of handling failure is above 10 times just to revise the value of the ChargeFlag of that corresponding message according to the number of times of handling failure, be considered as can not handling for a certain reason, the recording processing number of times surpasses 10 times message in daily record simultaneously).When synchronization means reads the message queue table again, still can handle this message, at this moment, the data of having inserted are not operated, directly return true.
Upgrade operation: when action type is renewal, IDList (ID tabulation) according to this message, from the corresponding tables of data of database, obtain data, be encapsulated among the DataSet, metadata service-the UpdateDataInfo of invocation target node, the corresponding data in the corresponding data table of renewal destination node.At first inquire about whether there are corresponding data in this table,, return false if do not have, otherwise, operation continued.To return the operation flag of total data that this message comprises at last, if the update all success just changes corresponding condition of information ChargeFlag=255 in the local message queue table.If there is one not upgrade successfully, (number of times of handling failure is above 10 times just to revise the value of the ChargeFlag of that corresponding message according to the number of times of handling failure, be considered as can not handling for a certain reason, the recording processing number of times surpasses 10 times message in daily record simultaneously).So, when synchronization means reads message queue again, this message is proceeded to handle.
Deletion action: when operation types is deletion, metadata service--the DeleteDataInfo of invocation target node, according to the value among the IDList (ID tabulation) of this message, corresponding data in the corresponding data table under the deletion destination node.At first inquire about whether there are corresponding data in this table,, return false if do not have, otherwise, operation continued.Return the sign of total data that this message comprises operation at last, if all delete successfully, then change corresponding condition of information ChargeFlag=255 in the local message queue table, otherwise, (number of times of handling failure is above 10 times just to revise the value of the ChargeFlag of that corresponding message according to the number of times of handling failure, be considered as can not handling for a certain reason, the recording processing number of times surpasses 10 times message in daily record simultaneously).When synchronization means reads the message queue table again, this message is proceeded to handle.
System flow, source synchronous operation generally are divided into eight steps, as shown in Figure 3, are described below in the mode of flow chart:
Step 1 uploads to local metadata local data base or operation is deleted, upgraded to the data in the metadatabase by the metadata management instrument;
Step 2 is newly-increased data record message, and condition of information is set for being untreated in the message queue table;
Step 3, resource are unified the catalogue Sync Manager and read untreated message from the message queue tables;
Step 4 is obtained data according to message content from corresponding tables of data;
Step 5, according to the action type of data, the metadata service on the invocation target node;
Step 6, by the metadata service on the destination node, the synchronous of (add, revise or deletion) operated in realization;
Step 7 is obtained operation flag (success or failure) the metadata service on destination node;
Step 8 is revised the corresponding condition of information in the message queue table on the node of source, is 255 successfully, and failing is＜255.
Temporary data resource aggregation container, it is similar to virtual node resource clustering device, and except that having virtual node resource clustering device characteristics, can migrate to virtual node resource clustering device after audit.Usually workflow is not only a long affair mechanism, and has duality usually at the data resource of workflow, promptly be that achievement (output) data are again basis (input) data, for avoiding Resource TOC load and randomness, set up temporary data resource aggregation container, as the container of provisional log-on data.Temporary data resource aggregation container comprises: node metadata table, node spatial data catalogue listing and temporary data resource catalog manager.
The node metadata table can be expanded on above-mentioned apply metadata table.
Node spatial data catalogue listing can be expanded on above-mentioned copy metadata catalog table.
Virtual node portal configurator, PORTAL is divided into primary and secondary, but emphasizes equity.Main PORTAL emphasizes of overall importance, and inferior PORTAL can consider the resource of this virtual node except having resource of overall importance.Primary and secondary PORTAL takes one-stop entry mode to solve mutual trust mode (sharing and synergistic mechanism).PORTAL is provided masterplate.But main PORTAL discovery and integration mechanism bulk migration.Security permission management and mechanism (binding).Virtual node portal configurator comprises: user's registration and role assignments table, and role and right assignment table, the domain logic allocation list, authority and resource binding table, overall user domain Sync Manager, the node portal service is migrated device.
Mesh node portal mainly is divided into main portal and divides portal, adopts the design of Portlet know-why and realizes.Node portal mechanism provides the unified container of realizing resource and service registry, cancellation, renewal, issue and discovery under the distributed heterogeneous network environment, also realized simultaneously the Service Source that shared platform provides, be that node does not need to work out any program, node not only can be issued the resource of oneself, also can manage and use own resource (embodiment autonomy), simultaneously, node can also be shared the resource (global resource) of other nodes, and all software functions of universal succession platform, realized the shared and collaborative of resource.
Main portal is a relative notion with dividing portal, and the function that embodies is identical, difference only be user capture the inlet difference, obtain the approach difference of data, the efficient difference of execution (mainly being subjected to the influence of Network Transmission).What the user visited when main portal enters is operations such as the database of node this locality, the service execution inquiry of calling node this locality, demonstration, so efficient wants high; Otherwise, from minute portal enters time visit be operations such as the long-range strange land database that divides node, the service execution inquiry of calling the branch node, demonstration, so efficient is more relatively low.In addition, the difference of main portal and branch portal is also different with the data category that can visit, and main portal can have access to all data that are registered to this node, and divides portal can only have access to the resource that this node is designated " overall situation ".
Mesh node portal is the impression window of node information, and the platform of inquiry, retrieval metadata and service is provided for the user.The branch portal of each node is an isomorphism from function, different only be the data resource difference that has.Node divides Portal that information issuing function, system management function, data, services function, data resource directory services functionality, metadata service function, body search function, geological work degree function, the service of gridding resource Potential Evaluation and node monitoring function are provided.Node divides is managing a large amount of spatial datas and metadata among the portal, these data are seen from the angle of data format and are comprised MapGIS data, ArcGIS data and GeoStar data, see from the angle of data organization mode to comprise vector data and raster data, see ten thousand geochemistry datas that comprise 1: 20,1: 20 ten thousand geologic map data, 1: 20 ten thousand hydrological map data, 1: 5 ten thousand geologic map data, 1: 50 ten thousand geologic map data, orefield data and 1: 5 ten thousand ore deposit adjusting data etc. from the angle of data content.
Node divides between the portal by the Internet intercommunication interconnection, and unified node distribution map is provided, and provides ground to transfer all nodes in the grid environment to divide the unified inlet of portal in the node distribution map.
In addition, the data that node divides portal can also share other node can be published to the data message of this node on other node simultaneously, and then realize distributed collaboration.This resource-sharing and distributed collaboration are based on that message mechanism and synchronization policy finish, and adopt the middleware Technology to realize.Resource-sharing among the node portal and distributed collaboration middleware are called " node portal Sync Manager ".
Node portal Sync Manager adopts the firewall-penetrating characteristic of message mechanism based on the web service technology, realizes registration, cancellation, renewal, issue and the discovery of resource between different node.
A message queue is all safeguarded in each node inside, and it and conventional message formation have bigger difference as MSMQ (MicroSoft Message Queue, Microsoft's message queue).The realization principle of MSMQ: the sender of message puts into a container (we are referred to as Message) to the information of oneself wanting to send, and then it is saved in the message queue (Message Queue) in a system share space; The message sink program in this locality or strange land is taken out the message of issuing it again and is handled from this formation.Message queue in the grid of the present invention is only limited at local operation, need not to send message, and it all is that Web service by local node initiatively calls other nodes is finished that concrete resource registering, cancellation, renewal, issue and discovery are operated.
Node divides portal only to need in the resource of managing local node, after resource being implemented in registration, cancellation and the renewal of this locality, only needing to write corresponding Message Record in the message queue table of this locality gets final product, the Message Record that is read log-on message by node portal Sync Manager is published to respective resources information on other node then, and all resources are all found by node portal is unified at last.
Node portal Sync Manager is based on message-driven mechanism and the realization of web service mechanism.The architecture of synchronization mechanism is as shown in Figure 2:
Under distributed environment, data synchronization is the data sharing between the node portal, it is a kind of acentric framework, each node portal safeguards the database in metadatabase, user information database, trend of work storehouse and the notice information storehouse of this node, and correspondingly metadata service and notice information service are provided, safeguard the message queue of this node simultaneously.Local node portal Sync Manager calls metadata service and notice information service by calling this node and other nodes, realizes registration, cancellation, renewal and the issue of resource, finally finishes the synchronous of resource.
Being present in synchronously between the node of resource, suppose to have N node, when having registered new resource on some nodes, to call metadata service of this node or the particular content that new resources are obtained in the notice information service by node portal Sync Manager, and then the service of calling other (N-1) individual nodes uploads to other (N-1) individual node with new resources, thus realized resource registering synchronously.The synchronization principles of nullifying, upgrade operation is similar.
Each step of source synchronous can be an off-line mode, if the network disrupted condition occurs between certain two node, when network connect to recover, synchronization mechanism still can read the message queue table and work on and can not produce omission.The message queue table is the core of synchronization policy, and the message queue table is safeguarded all operations record and mode of operation, thereby has guaranteed synchronous the carrying out smoothly of offline resources.Node manages and operates data by the metadata management instrument.Each node is disposed identical metadata service, synchronization means and metadatabase according to design.Its concrete process is as follows:
The first step: this locality registration of resource
By local node portal resource registering instrument, resource registering in the middle of local metadatabase, is write resource registering message simultaneously in the message queue table, Status Flag is " being untreated ", i.e. ChargeFlag=0.
Second step: node portal Sync Manager carries out the resource issue
Node portal Sync Manager obtains ID number of other node and then obtains the IP address from message queue, call the issue of the web service realization resource on other node then.
Based on above architecture and synchronization policy, node portal source synchronous flow process is illustrated in fig. 3 shown below:
As can be seen from the figure, the source synchronous between node portal generally is divided into eight links.
1) by node portal resource registering instrument local resource is registered to local data base or operation is nullified, upgraded to existing resource in the database;
2) in the message queue table, write message, and condition of information is set for being untreated for newly-increased data;
3) node portal Sync Manager reads untreated message from the message queue table;
4) obtain corresponding resource according to message content;
5) according to the action type of data, the metadata service on the invocation target node;
6), realize carrying out registration, cancellation or the renewal of resource by the metadata service on the destination node;
7) obtain operation flag (success or failure) the service of the metadata on destination node;
8) revising the corresponding condition of information in the message queue table on the node of source, is 255 successfully, and failing is＜255;
Mesh node unit service library comprises: unit's service is calculated storehouse, workflow interpreter, global resource scheduler and intelligent service-Engine with unit.
Unit's service is calculated the storehouse with unit, comprises the service of scheduling unit: computational resource monitoring and scheduling; The basic unit of grid service: basic GIS function service; The first calculation services of specialty: geology field specialty calculation services; Node territory unit service: the service that autonomy provides; Workflow service: by the service (by agreement) of unit's service with first calculation combination.
The workflow interpreter is resolved (pressing standard agreement) to the overall situation of workflow service
The global resource scheduler is monitored and is dispatched the node resource by calling the service of scheduling unit.
The intelligence service-Engine is realized service call according to asynchronous with technology such as equally loaded (passing through Resource Scheduler), life cycle, service and authority binding, resource entity operations.
The above embodiment is the preferred embodiment that proves absolutely that the present invention lifts, and protection scope of the present invention is not limited thereto.Being equal to that those skilled in the art are done on basis of the present invention substitutes or conversion, all within protection scope of the present invention.Protection scope of the present invention is as the criterion with claims.
Claims (8)
1. one kind to equation connection point manager (CP manager) device, be used for realizing the Organization And Management of grid computing environment gridding resource, it is characterized in that, comprise virtual node resource clustering device, mesh node unit service library, virtual node portal configurator and temporary data resource aggregation container, wherein:
Described virtual node resource clustering device is used for that permanent data resource to gridding computer environment is described, Organization And Management, and by synchronization mechanism global resource is upgraded synchronously at each node;
Described temporary data resource aggregation container, be used to manage the new data resource that produces by service layer, this new data is middle achievement data, and to make this new data resource conversion by synchronization mechanism be the permanent data resource that described virtual node resource clustering device is managed;
The first service library of described mesh node is used to constitute service layer, management service and distribution on-line hardware resource;
Described virtual node portal configurator is used for providing the service entrance to the user, and user and service layer are coupled together;
Described virtual node resource clustering device comprises that system metadata catalogue listing, apply metadata table, copy metadata catalog table, system's unit's agent list and resource unify the catalogue Sync Manager, wherein:
Described system metadata catalogue listing is used to store the information about this body structure of grid;
Described apply metadata table is used to store content and structure about the data relevant with concrete application, obtains the conditional information of data;
Described copy metadata catalog table is used to store the information about the data trnascription of apply metadata;
Unit of described system agent list is used to store the information about first service structure;
Described resource is unified the catalogue Sync Manager and is used to use the synchronization mechanism based on message queue to equation to upgrade described system metadata catalogue listing, apply metadata table, copy metadata catalog table and unit of system agent list synchronously.
2. according to claim 1ly equation connection point manager (CP manager) device be is characterized in that described temporary data resource aggregation container comprises node metadata table, node spatial data catalogue listing and temporary data resource catalog manager, wherein:
Described node metadata table is expanded on described apply metadata table;
Described node spatial data catalogue listing is expanded on described copy metadata catalog table;
Described temporary data resource catalog manager is used for management and upgrades described node metadata table and node spatial data catalogue listing.
3. according to claim 2 equation connection point manager (CP manager) device be is characterized in that described mesh node unit service library comprises that unit's service and unit calculate storehouse, workflow interpreter, global resource scheduler and intelligent service-Engine, wherein:
The service of described unit is calculated the storehouse with unit, is used for computational resource monitoring and the service of scheduling unit; Wherein unit's service comprises: the service of the basic unit of grid is basic GIS function service; The first calculation services of specialty is geology field specialty calculation services; The unit service of node territory, the service that provides for autonomy;
Described workflow interpreter is used for the overall situation of workflow service is resolved by standard agreement, and wherein said workflow service is by the service of unit's service with first calculation combination;
Described global resource scheduler is used for by calling the service of scheduling unit the node resource being monitored and being dispatched;
Described intelligent service-Engine is used for realizing service call according to asynchronous with equally loaded, life cycle, service and authority binding and resource entity operation.
4. according to claim 3 to equation connection point manager (CP manager) device, it is characterized in that described virtual node portal configurator comprises that the user registers with role assignments table, role and right assignment table, domain logic allocation list, authority and resource binding table, overall user domain Sync Manager and node portal service and migrates device.
5. one kind to equation connection point manager (CP manager) method, it is characterized in that, comprises the steps:
Step 1 uploads to local metadatabase by the metadata management instrument with local metadata;
Step 2 is newly-increased data record message, and condition of information is set for being untreated in the message queue table;
Step 3, resource are unified the catalogue Sync Manager and read untreated message from the message queue tables;
Step 4 is obtained data according to message content from corresponding tables of data;
Step 5, according to the action type of data, the metadata service on the invocation target node;
Step 6 by the metadata service on the destination node, realizes the synchronous of operation;
Step 7 is obtained the operating result sign of success or failure the metadata service on destination node;
Step 8 is revised the success of the corresponding message in the message queue table on the node of source or the state of failure.
6. according to claim 5 equation connection point manager (CP manager) method be is characterized in that, in the described step 1, also comprise the operation of the data in the local metadatabase being deleted and/or upgrading by the metadata management instrument.
7. according to claim 5ly equation connection point manager (CP manager) method be is characterized in that in the described step 6, described operation comprises interpolation, modification or deletion action.
8. according to claim 5 equation connection point manager (CP manager) method be is characterized in that in the described step 8, the method for revising the state of the success of the corresponding message in the message queue table on the node of source or failure is: as success, be 255 with the message assignment; As failure, be value less than 255 with the message assignment.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2009102385031A CN101741614B (en) | 2009-11-20 | 2009-11-20 | Equivalent type node manager and equivalent type node management method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2009102385031A CN101741614B (en) | 2009-11-20 | 2009-11-20 | Equivalent type node manager and equivalent type node management method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101741614A CN101741614A (en) | 2010-06-16 |
| CN101741614B true CN101741614B (en) | 2011-11-30 |
Family
ID=42464548
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2009102385031A Expired - Fee Related CN101741614B (en) | 2009-11-20 | 2009-11-20 | Equivalent type node manager and equivalent type node management method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101741614B (en) |
Families Citing this family (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8572241B2 (en) * | 2010-09-17 | 2013-10-29 | Microsoft Corporation | Integrating external and cluster heat map data |
| CN102222090B (en) * | 2011-06-02 | 2012-12-05 | 清华大学 | Mass data resource management frame under cloud environment |
| CN102799659B (en) * | 2012-07-05 | 2015-01-21 | 广州鼎鼎信息科技有限公司 | Overall repeating data deleting system and method based on non-centre distribution system |
| CN104219280B (en) * | 2014-07-01 | 2018-03-16 | 武汉众廉科技有限公司 | A kind of intelligent use data transmission channel |
| CN104950757B (en) * | 2015-06-12 | 2018-03-16 | 北京奇虎科技有限公司 | The method and system of monitoring of containers |
| CN105512332A (en) * | 2015-12-25 | 2016-04-20 | 北京奇虎科技有限公司 | Method for linking and adjusting data between databases, device and system |
| US10193977B2 (en) * | 2016-04-29 | 2019-01-29 | Huawei Technologies Co., Ltd. | System, device and process for dynamic tenant structure adjustment in a distributed resource management system |
| CN106506542B (en) * | 2016-12-16 | 2019-11-22 | 清华大学 | Unified cloud storage resource share method and device based on cloud catalogue |
| CN110909057B (en) * | 2019-11-22 | 2023-06-16 | 哈尔滨工程大学 | Numerical pool virtual test data middleware system and working method thereof |
| CN115017222B (en) * | 2022-08-01 | 2022-11-08 | 深圳市其域创新科技有限公司 | Information processing system and method based on multiple information sources |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1713595A (en) * | 2004-06-15 | 2005-12-28 | 国际商业机器公司 | Use method and system of independent external resources within requesting grid environments |
-
2009
- 2009-11-20 CN CN2009102385031A patent/CN101741614B/en not_active Expired - Fee Related
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1713595A (en) * | 2004-06-15 | 2005-12-28 | 国际商业机器公司 | Use method and system of independent external resources within requesting grid environments |
Non-Patent Citations (1)
| Title |
|---|
| 李超岭等.国家地质空间信息网格结点计算池构架及其实现.《国土资源信息化》.2006,(第5期),2-8. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101741614A (en) | 2010-06-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101741614B (en) | Equivalent type node manager and equivalent type node management method | |
| CN100573528C (en) | Digital museum gridding and building method thereof | |
| Lea et al. | COOL: System support for distributed programming | |
| CN112685385A (en) | Big data platform for smart city construction | |
| US7735115B2 (en) | System which enforces policy for virtual private organization and method thereof | |
| CN101587639B (en) | City bus information management and dispatch decision support system based on network | |
| Assante et al. | The gCube system: delivering virtual research environments as-a-service | |
| CN101360123B (en) | Network system and management method thereof | |
| CN103473696A (en) | Method and system for collecting, analyzing and distributing internet business information | |
| CN102591910A (en) | Computer method and system combining OLTP database and OLAP database environments | |
| CN101741696A (en) | Multi-user real-time cooperative system in distributed geographic information environment | |
| Ferreira et al. | PerDiS: design, implementation, and use of a PERsistent DIstributed Store | |
| CN108268614B (en) | Distributed management method for forest resource spatial data | |
| Zhang et al. | Towards building a multi‐datacenter infrastructure for massive remote sensing image processing | |
| Cremonini et al. | Multi-agent systems on the Internet: Extending the scope of coordination towards security and topology | |
| CN103473247B (en) | Geological data information cluster mechanism and interface aggregation system | |
| CN111209323A (en) | Spatial geographic information big data processing system | |
| KR20070083241A (en) | Systems and methods for the implementation of a synchronization schemas for units of information manageable by a hardware/software interface system | |
| Henderson | Systems Engineering for Business Process Change: collected papers from the EPSRC research programme | |
| Arcieri et al. | Coherence maintainance in cooperative information systems: the Access Key Warehouse approach | |
| Watson | Databases in grid applications: Locality and distribution | |
| CN113822557A (en) | Data fusion management system, data fusion management device, electronic device, and medium | |
| Shi et al. | Build city traffic information service system based on grid platform | |
| Kolarić et al. | Toward an open IoT implementation for urban environments: The architecture of the DBL SmartCity platform | |
| CN106570052A (en) | Two-dimensional GIS model-based method for constructing offshore oil application platform |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20111130 Termination date: 20141120 |
|
| EXPY | Termination of patent right or utility model |