CN1897570A

CN1897570A - Large data-set task node transferrying method based on content and relation separation

Info

Publication number: CN1897570A
Application number: CNA2006100122659A
Authority: CN
Inventors: 徐恪; 吴鲲; 王海洋
Original assignee: Tsinghua University
Current assignee: Tsinghua University
Priority date: 2006-06-15
Filing date: 2006-06-15
Publication date: 2007-01-17
Anticipated expiration: 2026-06-15
Also published as: CN100471170C

Abstract

The invention belongs to the technical field of Internet IP router data and memory management methods, and is characterized in that it contains the following steps in sequence: sequentially establishing a content data memory pool containing routing information and a structural data memory pool containing the relationship between routing entries; according to the above two The difference between the first address of the memory address pool is obtained and the mapping relationship of the memory pool pair on the storage address is established; the respective free memory linked lists of the above memory pools are respectively established; when the routing table is to be moved to an independent memory area, it is necessary to use The size of the respective memory pools establishes content data memory pools and structural data memory pools in the new memory area, and re-establishes the mapping relationship and free memory list in the new memory space to manage address holes that may appear during routing maintenance. The invention realizes the dynamic maintenance of large data sets in the software system structure of the IP router.

Description

The node moving method of content-based large data sets task of separating with relation

Technical field

The node moving method of content-based large data sets task of separating with relation belongs to internet ip router data and memory management algorithm research field.

Background technology

In the computer network system in modern times, router is most crucial connection device.Rapid development of Internet has been brought the innovation of router architecture.Next Generation Internet has proposed new demand to ip router architecture of future generation.Single node is optimized, improves its disposal ability and no longer be the improved fundamental way of system, because this mode is subject to the development speed influence of physical device.Traditional IP router architecture based on single node can't adapt to rapid development of network.Extendible architecture is the main developing direction of ip router of future generation.

In the expandable route software architecture, the migration of task is basic structural support.The computation model that can expand software architecture only solved the static abstract problem of system, it answered system this how to organize basic problem with allocating task.But for a real extendible system, the dynamic expansion of system configuration is vital.Because search efficiency, the storage of routing table is not to store with uncorrelated " table " type structure, normally stores based on Trie tree structure or other organized structures.Rebuild the copy that means data incessantly, these incidence relations also will obtain recovering, and for large data sets, the relation of restore data set is the comparison difficulty at short notice.That summarizes says that the complexity that data relationship is rebuild fully non-reproduction excessive at the time overhead of large-scale dataset and the data incidence relation has restricted the migration of these tasks between router control plane node.Therefore, reach the real software architecture expanded, just must solve the migration problem of large data sets task.

Summary of the invention

The object of the present invention is to provide a kind of node moving method of content-based large data sets task of separating with relation.

The invention is characterized in that it contains following steps successively:

Step (1.): set up the content-data storage pool, write down the first address of each memory pool, and storage comprises targeted subnet, next jumping, outgoing interface and the priority routing iinformation in interior router interior;

Step (2.): set up the structured data memory pool, write down the first address of this memory pool and the association between the inner routing iinformation of storage router, wherein comprise pointer and two pointers that are used for quick retrieval of pointing to the content-data memory pool;

Step (3.): initialization content-data memory pool and structured data memory pool;

Step (4.): set up the mapping relations one to one on memory address between described content-data memory pool and the structured data memory pool; Simultaneously, set up each memory address and free memory chained list, this chained list is a pointer chained list that is used to store the free memory address, so that the memory address cavity that forms owing to cancelling of route entry is managed; Wherein, the side-play amount of corresponding clauses and subclauses on memory address is identical between the described mapping relations difference that is in particular in content-data memory pool and structured data memory pool first address in internal memory and the described memory pool;

Step (5.): after executing the deletion action of route entry, kept a record by the details of the free memory chained list in the step (4.) to the memory address cavity of being brought by deletion, these details comprise: the first address pointer of free memory chained list and the address of next free space region of memory;

Step (6): when carrying out route entry increase operation, whether the first address of at first judging described free memory chained list is empty, if this address non-NULL, then the new route entry that needs are added preferentially is stored in the cavity, address by the free memory linked list maintenance, and the first address pointer of modification free memory chained list, if this address is empty, then there is not the cavity, address in the memory address of explanation storage route entry, need deposit newly-increased route entry to the new memory address of system's application;

Step (7.): when needs move to another piece to routing table independently during region of memory, carry out according to the following steps successively:

Step (7.1): the size of the content-data memory pool of Yi Donging as required, the corresponding memory of application space in new region of memory, and write down the first address in this space;

Step (7.2): the size of the structured data memory pool of Yi Donging as required, the corresponding memory space of application in new region of memory, and write down the first address of this memory space;

Step (7.3): the new memory pool described in establishment step (7.1), the step (7.2) between mapping relations, write down the side-play amount between new content-data memory pool and structured data memory pool first address;

Step (7.4): the described new structured data memory pool of scanning step (7.2), and according to corresponding pointer address in the new structured data memory pool of the described new side-play amount modification of step (7.3);

Step (7.5): in new region of memory, set up the free memory chained list, the free memory address that may exist is managed.

We to said method have carried out emulation in the routing table scale under the situation from 100K to the 1M bar to this agreement according to above-mentioned execution mode, and said method and the traditional mode of rebuilding routing table are fully compared.Wherein Fig. 6 is to use this method to rebuild the analog simulation result of routing table institute spended time, can find that this process meets linear computation complexity.Fig. 7 adopts true routing table to experimentize under the situation, the contrast of said method and traditional complete reconstruction mode spended time under the routing table situation of movement, the node moving method that can find content-based large data sets task of separating with relation from figure is compared at the routing table traveling time with traditional approach very big advantage, and also is far smaller than traditional complete reconstruction mode with its growth rate of increase of route entry.Fig. 8 has shown the speed-up ratio of this mode and traditional approach, and circles mark discrete among the figure is represented under the corresponding route bar said conditions, the ratio of the traditional approach that records and the reconstruction time of the node moving method of content-based large data sets task of separating with relation.Speed-up ratio is distributed between 23.47 to 58.90, though these discrete speed-up ratio data do not have very level and smooth data trend relation.But, still can find out increase substantially with route bar number, speed-up ratio has the trend of increase.

Description of drawings

Fig. 1. routing table structrual description figure;

Fig. 2. memory pool and mapping relations figure

Fig. 3. free memory list structure figure;

Fig. 4. routing table is duplicated schematic diagram;

Fig. 5. overall process flow figure of the present invention;

Fig. 6. the emulated data curve chart;

Fig. 7. routing table reconstruction time experimental curve diagram;

Fig. 8. the speed-up ratio curve chart;

Fig. 9. application exemplary plot of the present invention.

Embodiment

Under the traditional large data sets task node migration and the efficient of reconstruction mode, had a strong impact on the development of expansion software architecture.Designing a kind of more high efficiency moving and storage means and satisfy that route in the expandable route is calculated and these two Core Features of management of route, is main contribution of the present invention.

In description of the invention, content-based thought of separating with structure, routing table is described by content-data memory pool, structured data memory pool and this two right mapping relations of memory pool, wherein stored the essential information of route entry in the content-data memory pool, comprising: targeted subnet, next jumping, outgoing interface and priority.The structured data memory pool then is used for storing the fundamental relation between the route entry, searches for convenience, because routing table generally uses the structure of Trie tree to store, the structured data memory pool is exactly a kind of special description at the Trie tree.The right mapping relations of memory pool wherein can be expressed as y=f (d (x))=x, and wherein x is the logical address of structured data, and y is the logical address of content-data.

Routing table structrual description figure sees Fig. 1

Memory pool and mapping relations are described and are seen Fig. 2

In the process of maintenance content datarams pond, structured data memory pool, if the release of route entry, the address that need write down this memory block timely also is recorded in this address in the free memory chained list, when the first address pointer of this chained list is not sky, for the save memory space, need be with preferential being stored in the cavity, address that the free memory chained list write down of initiate route entry.

The free memory list structure is seen Fig. 3

When needs carry out the routing table migration, at first in the purpose region of memory of migration, rebuild corresponding construction according to source contents datarams pond and structured data memory pool size, data content and the relationship description with correspondence duplicates again.Duplicate finish after, write down the new content-data memory pool and the first address of structured data memory pool, and set up new mapping relations, at last, by the structured data memory pool is scanned, finish the migration and the reconstruction of routing table according to the pointer address in the new first address mapping relations modification structured data memory pool.

Fig. 4 is seen in the reproduction process of routing table in memory block

Overall process flow figure of the present invention sees Fig. 5

The frame construction drawing of expandable route is that the present invention uses exemplary plot and sees Fig. 9

The present invention can remember processing expenditure with the calculating in reducing extensive routing table process of reconstruction separating of correlation by the content to route information, and, thereby by the free memory chained list the preferential use of the management of empty memory is discharged the empty memory that brings by internal memory and reduced the waste on the memory headroom, and, data show by experiment, this method is compared with traditional complete method for reconstructing has at the bottom of the computation complexity characteristics such as good stability.Solved support issue as the important component part in the expandable route software architecture to the large data sets task immigration.

Claims

1. The migration method of the large data set task based on the separation of content and relationship is characterized in that the method is realized by adding a dynamic task migration module in the software architecture of Internet IP router or terminal host, and contains the following steps successively:

Step (1.): Establish a content data storage pool, record the first address of each memory pool, and store the internal routing information of the router including the target subnet, next hop, outgoing interface and priority;

Step (2.): Establish a structural data memory pool, record the first address of the memory pool and store the association between router internal routing information, which contains a pointer to the content data memory pool and two pointers for fast retrieval;

Step (3.): Initialize the content data memory pool and the structure data memory pool;

Step (4.): establish a one-to-one mapping relationship on storage addresses between the content data memory pool and the structure data memory pool; meanwhile, set up each memory address and a free memory linked list, which is a list for storing free memory A linked list of pointers to memory addresses, so as to manage the memory address holes formed due to the revocation of routing entries; wherein, the mapping relationship is embodied in the difference between the first address in the memory of the content data memory pool and the structure data memory pool and the above-mentioned The offsets of the corresponding entries between the memory pools on the memory address are the same;

Step (5.): After executing the deletion operation of the routing entry, the detailed information of the memory address hole caused by the deletion is recorded by the free memory linked list in step (4.), the detailed information includes: free memory linked list The first address pointer and the address of the next free space memory area;

Step (6): When performing the routing entry addition operation, first judge whether the first address of the free memory linked list is empty, if the address is not empty, then store the new routing entry that needs to be added in the first address maintained by the free memory linked list In the address hole, modify the first address pointer of the free memory linked list. If the address is empty, it means that there is no address hole in the memory address storing the routing entry, and it is necessary to apply for a new memory address to the system to store the newly added routing entry;

Step (7.): When it is necessary to move the routing table to another independent memory area, follow the steps below:

Step (7.1): According to the size of the content data memory pool to be moved, apply for the corresponding memory storage space in the new memory area, and record the first address of the space;

Step (7.2): According to the size of the structural data memory pool to be moved, apply for a corresponding storage space in the new memory area, and record the first address of the storage space;

Step (7.3): Establish the mapping relationship between the new memory pool pair described in step (7.1) and step (7.2), and record the offset between the new content data memory pool and the head address of the structural data memory pool;

Step (7.4): scanning the new structural data memory pool described in step (7.2), and modifying the corresponding pointer address in the new structural data memory pool according to the new offset described in step (7.3);

Step (7.5): Establish a free memory linked list in the new memory area, and manage possible free memory addresses.