CN101572625A - IP partition method - Google Patents
IP partition method Download PDFInfo
- Publication number
- CN101572625A CN101572625A CNA2009100829324A CN200910082932A CN101572625A CN 101572625 A CN101572625 A CN 101572625A CN A2009100829324 A CNA2009100829324 A CN A2009100829324A CN 200910082932 A CN200910082932 A CN 200910082932A CN 101572625 A CN101572625 A CN 101572625A
- Authority
- CN
- China
- Prior art keywords
- subregion
- section
- scope
- data
- partition
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention belongs to the computer network field, particularly relates to an IP partition method in mass data store. World IP is divided into different hierarchies, even division is used for partition of IP in each hierarchy, thus adaptation partition according to IP quantity in different areas is realized and data can distribute in each subarea no matter which area the IP partition product is in actual operation. The invention reduces time for storing network service information table of certain area and shows superiority of partition technology.
Description
Technical field
The invention belongs to computer network field, be specifically related to the IP partition method in a kind of mass data storage.
Background technology
In the database application technology, the storage of mass data generally all can be used partitioning technique.This technology is that a very big table is divided into a plurality of little tables and stores different zones respectively into according to certain rule, table so in logic, during physical store, can be as many tables, be stored in different positions, simplify the management activity of database, but also can improve application performance.Because optimizer will be known the value range that uses as the subregion basis, so it just can only use specific subregion directly to inquire about when access list.Because in query procedure, only browse low volume data, just improved query performance naturally.Because external interface is still a table, for the user, application is transparent, the existence of their imperceptible subregion simultaneously.Therefore, showing partitioning technique greatly uses very extensive in mass data storage.
At present the partitioned mode that adopts be with all the IP uniform distributions in the 0.0.0.0-255.255.255.255 scope in each interval, but there is certain defective in the IP uniform distribution in actual applications in each interval method.One co-exists in 4,294 in this scope, 967,295 IP, and in 4096 subregions, approximately there are 1,048,576 IP in each subregion like this with the IP uniform distribution.That is to say to be under the situation of subregion benchmark with IP, have more than 100 ten thousand IP in each subregion.But a lot of areas all also do not reach such scale in the actual conditions, that is to say when practical application IP subregion product, if take such partitioned mode, do not have data substantially in most subregions, and data all concentrate in the subregion of only a few.And adopt such method, and to create time of table of the network information on services in a storage somewhere and want half an hour at least, the time is long.In the prior art method of IP uniform distribution is not embodied the superiority of partitioning technique, do not reach the purpose of improving management and performance by subregion yet.
Summary of the invention
At above-mentioned technical problem, the present invention proposes a kind of IP partition method that the IP scope is suitably reduced according to practical application.The user can be according to the different IP of partition method intelligence division scale of the present invention, what make when IP subregion product enters practice no matter in area, data can both be evenly distributed in each subregion, embody the superiority of big table partitioning technique in mass data processing.
A kind of IP partition method, its step comprises:
1) successively divide world IP scope, obtain various level IP range information and determine the level number n, n is the integer greater than 2;
The corresponding level of outermost world IP scope is 1, and the rest may be inferred, the final districts and cities IP scope respective layer progression n that uses.Because network data exists for the possibility of any possible IP, therefore, total IP scope 0.0.0.0 to 255.255.255.255 is that the first order is an inevitable choice.In case otherwise the IP of certain bar data is in any subregion, data can't be write in the table so.
2) determine to carry out the IP number of levels m in subregion area, set partitioned parameters pi0 according to the IP quantity in i layer area; M is the integer in [1, n], and i is the integer in [(n-m+1), n];
3) be the IP in the even five equilibrium of standard i layer area with pi0, finish the IP subregion.
Further, each divides section to allow the partitioned parameters pi1 of IP at most to set the i layer, last the sectional IP quantity of dividing determining step 3) adds that whether pi0 is less than pi1, if less than, then itself and previous minute section are merged into a branch section, otherwise nonjoinder is an independent partitions section.
In the IP subregion that the present invention carries out, be standard, the IP scope of i level is evenly divided, and the data volume in the non-standard segment can not surpass pi1 at most with the pi0 that sets.
With i level IP scope is example, supposes to have two IP segment limits: { [IP1, IP2], [IP3, IP4] }, its corresponding partitioned parameters is pi0, pi1.The meaning of pi0 is with IP1 to be starting point, is standard with pi0 IP, and [IP1, IP2] and [IP3, IP4] is divided into a plurality of continuous segments, and each segment all comprises pi0 IP.The meaning of pi1 is when marking off last section, if the IP quantity of this section very little, just this section is merged in the last section.But the IP quantity of this non-standard segment can not surpass pi1, if surpassed pi1, with regard to nonjoinder, the IP quantity of so final last section can be less than pi0.
The present invention has following advantage and good effect;
1. because subregion standard of the present invention is to determine that according to the data volume of concrete regional IP therefore, the present invention can be evenly distributed in data each different subregion as far as possible, has demonstrated fully the superiority of partitioning technique.
2. the present invention is divided into the multilayer level with IP, can be according to the IP practical situations in area, concentrate IP to divide by canonical parameter to these districts and cities, and can strengthen the granularity of dividing according to relevant parameter to the IP scope of Fei Ben districts and cities, thereby the least possible foundation does not have the subregion of data substantially, guarantee that each subregion has data as much as possible, reduce total subregion number, reduced the time of building table.
3. the present invention takes each sectional IP is carried out even five equilibrium technological means according to the discontinuity of network data in the IP distribution, can reduce number of partitions, thereby improves subregion efficient.
Embodiment
In order to make those skilled in the art person better understand the present invention, the present invention is described in further detail to provide specific embodiment below.
(1) among the present invention the IP scope is successively divided to local at different levels by world IP, IP is divided into different number of levels n; The number of levels of IP has embodied the complete degree of IP address details, and the n value is big more, can be accurate more to the IP address that IP divides time-like to obtain.Scope as world IP is 0.0.0.0 ~ 255.255.255.255, is 1 grade; The IP of China is a subclass in IP storehouse, the world, is 2 grades; The IP storehouse of a certain provinces and cities is again a subclass of Chinese IP scope, is 3 grades; An area inside the province is again the subclass in IP storehouse inside the province, is 4 grades; And the like.
The judgement of the different layers progression of IP scope can be divided the operating position of IP by concrete area.As in certain practical application example, if need build the table of storage Beijing network information on services, 2 kinds of situations of setting Beijing IP number of levels are arranged: in world IP, Chinese IP, Beijing IP, first kind of situation: obtain China and Pekinese's IP address range, number of levels is 3; Second kind of situation: looking all outside Beijing IP is a scope, only obtains Pekinese's IP address range, and number of levels is 2 so.In the concrete practical application, build when table at Pekinese's network data, these table storage Pekinese data are at most inevitable, will inevitably be much more and store the data of domestic other provinces and cities with respect to than external data, the IP of domestic other provinces and cities is necessary to carry out subregion.If selecting number of levels is 2, the IP of domestic other provinces and cities can only be distributed together with external IP so, can cause domestic data skewness like this.Therefore, selecting number of levels is 3, imports China and Pekinese's IP address range respectively, and the IP to the IP of Pekinese and domestic other provinces and cities carries out subregion with different standards respectively.Like this, the IP of domestic other provinces and cities has allocation rule equally and has participated in distribution, with regard to relative remission the data unbalance degree that distribute.
(2) IP scope: the basis of IP subregion certainly exists an IP scope for the subregion of selecting a certain number of levels.As 0.0.0.0 ~ 255.255.255.255 also is a scope, and only it is total IP section in the world.The scope of IP is determined by number of levels n.In this example, if adopt three IP scopes to divide---total IP scope, IP scope, the IP of the Pekinese scope of China, parameter is 3 so.If only need be just enough to two IP scopes, parameter be 2 so, and the IP scope that participates in dividing should be total IP section and the IP of Pekinese section.
(3) the IP quantity parameter that section post comprises is set: [pi0, pi1; Pn0, pn1], wherein i refers to the IP number of levels, this group parametric representation the IP number that is comprised in the carrying data partition, embody the degree of refinement of subregion.Wherein pi0 (divide the IP section as standard by the IP preferred number in the value representation i layer subregion of (n-m+1)≤i≤n).The pi0 value is big more, represents that the IP number that each subregion comprises is many more, and each subregion is can the logging data amount just big more, but total total points section quantity reduces relatively.The pi0 value is more little, represents that the IP number that each subregion comprises is few more, and data are just few, but total branch sector number amount is with regard to corresponding increase.Pi1 is the maximum permissible IP numbers of i layer subregion, and this parameter mainly is in order to prevent in certain IP scope, and the IP quantity that is divided into that last in each height section section does not successively far reach the value of pi0 and becomes the situation of section separately.By this parameter, if though the IP quantity after last that segment and the preceding paragraph merge in the IP scope has surpassed standard pi0, as long as but be no more than pi1---the IP number of maximum permissions of setting, just can be not with that last segment section of one-tenth separately, become a section but merge with the preceding paragraph.The pi0 value is set according to the network data actual conditions of different regions.The interior data of subregion are good to be no more than 1,000,000 grades in general, so the value of pi0 is just passable about ten thousand grades.Because the pi1 parameter only acts on the last division of initial IP range section, so the number of times of using is not too many, general 50% of the pi0 that do not surpass of the pi1 of setting.Can set according to real data situation and developer's experience in the time of concrete the setting.
(4) number of levels in clear and definite subregion area is not that the data of each level all need subregion in practice, generally is that IP section subregion is carried out in the big more area of number of levels, and the data balancing effect behind the subregion is good more.
Example 1 known China and Pekinese's IP scope are a plurality of not overlapping and discontinuous section, i.e. n=3.Make m=2, then need Beijing described IP scope [p20, p21; P30, p31] divide.Set p20=20000 herein, p21=30000; P30=5000, p31=8000.Domestic data are divided according to the standard that average 20000 IP are no more than 30000 IP at most, and the IP of Pekinese divides according to the standard that average 5000 IP are no more than 8000 IP at most, and external IP section is then no longer done segmentation.
The practical application in example 2 somewheres:
Adopt three IP scopes to divide: world IP scope, the IP scope of China and the IP scope in somewhere, so n=3.The scope of China IP is 371 discontinuous section, and the IP scope of certain province is 27 discontinuous section.IP quantitative criteria amount and maximum that the setting section post comprises are [20000,30000; 5000,8000].
Respectively take out an IP section from regional and domestic two-stage below and carry out partition description, one of them IP section of China is 124.220.0.0 ~ 124.240.191.155, and one of them IP section in somewhere is 124.226.0.0 ~ 124.227.255.255.
The subregion step is as follows in detail:
1. the IP section 124.226.0.0 ~ 124.227.255.255 in area is divided into each segment according to the standard of [5000,8000], as shown in table 1:
| Sequence number | IP From | By IP | The IP quantity that this section comprises |
| 1 | 124.226.0.0 | 124.226.19.135 | 5000 |
| 2 | 124.226.19.136 | 124.226.39.15 | 5000 |
| 3 | 124.226.39.16 | … | 5000 |
| … | … | 5000 | |
| 26 | 124.227.232.72 | 124.227.251.207 | 5000 |
| 27 | 124.227.251.208 | 124.227.255.255 | 1072 |
The partition table of table 1 IP section 124.226.0.0 ~ 124.227.255.255
Can be seen that by table 1 according to the standard of [5000,8000], this IP section evenly has been divided into 26 segments, the IP quantity that each section comprises is 5000.Because the 27th section is had only 1072 IP, the standard number much smaller than 5000,1072+5000=6072, the maximum less than 8000 allows IP quantity.Therefore, finally this IP section has been divided into 26 sections, and preceding 25 all comprise 25 IP, last section, and promptly the 26th section comprises 6072 IP.
1. the IP section of China of 124.220.0.0 ~ 124.240.191.155 is divided according to the standard of [20000,30000].Because the IP of the first step is included within this segment limit, therefore, in fact this IP section has been cut into two discontinuous IP sections by the first step, is 124.220.0.0 ~ 124.225.255.255 and 124.228.0.0 ~ 124.240.191.155.As shown in table 2:
The partition table of table 2 IP section 124.220.0.0 ~ 124.240.191.155
Can be seen that by table 2 finally this big IP section has been split into 62 segments, except the 20th section and the 62nd section, remaining each section all comprises IP20000.And if these two special segments merge itself and a last section of closing on, just surpassed the constraint of maximum no more than 30000 IP, therefore, it is not merged, become two independently sections separately.
2. the IP section in each different range is divided according to method of the present invention, 3841 subregions of final acquisition, each subregion or comprise 5000 IP of this area, maximum is no more than 8000, perhaps comprise 20000 domestic other regional IP, maximum is no more than 30000, perhaps comprises a complete external IP section, and quantity without limits.
Adopt partition method of the present invention, certain area in practical application, final this table can logging data subregion reached 2135, the data coverage rate of subregion (being the percentage that the subregion number that has data accounts for the subregion sum) has brought up to present 55.6% by former 4.8%.And reach in 300,000 in this table data volume, carrying out the time that 30,000 batch data upgrades only needed about 20 minutes, created the time of table compared to existing technology, had shortened 33% at least, had improved speed.
Experiment showed, that technical scheme of the present invention can realize each subregion that is distributed in tables of data with data balancing preferably, optimize the performance and the storage organization of data loading.
Although disclose specific embodiments of the invention and accompanying drawing for the purpose of illustration, its purpose is to help to understand content of the present invention and implement according to this, but it will be appreciated by those skilled in the art that: without departing from the spirit and scope of the invention and the appended claims, various replacements, variation and modification all are possible.The present invention should not be limited to this specification most preferred embodiment and the disclosed content of accompanying drawing, and the scope of protection of present invention is as the criterion with the scope that claims define.
Claims (5)
1. IP partition method comprises step:
1) successively divide world IP scope, obtain various level IP range information and corresponding number of levels n, n is the integer greater than 2;
2) determine to want the IP number of levels m in subregion area, set partitioned parameters pi0 according to the IP quantity in i layer area; M is the integer in [1, n], and i is the integer in [(n-m+1), n];
3) be the IP in the even five equilibrium of standard i layer area with pi0, finish the IP subregion.
2. a kind of IP partition method as claimed in claim 1, it is characterized in that: set the i layer and divide section to allow the partitioned parameters pi1 of IP at most, last the sectional IP quantity of dividing determining step 3) adds that whether pi0 is less than pi1, if less than, then itself and previous minute section are merged into a branch section, otherwise nonjoinder.
3. a kind of IP partition method as claimed in claim 1 is characterized in that: m in the step 3)＞2 o'clock, the order of carrying out the IP subregion is: n layer IP scope, n-1 layer IP scope is up to (n-m+1) layer IP scope.
4. a kind of IP partition method as claimed in claim 1 is characterized in that: the pi0 numerical value of setting is ten thousand grades.
5. a kind of IP partition method as claimed in claim 2 is characterized in that: the pi1 numerical value of setting is no more than 50% of pi0.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNA2009100829324A CN101572625A (en) | 2009-04-24 | 2009-04-24 | IP partition method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNA2009100829324A CN101572625A (en) | 2009-04-24 | 2009-04-24 | IP partition method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN101572625A true CN101572625A (en) | 2009-11-04 |
Family
ID=41231864
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CNA2009100829324A Pending CN101572625A (en) | 2009-04-24 | 2009-04-24 | IP partition method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101572625A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103207835A (en) * | 2013-04-15 | 2013-07-17 | 东莞市公安局 | Mass data storage method through self-adaptive Range partitions |
| CN107451154A (en) * | 2016-05-31 | 2017-12-08 | 华为技术有限公司 | A kind of processing method of tables of data, apparatus and system |
| CN113132499A (en) * | 2019-12-30 | 2021-07-16 | 中国移动通信集团山西有限公司 | IP address information management method, device, equipment and computer storage medium |
-
2009
- 2009-04-24 CN CNA2009100829324A patent/CN101572625A/en active Pending
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103207835A (en) * | 2013-04-15 | 2013-07-17 | 东莞市公安局 | Mass data storage method through self-adaptive Range partitions |
| CN107451154A (en) * | 2016-05-31 | 2017-12-08 | 华为技术有限公司 | A kind of processing method of tables of data, apparatus and system |
| CN107451154B (en) * | 2016-05-31 | 2021-03-30 | 华为技术有限公司 | Data table processing method, device and system |
| CN113132499A (en) * | 2019-12-30 | 2021-07-16 | 中国移动通信集团山西有限公司 | IP address information management method, device, equipment and computer storage medium |
| CN113132499B (en) * | 2019-12-30 | 2022-10-18 | 中国移动通信集团山西有限公司 | IP address information management method, device, equipment and computer storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102214228B (en) | Large-grid distributed real-time database system and data management method thereof | |
| Cirtautas | Urban sprawl of major cities in the Baltic States | |
| CN109544316B (en) | Method and system for urging collection of real-time case division according to proportion | |
| CN102419752B (en) | Industrial database message storage method | |
| CN107679684A (en) | Declaration form distribution method, device, storage medium and computer equipment | |
| CN107566531A (en) | A kind of Elasticsearch cluster expansion methods for supporting balanced resource | |
| CN102930062A (en) | Rapid horizontal extending method for databases | |
| CN105373480A (en) | ABTest system shunt optimizing method and system | |
| CN1963788A (en) | A managing method for EMS memory | |
| CN102156717B (en) | Method and device for mapping entity object into database | |
| CN104111924A (en) | Database system | |
| CN106528787A (en) | Mass data multi-dimensional analysis-based query method and device | |
| CN102737123B (en) | A kind of multidimensional data distribution method | |
| CN106095589A (en) | Partition allocation method, device and system | |
| CN106919590A (en) | A kind of multi-business multiplexing processing method, device and system | |
| CN106603729A (en) | Distributed-file-system multi-client synchronization method and system thereof | |
| Xing et al. | China-US rivalry: a new Cold War or capitalism’s intra-core competition? | |
| CN101572625A (en) | IP partition method | |
| CN103488644A (en) | Method for carrying out data storage and database system | |
| CN104461736B (en) | Resource allocation and searching method, resource allocation and search system and Cloud Server | |
| CN103207835A (en) | Mass data storage method through self-adaptive Range partitions | |
| Gan et al. | Relocating or redefined: A new perspective on urbanization in China | |
| CN107330723A (en) | A kind of method and device for distributing pre-determined advertisement flow | |
| CN110008289A (en) | A kind of relational database, grid model data storage and search method | |
| CN101217690A (en) | A system and method for realizing distributed storage of data operation and data information |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C12 | Rejection of a patent application after its publication | ||
| RJ01 | Rejection of invention patent application after publication |
Open date: 20091104 |