CN103345491B - A kind of method applying Hash Hash division bucket quickly to obtain neighborhood - Google Patents

Abstract

The invention discloses a kind of quick neighborhood computational methods applying Hash division bucket to reduce neighborhood information particle search space.The method, according to the distance between sample record, utilizes the method for Hash that the set of sample record is divided into bucket, and in set, the search volume of the neighborhood information particle of any one sample record will be narrowed in adjacent three bucket.On this basis, deeply it has been observed that when the neighborhood space searching for sample record uses alternative manner, according to the symmetry principle of neighborhood system, the search volume of neighborhood information particle can be narrowed down in the range of two buckets further.The inventive method realizes the simplest, greatly reduces number of comparisons and the search volume of sample, and the process to processing big data possesses advantage.

Description

A kind of method applying Hash Hash division bucket quickly to obtain neighborhood

Technical field

The invention belongs to field of information processing, particularly relate to a kind of with two norm distances for tolerance, use Hash (Hash) divide bucket and reduce the method quickly obtaining neighborhood in neighborhood information particle search space.

Background technology

Along with developing rapidly and the extensive application of data base management system, the data of people's record of information technology More and more.The most much important information of data increased sharply, it is desirable to carry out higher to it The analysis of level, in order to better profit from these data.

T.Y.Lin proposed the concept of neighbourhood model in 1988, and he realizes domain by using spatial neighborhood Granulation, and spatial neighborhood is interpreted as basic information particle, then utilizes these essential information particles to retouch Other concepts in review territory.In 1998, Yao Yiyu taught and professor Wu Weizhi in 2002 to neighborhood The fundamental system character of operator and neighborhood system conducts in-depth research respectively.Yao discusses Granule Computing with thick Relation between the Data Mining Tools such as rough collection, the quotient space, and by using logic decision making language to describe grain Degree, builds the logical framework of granular world.Skowron also illustrates a language in the literature, and he is by information Information regarded as by the meaning collection of the logical formula defined on table, and discusses the syntax and semantics of this information. On the basis of these are studied, neighbourhood model is incorporated in rough set by Hu Qinghua, enters neighborhood rough set model Go detailed definition, and design can yojan name type, numeric type, the Algorithm for Reduction of mixed type data simultaneously.

Along with the explosion type of data increases, when using neighbourhood model to process big data, time efficiency just becomes The factor of overriding concern.How to reduce search and calculate neighborhood information particle time, be one considerable Problem.

Neighborhood has two kinds of definition methods: a kind of be the quantity by object contained in neighborhood depending on, such as classical k- Near neighbor method；Another kind is that the ultimate range according to centre of neighbourhood point in a certain tolerance to border is defined. Neighborhood involved in the present invention is the 2nd kind of method.

Nonempty finite set U={x on real number space₁, x₂, x₃..., x_n, for any object on U x_i, its θ neighborhood is θ (x_i)={ x ∈ U, Δ (x, x_i)≤θ }, wherein, θ >=0, θ (x_i) be referred to as by x_iThe θ neighborhood information particle generated, is called for short x_iNeighborhood particle, with regard to two dimension real number space for, based on 1 model Number, 2 norms and Infinite Norm neighborhood as it is shown on figure 3, respectively rhombus, circle and square area.Degree The character of amount has: (1)Because x_i∈θ(x_i);(2)x_j∈θ(x_i)→x_i∈θ(x_j); (3)Neighborhood information particle race { θ (x_i) | i=1,2 ... n} constitutes a covering of U.

Neighborhood relationships N that neighborhood information particle race is guided out on the U of domain space, this relation can be by a pass It is that matrix is to represent M (N)=(r_ij)_n×nIf, x_j∈θ(x_i), then r_ij=1 otherwise r_ij=0。

Summary of the invention

It is an object of the invention to a kind of method applying Hash Hash division bucket quickly to obtain neighborhood, to reduce Search and the time of calculating neighborhood information particle, it is achieved use neighbourhood model to process the rapidity of big data.To this end, The present invention is by the following technical solutions:

Specifically comprising the following steps that of the inventive method

A kind of method applying Hash Hash division bucket quickly to obtain neighborhood, it is characterised in that it includes walking as follows Rapid:

Step one, seeks the zero x of point bucket coordinate system₀,

According to given neighborhood system NRS=<U, N, θ>, U is the set that whole sample record is constituted, N Representing neighborhood relationships, θ is the radius of neighbourhood；

Step 2, seeks the distance of sample,

ForSeek the distance between sample | | x_i-x₀||;

Step 3, according to the sample distance in step 2, sets up by Hash method and searches for bucket:

For ${\&ForAll;x}_i\&Element;U,i=\mathrm{1,2},\&CenterDot;\&CenterDot;\&CenterDot;,n,$ CalculateK is nonnegative integer. Hash table is set up as hash Key using k.Set up hash table Sphere Measurement Model in space: by hash Key As the radius of sphere, hash table is exactly a series of mutually nested balls in space；Under a certain hash Key Sample, is in the space between the sphere with hash Key as radius and the hashKey-1 sphere as radius, Space between adjacent sphere is referred to as described bucket, B₁, B₂..., B_b, for by b hash Key value as radius gained B bucket.

Step 4, it is thus achieved that neighborhood:

Bucket B_k-1, B_k, B_k+1Interior record, it is thus achieved that the neighborhood of sample x.

Using on the basis of technique scheme, the present invention also can use technical scheme further below:

x₀Take initial point or in N minima composition a characteristic vector.

When searching for sample record acquisition neighborhood in bucket, use alternative manner, it is only necessary to search B_k, B_k+1The neighborhood obtaining sample x sought in record in Tong.

The present invention, according to the distance between sample record, utilizes the method for Hash the set of sample record to be divided into Bucket, any one sample record x in set_iThe search volume of neighborhood information particle will be narrowed to adjacent three Individual bucket B_k-1, B_k, B_k+1In.On this basis, deeply it has been observed that the neighborhood working as search sample record is empty Between when using alternative manner, according to the symmetry principle of neighborhood system, can searching neighborhood information particle Rope space narrows down to two bucket B further_k, E_k+1In the range of.

The method of the present invention can obtain different point bucket effects according to size θ of different neighborhoods.Along with θ value Becoming big, the quantity of bucket will reduce, but the quantity of sample record contained in each barrel can increase, when a point bucket In the case of being continuous print, when calculating neighborhood information particle, the amplitude of the reduced space of search will reduce, point bucket institute The effect brought will weaken, and divided bucket is in discrete when, owing to adjacent bucket may not be there is, At this moment search volume will significantly reduce, the effect reducing neighborhood information particle search space that point bucket is brought Fruit strengthens.Along with θ value diminishes, the quantity of bucket will become and is gradually increased, and sample size contained in bucket will subtract Few, being so likely to result in the quantity of information included in each neighborhood information particle will tail off.

According to the above, according to the different situations of neighborhood system NRS=<U, N, θ>, suitable by selecting The θ value closed, gives full play to the effect reducing neighborhood information particle search space that point bucket is brought, and minimizing is searched Rope and the time of calculating neighborhood information particle.

Accompanying drawing explanation

Fig. 1 be point bucket in the inventive method be continuously in the case of schematic diagram.

Fig. 2 be point bucket in the inventive method be the schematic diagram under discrete case.

In Fig. 3 is two dimension real number space, based on 1 norm, 2 norms and the neighborhood of Infinite Norm.

Detailed description of the invention

In order to be better understood from technical scheme, it is further described below in conjunction with drawings and Examples.

Step one, seeks the zero x of point bucket coordinate system₀:

According to given neighborhood system NRS=<U, N, θ>, seek the zero x of point bucket coordinate system₀.Take N=CUD, then neighborhood system NRS=<U, N, θ>becomes neighborhood decision system NRS=<U, C ∪ D, θ>, x₀Take minimum property value one characteristic vector of composition of each attribute.Sample Set U={x₁,x₂,x₃..., x_n, the set C={a of sample attribute₁, a₂..., a_m), the i-th of sample Individual attribute a_i, sample decision attribute set D,

Then base is

x₀=(min{x_l(a₁), x₂(a₁) ... x_n(a₁),

min{x₁(a₂), x₂(a₂) ... x_n(a₂) ..., min{x₁(a₁), x₂(a₁) ... x_n(a₁)})

Step 2, seeks the distance of sample:

ForSeek the distance between sample | | x_i-x₀||;

$\left|\right|x_i-x_0\left|\right|=\sqrt{{[x_i\left(a_1\right)-x_0\left(a_1\right)]}^2+{[x_i\left(a_2\right)-x_0\left(a_2\right)]}^2+\&CenterDot;\&CenterDot;\&CenterDot;{[x_i\left(a_m\right)-x_0\left(a_m\right)]}^2}$

Step 3, according to the distance in step 2, Hash method sets up bucket:

For $\&ForAll;x_i\&Element;U,i=\mathrm{1,2},\&CenterDot;\&CenterDot;\&CenterDot;,n,$ CalculateK is nonnegative integer, with K sets up hash table as hash Key.Using hash Key as the radius of sphere, hash table in space just It is a series of mutually nested balls.Sample under a certain hash Key, is in the ball with hash Key as radius In space between face and the sphere that hashKey-1 is radius, the space between adjacent sphere is referred to as described bucket. B₁, B₂..., B_bFor by b hash Key value as b bucket of radius gained.Then the part of records in U exists As shown in Figure 1 and Figure 2, Fig. 1 gives a kind of more special situation in the distribution in space, i.e. k is continuous print； And it practice, for specific data, k is probably discontinuous, i.e. shown in Fig. 2.

Step 4, it is thus achieved that neighborhood:

Search bucket B_k-1, B_k, B_k+1Interior record, seeks its neighborhood.When sample note in search bucket When record solves neighborhood, use alternative manner, due to the symmetry of neighborhood system, it is only necessary to search B_k, B_k+1Record in Tong solves its neighborhood.As shown in fig. 1, to the sample in the bucket of k=2, The when of solving neighborhood, can only search for the sample in k=2 Yu k=3 bucket.As in figure 2 it is shown, at bucket not In the case of continuous print, for the sample in k=2 bucket, owing to being empty, in its bucket in the k=3 bucket that it is adjacent The solution room of sample neighborhood reduces.

It is O (n), n=| U | that application hash sets up the time complexity of bucket.Sample attribute set C={a₁, a₂..., a_mThe number of the sample attribute in } is m, and the number of point bucket of foundation is b, at sample In the case of being evenly distributed in each bucket, the complexity that neighborhood calculates isAt b convergence In the case of | u |, the complexity that neighborhood calculates will level off to O (m | U |).

Claims (3)

1. the application Hash Hash processed for big data divides the method that bucket quickly obtains neighborhood, and it is special Levy and be that it comprises the steps:

Step one, seeks the zero x of point bucket coordinate system₀,

In neighborhood system NRS=<U, N, θ>, U is the set that whole sample record is constituted, and N represents neighborhood Relation, θ is the radius of neighbourhood；

Step 2, seeks the distance of sample,

ForN is number of samples, seeks the distance between sample | | x_i-x₀||；

Step 3, according to the sample distance in step 2, sets up by Hash method and searches for bucket:

ForCalculateK is nonnegative integer, Hash table is set up as hash Key using k；Set up hash table Sphere Measurement Model in space: by hash Key As the radius of sphere, hash table is exactly a series of mutually nested balls in space；Under a certain hash Key Sample, is in the space between the sphere with hash Key as radius and the hashKey-1 sphere as radius, Space between adjacent sphere is referred to as bucket, B₁, B₂..., B_b, for by b hash Key value as the b of radius gained Individual bucket；

Step 4, it is thus achieved that neighborhood:

Search bucket B_k-1,B_k,B_k+1In record obtain sample x neighborhood.

A kind of application Hash Hash processed for big data divides bucket and quickly obtains The method obtaining neighborhood, it is characterised in that x₀Take initial point or in N minima composition a characteristic vector.

A kind of application Hash Hash processed for big data divides bucket and quickly obtains The method obtaining neighborhood, it is characterised in that when searching for sample record acquisition neighborhood in bucket, use alternative manner, Have only to search for B_k, B_k+1Record in Tong obtains the neighborhood of sample x.