KR102006283B1 - Dataset loading method in m-tree using fastmap - Google Patents

Abstract

The present invention relates to an M-tree loading method of a dataset using a fast map, which comprises the steps of: mapping a dataset on a metric space to a point on a k-dimensional Euclidean space; aligning the points on the k-dimensional Euclidean space into a one-dimensional sequence; dividing the points of the one-dimensional sequence into contiguous groups; and generating a leaf node for the contiguous group. The dataset is bulk-loaded to the M-tree. Accordingly, a large dataset having the similarity between data objects which is not defined by an Lp distance between two points in the Euclidean space can be efficiently loaded in the M-tree, so that indexing and searching efficiency is more improved than the performance of the existing M-tree.

Description

M-tree loading method of dataset using fast map {DATASET LOADING METHOD IN M-TREE USING FASTMAP}

The present invention relates to a method of loading an M-tree of a data set using a fast map, and a technique of loading a large amount of database objects into an M-tree using a fast map.

The M-tree is an index structure for efficient search for objects in metric space where the distance between any two objects is defined. Unlike the R-trees available only in Euclidean space, M-trees are Euclidean space as well as metric space. There is an advantage that can be used in.

In particular, research has demonstrated that M-trees perform better than R-trees for the same dataset.

In addition, a bulk loading algorithm has been proposed for uploading a large amount of data objects in order to improve the limitations of cost consumption and inefficiency by sequentially inserting a large number of data objects into an M-tree index.

On the other hand, similarities between data objects are similar between two points in Euclidean space, such as unstructured multimedia data with large capacity and irregular size such as unstructured multimedia data such as voice, image, text, or point of interest on road network. It is necessary to develop an algorithm for uploading data that is difficult to define by Lp distance to M-tree having excellent data retrieval performance.

Republic of Korea Patent Publication No. 10-2012-0096894 (database search method, navigation device and index structure generation method)

An object of the present invention is to propose a method of uploading a large dataset, which is difficult to define the Lp distance between two points in Euclidean space, to a M-tree with high search efficiency. To improve my index creation and search performance.

The M-tree loading method of a dataset using a fast map according to an embodiment of the present invention is performed in a processing apparatus that loads a data object into an M-tree, and k-dimensional Euclidean dataset on a metric space. Mapping to a point in space; aligning the points on the k-dimensional Euclidean space into a one-dimensional sequence; Dividing the points of the one-dimensional sequence into contiguous groups; Creating a leaf node for the contiguous group; It characterized in that to load the data set, including the M-tree.

The data set on the metric space is a data set in which the similarity between data objects is not defined as the Lp distance between two points in Euclidean space.

The dataset on the metric space is characterized in that it is a point of interest (POI) of unstructured multimedia data or road network.

Mapping the dataset on the metric space to a point on a k-dimensional Euclidean space is performed such that the inter-object spacing of data on the metric space is mapped so as to remain on the k-dimensional Euclidean space. It is characterized by.

The mapping of the data set on the metric space to a point on the k-dimensional Euclidean space is performed from Equation 1 below.

[Equation 1]

는 유클리디안 스페이스로 매핑된 두 객체

간의 거리이고, i 및 j는 1부터 N까지의 자연수임.)(d _ij is the distance between two objects O _i , O _j in the metric space,

Are two objects mapped to Euclidean space

Distance, and i and j are natural numbers from 1 to N.)

The mapping of the dataset on the metric space to a point on a k-dimensional Euclidean space is performed to map the plurality of objects constituting the dataset to BulkLoading on the Euclidean space. .

Arranging the points on the k-dimensional Euclidean space in a one-dimensional sequence is performed using a space filling curve.

Arranging the points on the k-dimensional Euclidean space in a one-dimensional sequence is performed using any one of Z curve (Morton curve), Hilbert curve, and Gray-code curve as a space filling curve. .

The dividing the points of the one-dimensional sequence into successive groups may be performed by dividing the consecutive points arranged in the one-dimensional sequence into groups, but grouping all the groups uniformly so as to include the maximum objects. It is characterized in that performed using.

In the step of dividing the points of the one-dimensional sequence into a contiguous group, the consecutive points arranged in the one-dimensional sequence are sequentially divided into a group, the distance between the points included in the group and the number of points included in the group By using the heuristic method to calculate the number of points to be included in the group and to configure the group to the extent that the size of the area of the group does not extend by using the calculation result.

In the heuristic method, the point-to-point distance included in the group may be calculated as the point-to-point distance in the actual metric space, or may be calculated by substituting the distance between two points in the k-dimensional Euclidean space.

The dividing the points of the one-dimensional sequence into consecutive groups may be performed by sequentially dividing the consecutive points arranged in the one-dimensional sequence into groups, generating candidate groups, and a radius of an area size included in each candidate group. And calculating the number of points included in the candidate group and using the calculation result to configure the group.

In the resource method, the radius of the candidate group region may be calculated as the radius on the actual metric space or by replacing the distance between two points in the k-dimensional Euclidean space.

The step of creating a leaf node for the continuous group may include creating a leaf node including an object on a metric space corresponding to the points included in the divided group in the step of dividing the points of the one-dimensional sequence into successive groups. Characterized in that it is carried out.

Generating leaf nodes for the consecutive groups is characterized in that it is performed to calculate representative objects of the leaf nodes using points on the k-dimensional Euclidean space.

After generating leaf nodes for the contiguous groups, generating non-leaf entries for leaf nodes created for each group; and according to the number of non-leaf entries, Dividing the leaf entries into groups of entries, and storing the entire entries in one root node; It is characterized in that any one of the steps further performed.

Dividing the non-leaf entry into an entry group may be performed by a full page method, a heuristic method, or a resource method.

The entry generated in the step of dividing the non-leaf entry into an entry group is performed to be stored in a buffer, and when the buffer is full, the entire entry is stored in one root node to store the entry in a memory disk. It is characterized by recording.

A computer-readable recording medium according to an embodiment of the present invention is characterized by performing an M-tree stacking method of a data set using the fast map.

According to this aspect, the M-tree stacking method of a data set using a fast map according to an embodiment of the present invention has a structure for loading a large data set into the M-tree using a fast map and a space filling curve In addition, a dataset whose similarity between data objects is not defined by the Lp distance between two points in Euclidean space can be efficiently loaded into the M-tree in a large amount, and its indexing and retrieval efficiency is searched in the existing M-tree. This has the effect of improving performance.

1 is a flowchart illustrating a flow of an M-tree loading method of a data set using a fast map according to an embodiment of the present invention.
2 is a diagram illustrating various embodiments of a process of dividing a one-dimensional sequence into groups in an M-tree loading method flow of a data set using a fast map according to an embodiment of the present invention.
3 is a flowchart illustrating a process of dividing a one-dimensional sequence into groups using a heuristic method in the flow of an M-tree loading method of a dataset using a fast map according to an embodiment of the present invention.
4 is a flowchart illustrating a process of dividing a one-dimensional sequence into groups using a resource method in the M-tree loading method flow of a data set using a fast map according to an embodiment of the present invention.
5 is a flowchart illustrating a process of creating a leaf node for a group in an M-tree loading method flow of a dataset using a fast map according to an embodiment of the present invention.
FIG. 6 is a diagram illustrating an embodiment of a process of creating a leaf node for a group in an M-tree loading method flow of a data set using a fast map according to an embodiment of the present invention.
7 is a diagram illustrating an embodiment of a non-leaf entry in a process of generating a non-leaf entry for a leaf node in an M-tree loading method flow of a dataset using a fast map according to an embodiment of the present invention. .
8 is a graph comparing performance of data sequential insertion and data retrieval speed according to an M-tree loading method of a data set using a fast map according to an embodiment of the present invention.

DETAILED DESCRIPTION Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art may easily implement the present invention. As those skilled in the art would realize, the described embodiments may be modified in various different ways, all without departing from the spirit or scope of the present invention. In the drawings, parts irrelevant to the description are omitted in order to clearly describe the present invention, and like reference numerals designate like parts throughout the specification.

1 is a flowchart illustrating a method of loading an M-tree of a data set using a fast map according to an embodiment of the present invention, and FIG. 2 is an M- of a data set using a fast map according to an embodiment of the present invention. 3 is a view illustrating various embodiments of a process of dividing a one-dimensional sequence into groups in a tree loading method flow, and FIG. 3 is a one-dimensional view of an M-tree loading method flow of a dataset using a fast map according to an embodiment of the present invention. 4 is a flowchart illustrating a process of dividing a sequence into groups using a heuristic method, and FIG. 4 illustrates a method of storing a 1-dimensional sequence in a M-tree loading method flow of a data set using a fast map according to an embodiment of the present invention. FIG. 5 is a flowchart illustrating a process of dividing the data into groups using the fast map. FIG. FIG. 6 is a flowchart illustrating a process of generating a new node, and FIG. 6 illustrates an embodiment of a process of generating a leaf node for a group in an M-tree loading method flow of a dataset using a fast map according to an embodiment of the present invention. 7 is a view illustrating an embodiment of a non-leaf entry in a process of generating a non-leaf entry for a leaf node in an M-tree loading method flow of a dataset using a fast map according to an embodiment of the present invention. 8 is a graph comparing data sequential insertion with performance of data retrieval speed according to an M-tree loading method of a data set using a fast map according to an embodiment of the present invention.

In the M-tree loading method of a dataset using a fast map according to an embodiment of the present invention, similarities between objects, such as unstructured multimedia data or POI of road networks, do not define the Lp distance between two points in Euclidean space. A method of loading a set into an M-tree, which is a database structure, may be applied to a data set in which similarity between objects is defined as an Lp distance between two points in Euclidean space.

In an embodiment of the present invention, the Lp distance between two points in Euclidean space will be described based on a method of loading an undefined data set into an M-tree.

In addition, the M-tree loading method of a data set using a fast map according to an embodiment of the present invention may be performed in a computing device including a database, a server, or a network including the same, but is not limited thereto.

In the following specification, an algorithm according to the present embodiment is performed by a processing apparatus for loading data objects into an M-tree in a computer apparatus, a server, or a network, that is, a processor, that is, storing an algorithm according to the present embodiment. The description will be made based on what is a computer-readable recording medium, and does not limit the subject on which the algorithm is performed.

In addition, the processing device for performing the algorithm according to the present embodiment may be connected to a separate storage device to refer to the data stored in the storage device or to store the processing result in the storage device. It should not be limited.

In the present embodiment, the M-tree loading method of a dataset using a fast map is a step of mapping the dataset to a point on a k-dimensional Euclidean space (S100), by arranging the k-dimensional points in one dimension. Generating a sequence (S200), dividing a one-dimensional sequence to generate a continuous group (S300), generating a leaf node for each group (S400), non for the leaf node generated for each group generating a leaf entry (S500), comparing the number of non-leaf entries with a setting value M (Q100), and performing a non-leaf entry as an entry group when the number of non-leaf entries is larger than the setting value M. Splitting (S610), generating a non-leaf node and non-leaf entry for an entry group (S620), and a whole root to be performed when the number of non-leaf entries is smaller than the set value M Storing in the node (S700). Achieved.

First, in the step of mapping the dataset to a point on the k-dimensional Euclidean space (S100), each object in the dataset D on the metric space is a Euclidean space of the k-dimensional space. Map to keep the relative distance between objects in the image as much as possible.

In this step (S100), k is an arbitrary natural number of one or more, and in this embodiment, the present invention will be described as an example of mapping objects of a dataset on a metric space onto a one-dimensional Euclidean space, but the present invention is limited. It should not be.

In this case, the data set D on the metric space includes an object Oj (j is 1 to N and N is a natural number).

In this step (S100), the objects of the dataset are mapped onto the Euclidean space through the FastMap technique. Instead of uploading the objects included in the dataset one by one, a plurality of objects are simultaneously placed on the Euclidean space. Mapping, that is, bulkloading.

In one example, in mapping objects of a dataset on the metric space onto Euclidean space via FastMap, to map the distance between objects of the dataset on the metric space to be maintained in Euclidean space, The objects satisfy the stress function in Equation 1 below, but map to the k-dimensional Euclidean space to minimize the stress value.

[Equation 1]

는 유클리디안 스페이스로 매핑된 두 객체

간의 거리이고, i 및 j는 1부터 N까지의 자연수이다.In Equation 1 above, d _ij is the distance between two objects O _i , O _j in the metric space,

Are two objects mapped to Euclidean space

Distance, and i and j are natural numbers from 1 to N.

Next, in the step of generating a one-dimensional sequence by sorting the k-dimensional points (S200), the points mapped to the k-dimensional Euclidean space from the above step (S100) are arranged in a one-dimensional sequence.

At this time, the points of the k-dimensional Euclidean space are arranged in a one-dimensional sequence using a space-filling curve.

)들을 순차적으로 방문하고, 방문 순서에 따라 각 객체의 순번(order)을 부여하여, 순번에 따라 1차원 시퀀스로 정렬한다.In one example, the space fill curve is an object (i.e., all points mapped on a grid of Euclidean space).

) Are sequentially visited, and the order of each object is assigned according to the visit order, and the objects are arranged in a one-dimensional sequence according to the order.

As the space filling curve, Z curve (Morton curve), Hilbert curve, and Gray-code curve may be used. However, in this embodiment of FastLoading a data object, objects of k-dimensional Euclidean space are one-dimensional by using Hilbert curve. Sort by sequence.

The one-dimensional sequence generated in this step S200 allows the data objects mapped to two points to be adjacent on the k-dimensional Euclidean space to be aligned in the order of the adjacent one on the one-dimensional sequence.

Next, in the step of generating a continuous group by dividing the one-dimensional sequence (S300), the points arranged in order in the one-dimensional sequence generated in the step S200 are divided into groups.

In this case, the groups divided in the one-dimensional sequence include at least one continuous point arranged in the one-dimensional sequence, preferably a plurality of points, and each group generated by dividing from the one-dimensional sequence receives a group number. Can be.

In one example, if the first group is created to contain the first point, the second point, arranged in a one-dimensional sequence, the second group, which is a group that is contiguous to the first group, starts from the third point, arranged in the one-dimensional sequence. It is generated to include.

This step (S300) may be made such that the group divided from the one-dimensional sequence includes μM to M consecutive points arranged in the one-dimensional sequence, where μ (0.0 <μ≤1.0) is the minimum space utilization ratio. (minimum storage utilization), M is the maximum number of objects in one M-tree terminal node, or the maximum number of entries in the M-tree non-terminal node.

In one example, this step (S300) scans consecutively from the first point of the consecutive points arranged in the one-dimensional sequence to include in the group, and scans from the continuously aligned points after the points already included in the group It is repeatedly performed to include in a new group, and the operation of scanning and dividing the points of the sequence into groups is repeated until the number of points aligned in the sequence is less than or equal to μM points.

This step (S300) may be performed using any one of a full page method, a heuristic method, and a rigorous method.

First, an example of performing the step S300 of dividing the points of the one-dimensional sequence into groups using the full-page method will be described. In the full-page method, M points uniformly aligned in the one-dimensional sequence are uniformly included in the group. In this method, the one-dimensional sequence is divided into groups such that all groups generated from the one-dimensional sequence include M points.

When the one-dimensional sequence is divided into groups using the full page method, space utilization of the M-tree can be maximized.

However, using the full-page method, an example of dividing the points arranged in a one-dimensional sequence into a group containing M points is simply performed. Two consecutive points in the one-dimensional sequence are k-dimensional Euclidean space or actual. When two dataset objects corresponding to two points are located apart in a metric space, and when a group is created that includes two objects apart in a real space, the size of the region for the group is increased so that the search performance in the M-tree is increased. This greatly degraded problem may occur.

Referring to this in detail with reference to the embodiment of Fig. 2, objects continuously aligned in a one-dimensional sequence are arranged as shown in Fig. 2 (a) on the Hilbert curve in the actual k-dimensional Euclidean space, When objects from O ₁ to O _x that are continuously aligned in a one-dimensional sequence are divided into a group, a minimum area of a group having a circular shape is formed.

On the other hand, as shown in (b) of FIG. 2, when the objects O ₁ to O _{x + 1} arranged in the one-dimensional sequence are divided as a group, the O _x object and the O _{x + 1} object are one-dimensional sequence. In the continuous but actual Euclidean space in the two objects are located at a distance from each other, even though the structure includes only O _{x +1} objects more than the embodiment shown in Fig. 2 (a) of the group containing the objects The minimum area extends larger than the size of the minimum area formed in FIG. 2 (a), and the increase in the size of the region of the group decreases the object search performance efficiency as mentioned above.

Accordingly, the heuristic method and the resource method show that when the objects corresponding to the points included in the group divided in the one-dimensional sequence are distributed on the actual metric space or the Euclidean space, the size of the minimum area including these objects is the object. The number of points included in the group is limited so as to be formed in a size range that does not reduce the search efficiency.

An embodiment of dividing points of a one-dimensional sequence into groups using a heuristic method will be described with reference to FIG. 3. First, μM points ranging from the first point to a μM point among points aligned in the one-dimensional sequence are described. To create the first group (S210).

을 산출(S220)한다. 이때, d는 그룹에 포함되는 첫 번째 점과 그룹에 포함되는 마지막 점(μM번째 점) 간의 거리이고, n은 그룹에 포함된 점들의 개수이다.Then, by using the μM points included in the first group generated from the step (S210)

It is calculated (S220). In this case, d is the distance between the first point included in the group and the last point (μM th point) included in the group, n is the number of points included in the group.

을 산출(S230)한다. 이때, 그룹에 추가되는 점은 위 단계(S210)에서 생성된 그룹에 기 포함된 마지막 점인 μM번째 점에 대해 연속하는 점, 즉, 1차원 시퀀스에 정렬된 (μM+1)번째 점이고, d'는 그룹 내의 첫 번째 점과 이 단계(S230)에서 새로 추가된 점 간의 거리이며, n'은 이 단계(S230)로부터 점이 추가된 그룹 내 포함된 점들의 개수이다.Then, further include points in the group subsequent to the last point included in the group generated from the step (S210) and

To calculate (S230). In this case, the point added to the group is a continuous point with respect to the μM-th point which is the last point included in the group generated in the above step (S210), that is, the (μM + 1) -th point arranged in the one-dimensional sequence, and d ' Is the distance between the first point in the group and the newly added point in this step S230, and n 'is the number of points included in the group to which the point was added from this step S230.

이

보다 큰 경우, 예 화살표 방향을 따라 이동하여 (n'-1)개의 점으로 그룹을 생성(S241)하는 단계를 수행하고,

이

보다 크지 않은 경우, 아니오 화살표 방향을 따라 이동하여 S230단계를 수행(S242)하도록 한다.In the above comparison step (Q200)

this

If larger, the step of moving along the direction of the example arrow (n'-1) to create a group (S241),

this

If it is not greater than, move in the direction of the arrow No to perform step S230 (S242).

,

), 즉, 그룹의 영역 크기를 비교하는 단계로서, 위 단계(S230)에서 점이 추가된 그룹의 영역 크기가 최초 단계(S210)에서 생성된 그룹의 영역 크기보다 증가했는지의 여부를 판단하는 과정을 수행한다.At this time, the comparison step (Q200) is a value obtained by dividing the maximum distance between the points in the group by the number of points (

,

In other words, as a step of comparing the area size of the group, determining whether the area size of the group to which the dot is added in step S230 is greater than the area size of the group generated in the first step S210. Perform.

이

보다 큰 경우, 위 단계(S230)에서 그룹에 점을 추가함에 따라 그룹의 영역 크기가 최초 단계(S210)에서 생성한 그룹의 영역 크기보다 증가한 것으로, 이와 같이 그룹의 영역 크기가 증가하는 경우 그룹 내 객체의 검색 효율이 저하될 수 있으므로, 이를 방지하기 위해, 최초 단계(S210)에서 생성한 그룹의 영역 크기가 확장되지 않는 범위를 유지하는 점들의 개수로 그룹을 구성하도록 한다. 따라서, 위 비교단계(Q200)로부터 그룹에 추가된 점에 의해 그룹 영역 크기가 확장된 것으로 판단되는 경우, 그룹에 마지막으로 추가된 n'번째 점을 포함하지 않도록, 즉, 그룹이 n'개의 점보다 1개 작은 (n'-1)개의 점을 포함하도록 그룹을 생성(S241)한다.In one example,

this

If larger, the area size of the group increases as the area size of the group is created in the first step (S210) as the point is added to the group in the above step (S230). Since the search efficiency of the object may be deteriorated, in order to prevent this, the group is composed of the number of points that maintain the range in which the area size of the group created in the first step S210 does not extend. Therefore, when it is determined that the size of the group region is extended by the points added to the group from the comparison step (Q200), the group does not include the n'th point added last, that is, the group has n 'points. A group is generated to include one smaller (n'-1) point (S241).

이

보다 크지 않은 경우, 즉, 그룹에 점을 추가하였으나 그룹의 영역 크기가 확장되지 않은 경우, 최초 단계(S210)에서 생성한 그룹에 점을 추가하는 단계(S230)를 반복 수행(S242)한다. 이 단계(S242)를 수행함에 따라, 그룹의 영역 크기가 확장되지 않는 범위에서 그룹에 포함되는 점의 개수를 최대한으로 갱신할 수 있고, 이에 따라, 그룹 내 객체 검색 성능을 저하시키지 않는 범위에서 1차원 시퀀스에 정렬된 점을 그룹 내에 최대로 포함시킬 수 있어, 효율적으로 1차원 시퀀스를 그룹으로 분할할 수 있게 된다.On the other hand, in the comparison step (Q200) above

this

If it is not larger, that is, if the point is added to the group but the area size of the group is not expanded, the step S230 of adding the point to the group created in the first step S210 is repeated (S242). By performing this step (S242), it is possible to update the number of points included in the group to the maximum as long as the area size of the group does not extend, and accordingly, 1 in a range that does not degrade the performance of searching for objects in the group. The points arranged in the dimensional sequence can be included in the group to the maximum, thereby efficiently dividing the one-dimensional sequence into groups.

In addition, an embodiment of dividing the points of the one-dimensional sequence into groups using the resource method will be described with reference to FIG. 4. First, μM to M consecutive points are respectively formed from the points arranged in the one-dimensional sequence. In operation S310, candidate groups that include the IDC are generated.

The candidate groups generated in this step (S310) do not continuously generate candidate groups of points aligned in a one-dimensional sequence, but generate continuous points from the first point to the μM-th point as the first candidate group. A method for generating points consecutive from a first point to an Mth point as a second candidate group, wherein points arranged in a one-dimensional sequence are generated as several candidate groups including different numbers of points.

을 각각 산출(S320)한다. 이때, r은 후보 그룹의 영역 크기의 반지름이고, n은 후보 그룹에 포함된 점의 개수이다.And, for each candidate group generated in the above step (S310)

Are respectively calculated (S320). Where r is the radius of the area size of the candidate group and n is the number of points included in the candidate group.

(S320)을 비교하여

값이 가장 작은 후보 그룹을 선택(S330)한다.Then, calculated for the candidate groups generated in the above step (S310)

Compare (S320)

The candidate group having the smallest value is selected (S330).

로 대체하는 방법 중 어느 하나를 이용할 수 있으며, 이를 한정하지는 않는다.In one example, to obtain the distance d between two points included in a group in the heuristic method or r, the radius of the group region in the resource method, to obtain the distance between the actual dataset objects on the metric space corresponding to the two points. And the distance between two points in k-dimensional Euclidean space

Any one of the methods may be used, but the present invention is not limited thereto.

In this case, when the distance between two points is obtained by using the actual distance between objects in the metric space with respect to two points, it is effective in that the distance between points can be accurately calculated. However, this method is used to calculate the radius of the group region through the resource method. In application, since the radius of the group region must be calculated for all the candidate groups generated from the points of the one-dimensional sequence, there may be a problem that it takes a long time to find the distance between objects in the actual metric space.

로 대체하는 방법을 적용하는 것을 기준으로 삼도록 한다.On the other hand, when the distance between two points is obtained by the method of replacing the distance of k-dimensional Euclidean space, the accuracy of the distance between two points is somewhat inaccurate than the method of calculating the distance between two points in the actual metric space, The advantage of saving time in calculating distances is that you can quickly find the radius of a group area. According to an embodiment of the present invention, in calculating the distance d between two points or the radius r of the candidate group region, the distance between two points on the k-dimensional Euclidean space

Use the alternative method as a guideline.

In this case, when the step (S300) of generating a group from the one-dimensional sequence is performed from the heuristic method referring to FIG. 3 or the reservoir method referring to FIG. 4, the one-dimensional method is performed using the method of FIG. 3 or 4. After generating a group of some of the points in the sequence, the method of FIG. 3 or 4 is repeatedly performed on the points on the continuous one-dimensional sequence after the points included last in the generated group. Split the points arranged in groups into groups.

That is, when the step of generating a continuous group by dividing the one-dimensional sequence (S300) is performed by the method of FIG. 3 or 4, although not shown in the drawing, the methods of FIGS. 3 and 4 are arranged in the one-dimensional sequence. It is performed repeatedly until all points are divided into groups.

Next, referring to FIG. 1 again, the step of generating a leaf node for each group (S400) will be described. In this step (S400), a leaf node is generated for each group generated in the step S300. do.

The leaf node is a circular region formed around the representative object Op and is formed based on the radius r (Op). The smaller the region is, the better the search performance in the leaf node is by preventing access to unnecessary nodes.

In this case, one leaf node is generated for one group, and the leaf node includes objects in a metric space corresponding to at least one point included in each group generated by splitting from a one-dimensional sequence. Is generated from

Specifically, the points in the group generated from the points in the one-dimensional sequence are points mapped to the k-dimensional Euclidean space, so the dataset that existed on the metric space before the point was mapped to the Euclidean space. Create an object as a leaf node.

At this time, since the one-dimensional sequence is generated from the above steps (S100, S200) so that the relative distance on the metric space of the dataset object is maintained on the Euclidean space, the points of each group generated in the above step (S300) That is, the points included in the group divided from the one-dimensional sequence arranged on the Euclidean space may be adjacent objects on the metric space.

Referring to FIG. 5 and FIG. 6, the leaf node generation for the group (S400) will be described in detail. First, a representative object Op is searched for from the objects for the points included in the group (S410).

This step (S410), in searching for a parent object Op representing all objects of the leaf node with respect to the leaf node including the objects corresponding to the points in the group, the leaf node because the center of the region does not exist in the metric space. The representative object Op is searched by calculating the distance d () between all objects included in the.

를 계산하고 이를 이용하여 리프 노드의 대표 객체

를 산출하고, 이로부터 메트릭 스페이스 상의 대표 객체 Op를 탐색하는 방법으로 대체한다. However, at this time, since the distance d () between objects in the metric space has a distance calculation complexity of O (M ² ) with respect to the number M of points included in the group, in an embodiment of this step S410, the distance between objects Distance between objects on a k-dimensional space whose computational complexity is relatively simpler than the metric space

Representation of leaf nodes using

Is calculated and replaced by a method of searching for the representative object Op in the metric space.

As described above, in searching for the representative object Op among the objects included in the leaf node, the distance between objects is replaced by performing the k-dimensional Euclidean space to calculate the distance between objects in the metric space. Time and cost can be reduced.

의 최대값이 가장 작게 형성되는 객체를 대표 객체

로 산출하도록 수행된다.In calculating the representative object from the objects included in the leaf node in step S410, the maximum value max (d {d ()}) of the distance d () between two objects O _i and O _j is the smallest value. The distance between the points is performed from the following Equation 2 to calculate the object formed as a representative object, and applied from the concept of Equation 2 to the points mapped in the k-dimensional Euclidean space.

A representative object whose largest value is the smallest

To be calculated.

[Equation 2]

를 선정할 수 있고, 이에 따라, 도 6에 도시한 것처럼 동일한 객체를 포함하더라도 대표 객체 선정 오류로 region이 확장되어 이로 인해 검색 성능이 저하되는 문제점을 예방할 수 있는 효과를 기대할 수 있다.As the representative object of the leaf node is calculated (S410) using Equation 2 above, the representative object for the region formed in the k-dimensional Euclidean space

As shown in FIG. 6, even if the same object is included as shown in FIG. 6, a region may be expanded due to a selection error of the representative object, thereby preventing the problem that the search performance is deteriorated.

을 산출(S420)한다. 이 단계(S420)는 다음의 식 3을 통해 수행된다.Then, the radius of the leaf node region from the representative object Op on the metric space of the leaf node calculated from step S410 above.

It is calculated (S420). This step (S420) is performed through the following equation 3.

[Equation 3]

은 리프 노드의 각 객체들 O_i와 메트릭 스페이스 상의 리프 노드 대표 객체 Op간의 거리이고, max_i{}는 i 값들에 따른 d()값 중 가장 큰 값을 추출하는 함수이다. 위의 식 3으로부터, 메트릭 스페이스 상의 리프 노드의 객체들 중 리프 노드의 대표 객체와 거리가 가장 크게 형성되는 값을 리프 노드의 region에 대한 반지름으로 산출한다.In Equation 3 above,

Is the distance between each object O _i of the leaf node and the leaf node representative object Op in the metric space, and max _i {} is a function for extracting the largest value of d () values according to i values. From Equation 3 above, the value of forming the largest distance from the representative object of the leaf node among the objects of the leaf node on the metric space is calculated as the radius of the region of the leaf node.

Finally, in step S430 of creating a leaf node for a group, a leaf node is generated using the representative object and region radius of the leaf node calculated from the above steps S410 and S420.

The leaf node generation step S400 performed according to the example of FIG. 5 is performed for all groups generated from the step S300 of generating a continuous group from the one-dimensional sequence.

Referring to FIG. 1 again, the next step will be described continuously. A non-leaf entry is generated for the leaf node S400 generated for each group in the above step S400 (S500).

A non-leaf entry is a feature value f (Or) representing a routing object Or of a leaf node, a radius r (Or) around the object Or, and a sub-tree for that entry. Ptr (T (Or)), which is a pointer to T (Or), and d (Or, Op), which is the distance between Or and its parent object, that is, the representative object Op of the leaf node. It has a structure of 7.

Next, in comparing Q100 whether the number of non-leaf entries is greater than M, the number of non-leaf entries generated from the above operation S500 is compared with the maximum number M of objects included in the group.

In the comparison step (Q100), if the number of non-leaf entries is larger than M, which is the maximum number of objects included in the group, moving along the example arrow direction to divide the non-leaf entries into entry groups (S610). Perform In one example, this step (S610) may be performed using any one of the full page, heuristic, and resource methods described above in step S300.

로 대체할 수 있다. In this case, in the example in which the above step S610 is performed from the resource method, when dividing the entry from the non-leaf entry, the distance d between the entries included in the non-leaf entry is calculated, Clidian Space Top Street

Can be replaced with

As a more detailed example, when step S610 is performed from the resource method, the non-leaf entry is divided into entries in consideration of not only the distance between entries, but also the radius of the region of the entry.

는 다음의 식 4로부터 도출될 수 있다.Where the distance between any two entries Ei and Ej on the k-dimensional Euclidean space

Can be derived from Equation 4 below.

[Equation 4]

In Equation 4 above, Ei.Or is the routing object Or of the entry object Ei, Ej.Or is the routing object Or of the entry object Ej, and r (Ei.Or) is Ei.Or is the routing object Or of the entry object Ei Is the radius of and r (Ej.Or) is the radius of the routing object Or of the entry object Ej.

For the entry group divided in this step (S610), a step of generating a non-leaf node and a non-leaf entry (S620) is performed.

In one example, in order to continue performing the entry group partitioning step by determining whether the number of non-leaf entries exceeds M even after the number of non-leaf entries is reduced by performing the above steps (S610 and S620), After performing the steps S610 and S620, the comparison step Q100 is performed again, and the steps S610 and S620 are repeatedly performed or another step S700 is performed according to the comparison result.

When the number of non-leaf entries does not exceed M as a result of the determination of the comparison step Q100 according to the repetitive execution of the steps S610 and S620, for example, the non-leaf entries are divided into entry groups. If the number of non-leaf entries is less than M, the mobile station moves in the direction of the NO arrow to store the entire entries in one root node (S700).

At this time, as the entire entries are stored in one root node (S700), each time one entry is generated, the entire entries are loaded into the M-tree, that is, not written to the memory disk, but the entire generated entries are written at once. Since it is loaded on the tree, it is possible to reduce the time and cost required by repetitive access of the disk, and to reduce the consumption of the disk.

As an example, the entries generated from the above steps S610 and S620 may be stored in a buffer, and the above step S700 may be performed when the buffer is full.

로 대체하므로 객체간 거리 계산 복잡도가 O(N)로 형성된다. 이는, 본 발명의 일 실시예를 적용한 경우, 대량의 데이터를 BulkLoading하는 기존의 FastLoad 방법에서의 객체간 거리 계산 복잡도가

인 것에 비해, 객체간 거리 계산 복잡도가 현저히 낮으므로 대량의 데이터셋을 M-트리에 적재하는 데 소요되는 처리시간 및 처리비용을 감소시키는 효과가 있다.According to the M-tree stacking method of a data set using a fast map according to an embodiment of the present invention described with reference to FIGS. 1 to 7, a fast map of a large data set whose distance is not defined on Euclidean space In loading into the M-tree by using, map the dataset object on the metric space on the k-dimensional Euclidean space to maintain the inter-object spacing to generate a one-dimensional sequence, and group the generated one-dimensional sequence Create a leaf node by dividing by

As a result, the complexity of calculating the distance between objects is formed by O (N). In the case of applying an embodiment of the present invention, the complexity of calculating the distance between objects in the existing FastLoad method for bulkloading a large amount of data

Compared to, the complexity of calculating the distance between objects is significantly low, thereby reducing the processing time and processing cost for loading a large data set into the M-tree.

In addition, when a large data set is loaded into the M-tree by applying the method of the present embodiment, the data sets are grouped sequentially and loaded into the M-tree, which is more efficient than when data is recorded by random access to a disk. Data can be recorded.

And, among the steps according to the present embodiment, the first step (S100) and the second step (S200) can be performed independently for each object, so that the above steps (S100, S200) using a multi-core By processing concurrent threads, the time to load large amounts of data into the M-tree can be effectively reduced.

In addition, in the above steps (S300, S400, S500), the group divided from the one-dimensional sequence can be allocated to a plurality of threads, respectively, to be processed in parallel, thereby effectively reducing the time for loading a large amount of data into the M-tree. have.

By using an embodiment of the present invention, the execution time, the number of calculation distances, and the disk accesses according to the data size when a large data set is loaded into the M-tree according to the data size when the conventional technology is used. Compared with the execution time, the number of calculation distances, and the disk access, as shown in the performance graph of FIG. 8, in the case of performing FastLoad according to an embodiment of the present invention (FastLoad-F, FastLoad-H, FastLoad-R) As the data size increases from 4K to 1M, the data upload execution time (FIG. 8A), the number of distance calculations (FIG. 8B), and the number of disk accesses (FIG. 8C) are all conventional. The simulation shows that it is superior to simple insertion and simple BulkLoad.

At this time, FastLoad-F shown in (a) to (c) of FIG. 8 is a case of using the full-page method, FastLoad-H is a case of using the heuristic method, FastLoad-R relates to the case of using the resource method The graph of FIG. 8 is simulated for a disk page having a size of 4 KB in a two-dimensional Euclidean space.

Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements of those skilled in the art using the basic concepts of the present invention defined in the following claims are also provided. It belongs to the scope of rights.

Claims (19)

Performed by a processing unit that loads a data object into an M-tree,
Mapping the dataset on the metric space to a point on the k-dimensional Euclidean space;
aligning the points on the k-dimensional Euclidean space into a one-dimensional sequence;
Dividing the points of the one-dimensional sequence into contiguous groups; And,
Creating a leaf node for the contiguous group;
M-tree loading method of a data set using a fast map for loading the data set in the M-tree including.
The method of claim 1,
And the data set on the metric space is a data set whose similarity between data objects is not defined as the Lp distance between two points in Euclidean space.
The method of claim 2,
And the data set on the metric space is an unstructured multimedia data or a point of interest (POI) of a road network.
The method of claim 1,
Mapping the dataset on the metric space to a point on a k-dimensional Euclidean space is performed such that the inter-object spacing of data on the metric space is mapped so as to remain on the k-dimensional Euclidean space. M-tree loading method of a data set using a fast map, characterized in that.
The method of claim 4, wherein
The mapping of the data set on the metric space to a point on the k-dimensional Euclidean space is performed by Equation 1 below.
[Equation 1]

(d _ij is the distance between two objects O _i , O _j in the metric space,

Are two objects mapped to Euclidean space

Distance, and i and j are natural numbers from 1 to N.)
The method of claim 1,
Mapping a dataset on the metric space to a point on a k-dimensional Euclidean space is performed to map a plurality of objects constituting the dataset to BulkLoading on the Euclidean space. M-tree loading of datasets using fast map.
The method of claim 1,
Arranging the points on the k-dimensional Euclidean space in a one-dimensional sequence is performed using a space filling curve M-tree loading method of a data set using a fast map.
The method of claim 7, wherein
Arranging the points on the k-dimensional Euclidean space in a one-dimensional sequence is performed using any one of a Z curve (Morton curve), a Hilbert curve, and a Gray-code curve as a space filling curve. M-tree loading of datasets using fast map.
The method of claim 1,
In the step of dividing the points of the one-dimensional sequence into a contiguous group, the successive points arranged in the one-dimensional sequence are divided into groups in sequence,
A method of loading an M-tree of a dataset using a fast map, characterized in that the full page method is used to uniformly group all groups to include the maximum objects.
The method of claim 1,
In the step of dividing the points of the one-dimensional sequence into a contiguous group, the successive points arranged in the one-dimensional sequence are divided into groups in sequence,
By using the heuristic method that compares the distance between the points in the group and the number of points in the group, calculates the number of points to be included in the group and uses the calculation result to form the group in a range in which the area of the group does not expand. M-tree loading method of a dataset using a fast map, characterized in that performed by.
The method of claim 10,
In the heuristic method, the point-to-point distance included in the group is calculated to be calculated as the point-to-point distance in the actual metric space or replaced by the distance between two points in the k-dimensional Euclidean space. M-tree loading method.
The method of claim 1,
In the step of dividing the points of the one-dimensional sequence into a contiguous group, the successive points arranged in the one-dimensional sequence are divided into groups in sequence,
Fast candidates are generated by generating candidate groups, calculating the radius of the area size included in each candidate group and the number of points included in the candidate group, and forming a group using the calculation result. M-tree loading of datasets using maps.
The method of claim 12,
In the resource method, the radius of the candidate group region is calculated as a radius on an actual metric space or is replaced by a distance between two points in a k-dimensional Euclidean space. M-tree loading method.
The method of claim 1,
The step of creating a leaf node for the continuous group may include creating a leaf node including an object on a metric space corresponding to the points included in the divided group in the step of dividing the points of the one-dimensional sequence into successive groups. M-tree loading method of a dataset using a fast map, characterized in that performed.
The method of claim 14,
Generating leaf nodes for the contiguous groups is performed to calculate representative objects of leaf nodes using points on k-dimensional Euclidean space. How to load the tree.
The method of claim 1,
After creating a leaf node for the contiguous group,
Generating a non-leaf entry for the leaf node created for each group;
Dividing the non-leaf entries into an entry group according to the number of non-leaf entries; and storing the entire entries in one root node; M-tree loading method of a dataset using a fast map, characterized in that further performing any one of steps.
The method of claim 16,
And dividing the non-leaf entries into a group of entries is performed by a full page method, a heuristic method, or a resource method.
The method of claim 16,
The entry created in the step of dividing the non-leaf entry into an entry group is performed to be stored in a buffer,
And storing the entire entry in one root node when the buffer is full, thereby writing the entry to a memory disk.
A computer-readable recording medium having recorded thereon a program capable of executing on a computer the method of loading an M-tree of a dataset using a fast map according to any one of claims 1 to 18.

Images