KR101940251B1 - Spatial knowledge extraction system implemented in Hadoop Map Reduce parallel, distributed computing environment - Google Patents

Abstract

A spatial knowledge extraction system based on Hadoop MapReduce is disclosed. The system builds an R-tree index consisting of two layers, a global index and a local index, for the entire spatial data distributed on the Hadoop Distributed File System (HDFS) Based on the R-tree index, the spatial relation of spatial data is discriminated and spatial knowledge is extracted.

Description

In this paper, we propose a spatial knowledge extraction system based on Hadoop MapReduce.

The present invention relates to a qualitative spatial knowledge extraction technique, and more particularly, to a qualitative spatial knowledge extraction technique using a quantitative spatial reasoning algorithm.

There are qualitative spatial reasoning and quantitative spatial reasoning as methods for acquiring spatial knowledge. The qualitative spatial reasoning is based on the initial qualitative spatial knowledge base according to the spatial algebra theories such as RCC (Region Connection Calculus) -8 and CSD (Cone Shaped Direction Relations) And applying the rules to derive new phase and direction relationship knowledge. And quantitative spatial reasoning is a method of determining the qualitative spatial relation between two spatial objects through geometric computation based on spatial data representing unique shape and position information of each object. Quantitative spatial reasoning does not require the construction of the initial qualitative spatial knowledge base to obtain new qualitative spatial knowledge, but requires precise numerical data. Unlike qualitative spatial reasoning, which has a problem of lack of initial knowledge base, quantitative spatial reasoning is well established on semantic web such as OSM (Open Street Map), USGS (United States Geological Survey), OS OpenData Web-scale open data can be utilized. For this reason, quantitative spatial reasoning can be an alternative to the lack of initial qualitative spatial knowledge base in qualitative spatial reasoning. In addition, since there is uncertainty in the numerical data, quantitative spatial reasoning and qualitative spatial reasoning can be complementary to each other. However, since the quantitative spatial reasoning systems developed so far are designed to operate in a single machine computing environment, there is a performance limitation in performing quantitative spatial inference on web-scale open data.

Korean Patent Registration No. 10-0685791 (issued on February 22, 2007)

A technical solution for efficiently generating a qualitative spatial knowledge base showing a satisfactory spatial relationship between two arbitrary spatial objects from a large-capacity spatial data set is disclosed.

According to an aspect of the present invention, a spatial knowledge extraction system based on Hadoop mapper may include a first index construction unit, a data selection unit, a second index construction unit, and a spatial knowledge extraction unit. The first index construction unit constructs an R-tree index for all the spatial data distributed on the Hadoop Distributed File System (HDFS), and allocates the entire spatial data to the minimum nodes constituting the lowest nodes of the R- It is possible to group and redistribute them in units of minimum bounding rectangle (MBR), construct a local index for each redistributed lowest node MBR, and construct a global index containing meta information about the constructed regional indexes. The data selection unit may select some spatial data for spatial knowledge extraction among spatial data belonging to the lowest-order node MBRs indexed by the first index construction unit. The second index construction unit constructs an R-tree index for some spatial data selected by the data selection unit, redirects some of the spatial data to MBR units constituting the lowest nodes of the R-tree and redistributes them, You can build a local index for each node MBR and build a global index that contains meta information about the constructed local indexes. The spatial knowledge extracting unit can extract spatial knowledge by determining the spatial relationship of some spatial data selected by the data selection unit based on the R-tree index constructed by the second index building unit.

The predetermined MBR unit may have an HDFS block size.

The data selection unit may select some spatial data from the spatial data belonging to the lowest node MBRs indexed by the first index construction unit through a range query.

The data selection unit checks whether the areas of the lowest node MBRs indexed by the R-tree indexed by the input range and the first index construction unit intersect each other. Only the MBRs having the area intersecting the input range are distributed to Map functions A range query can be performed.

The spatial knowledge extracting unit includes a topological relation knowledge extracting unit for extracting the reference data and the knowledge of the phase relationship between the reference data and the spatial data belonging to the lowest node MBRs indexed by the R-tree by the second index building unit, And a direction relation knowledge extracting unit for extracting a knowledge of a phase relationship with spatial data belonging to the lowest-order node MBRs indexed by the R-tree indexing unit. The spatial data selected by the data selecting unit may be referred to as reference data Can be selected.

The phase relation knowledge extraction unit checks whether the lowest-order node MBRs indexed by the second index construction unit intersect with the lowest-order node MBR to which the reference data belongs. If the intersection labels or the cross- The comparative label is labeled, and the phase relation between the reference data and the spatial data belonging to the lowest node MBR labeled with the difference label is identified as 'disjoint', extracted as the phase relation knowledge, and the reference data and the crossing label are labeled The phase relation between the spatial data belonging to the lowest node MBR and the phase relation can be extracted by modeling for determination of the phase relation.

The phase relation knowledge extractor can determine the phase relation between the reference data and spatial data belonging to the lowest node MBR labeled with a cross-label using a dimensionally extended nine-intersection model (DE-9IM).

The direction relation knowledge extraction unit models the areas of the direction angle based on the center point of the lowest node MBR to which the reference data belongs, reads the spatial data belonging to each lowest node MBR using the Map function, And the center point, and the directional relationship between the reference data and the spatial data can be discriminated according to the direction angle region to which the MBR center point of the modeled spatial data belongs, and extracted as the directional relationship knowledge.

The direction relation knowledge extracting unit can determine the direction relation by calculating the direction angle when the MBR center point of the modeled spatial data does not belong to any one of the regions of the direction angle.

Meanwhile, according to one aspect of the present invention, a spatial knowledge extraction method based on Hadoop MapReduce is performed by constructing an R-tree index on all spatial data distributed on a Hadoop Distributed File System (HDFS) Are grouped into a minimum bounding rectangle (MBR) unit to constitute the lowest nodes of the R-tree, and a local index is constructed for each redistributed lowest node MBR, and meta information about the constructed local indexes is stored A first index construction step of constructing a global index having a root node index, a data selection step of selecting some spatial data for spatial knowledge extraction among spatial data belonging to the lowest-order node MBRs indexed by the R-tree by the first index construction step, Tree indexes are constructed for spatial data, and some spatial data are stored in MBR nodes constituting the lowermost nodes of the R-tree. , Constructing a local index for each redistributed least significant node MBR, constructing a global index containing meta information about the constructed local indexes, and constructing a global index by constructing a second index construction step And a spatial knowledge extraction step of extracting spatial knowledge by determining a spatial relation of some spatial data based on the extracted R-tree index.

The disclosed technique utilizes R-tree indexes and range queries for distributed spatial data files on the Hadoop distributed file system to efficiently extract web-scale qualitative spatial knowledge bases.

The disclosed technology can be widely used in the fields of intelligent geographic information systems, knowledge management systems, query-response systems, spatial databases, web information retrieval, and intelligent service robots in which spatial knowledge and spatial reasoning capability are indispensable.

1 is a conceptual diagram for representing spatial knowledge according to an embodiment.
FIG. 2 is a reference diagram for explaining a distributed R-tree index of spatial objects according to an exemplary embodiment. FIG.
FIG. 3 illustrates a logical structure of a spatial knowledge extraction system based on Hadoop MapReduce according to an embodiment.
FIG. 4 illustrates an R-tree index construction process according to an embodiment.
Figure 5 shows an example of a random sample selection operation.
Figure 6 shows an example of physical partitioning and local index building.
FIG. 7 illustrates a mapping process based on a data selection process according to an exemplary embodiment of the present invention.
FIG. 8 shows an example of a data selection job process according to FIG.
FIG. 9 shows a process of extracting a phase relationship extraction operation according to an embodiment.
FIG. 10 shows an example of a process of extracting a phase relationship extraction operation according to FIG.
FIG. 11 shows a processing procedure of a directional relationship extracting operation according to an embodiment.
FIG. 12 shows an example of a process of extracting a directional relationship extraction operation according to FIG.
13 is a block diagram of a spatial knowledge extraction system based on Hadoop MapReduce according to an embodiment.

BRIEF DESCRIPTION OF THE DRAWINGS The foregoing and further aspects of the present invention will become more apparent from the following detailed description of preferred embodiments with reference to the accompanying drawings. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a conceptual diagram for representing spatial knowledge according to an embodiment. Prior to describing the qualitative spatial knowledge extraction method, the geometric data and the spatial knowledge of the spatial object, that is, the spatial knowledge expression system, will be described with reference to FIG. Figure 1 shows a spatial ontology that adds nine directional descriptors defined in the CSD (Cone-Shaped Directional Relations) -9 theory to the core ontology of GeoSPARQL. A spatial object is a top-level class that represents all spatial objects. The boundaries and containment relationships between two spatial objects can be divided into seven categories: disjoint, touches, equals, overlaps, within, contains, and crosses. The directional relationship between the two spatial objects can be expressed in terms of north (N), north-east (NE), east (E), south- east-west (SE), south (S), south-west (SW), west (W), northwest (NW) and identical- And can be represented by directional properties. The subclasses of the spatial object include features and geometries. A feature represents a specific place in the real world, such as a city, road, or building, and geometry represents geometric data of a feature such as a Point, LineString, or Polygon. The geometry of the geometry is represented by a literal in WKT (Well-Known Text) format, while the feature is represented by a literal in the form of a String.

FIG. 2 is a reference diagram for explaining a distributed R-tree index of spatial objects according to an exemplary embodiment. In order to extract a large amount of qualitative spatial knowledge, a space object created in a Hadoop Distributed File System (HDFS) ≪ / RTI > is a distributed R-tree index for the < RTI ID = The structure of an R-tree index consists of two layers, a global index and local indexes. That is, the R-tree is composed of one parent node (highest node) and a plurality of lower nodes (lowest node) belonging thereto. Spatial data grouped in units of Minimum Boundary Rectangle (MBR), which is the lowest node of the R-tree index, is stored in each of the slave nodes 200 in the local index file (part-xxxxx, rtree) The meta information is stored in the master node 100 as one global index file (_master.rtree). The meta information stored in the global index includes an identifier (ID) of a local index, a boundary, a file name of a local index file, a record count, and a file size. In one embodiment, spatial data stored in each of the slave nodes is allocated and stored as much as the HDFS block size for load balancing. That is, the least significant MBRs of the R-tree, i.e., area indexed MBRs, are created in HDFS block size units. Construction of the R-tree index will be described later.

3 is a diagram illustrating a logical structure of a spatial knowledge extraction system based on Hadoop MapReduce according to an embodiment. Spatial knowledge extraction system extracts web - scale spatial knowledge based on MapReduce - based quantitative spatial reasoning algorithm to extract qualitative spatial knowledge of web scale. To this end, the spatial knowledge extraction system may include a first index construction unit 211, a data selection unit 220, a second index construction unit 212, and a spatial knowledge extraction unit 240. In one embodiment, the first index construction unit 211, the second index construction unit 212, and the spatial knowledge extraction unit 240 are configured for each slave node 200, and the processor of the slave node 200 Lt; / RTI > And the data selection unit 220 may be configured in the master node 100 and executed by the processor of the master node 100. [

The first index construction unit 211 performs a distributed R-tree index construction operation on all spatial data randomly distributed on the HDFS. In this work, spatial data scattered randomly on the HDFS is redistributed to several slave nodes in groups of the lowest node MBR units as much as the HDFS block size. Once constructed for the entire spatial data, the R-tree index is maintained for subsequent data selection operations. The data selection unit 220 performs a range selection operation of spatial data for the knowledge extraction operation using the R-tree index constructed by the first index construction unit 211. [ This task uses spatial queries such as range queries and k-nearest neighbors.

The second index construction unit 212 performs a separate R-tree index construction operation on the spatial data selected by the data selection unit 220. The R-tree index construction method of the first index construction unit 211 and the R-tree index construction method of the second index construction unit 212 are the same. Since the spatial knowledge extraction method of the spatial knowledge extraction unit 240, which will be described later, determines the spatial relation using the locality of the redistributed data based on the R-tree index, the R- Building work is essential. Unlike the R-tree index constructed by the first index construction unit 211, the R-tree index constructed by the second index construction unit 212 does not need to maintain the index after extracting all the spatial knowledge. Therefore, the R-tree index constructed by the second index construction unit 212 can be discarded after all the spatial knowledge has been extracted.

The first index construction unit 211 and the second index construction unit 212 are substantially one index construction unit. The first index construction unit 211 constructs an R-tree index on all the spatial data, and the second index construction unit 212 constructs an R-tree index by the first index construction unit 211. [ Tree index is constructed with respect to some selected spatial data after it is constructed. The R-tree index is divided into a first index construction unit 211 and a second index construction unit 212 to distinguish the selected spatial data. Hereinafter, the R-tree index constructed by the first index construction unit 211 is referred to as a first R-tree index, and the R-tree index constructed by the second index construction unit 212 is referred to as a second R- It is called an index.

The reference selection unit 230 selects one of the spatial data selected by the data selection unit 220 as reference data before extracting the spatial knowledge, and selects all the selected spatial data as reference data once. The spatial knowledge extraction unit 240 performs a knowledge extraction operation with all remaining spatial data selected by the data selection unit 220 every time the reference data is selected. 3, the spatial knowledge extracting unit 240 includes a phase relation knowledge extracting unit 241 and a direction relation knowledge extracting unit 242. The phase relation knowledge extracting unit 241 determines the phase relation between the reference data and all the remaining spatial data and extracts the phase relation as the phase relation knowledge. The direction relation knowledge extracting unit 242 extracts the direction relation between the reference data and all remaining spatial data Direction relation is extracted and extracted as direction relation knowledge. 3, the first index construction work, the data selection work, the second index construction work, and the spatial knowledge extraction work are the map deuce work, and the reference data selecting work is not the map deuce work.

Hereinafter, each of the spatial knowledge extraction processes based on the MapReduce will be described in more detail.

(1) R-tree index generation

4, each of the first index construction unit 211 and the second index construction unit 212 is divided into three parts: partitioning, local indexing, and global indexing. The index build process is performed. First, in the partitioning step, input data (entire spatial data) is divided into n number of partitions. The formula for determining n is shown in Equation (1).

In Equation (1), B is the HDFS block size, S is the size of the input data (total spatial data), and? Is the overhead ratio considering the input data that is copied and stored in each node for fault tolerance. The value of a is set to 0.2 as default. When n is determined, partition boundaries of each partition must be determined. The boundaries of partitions can be determined using a Sort-Tile-Recursive (STR) algorithm. However, since the STR algorithm is performed by a single processor, it is impossible to store all input data of a large capacity in memory. Therefore, we randomly select samples of the input data (random sampling), and use it to perform the STR algorithm. The random selection of the input data samples proceeds to a single MapReduce operation. The input data is divided and delivered to the input of the map function. If the shape of the input data is not a point, it is converted to the center point of the MBR and transferred to the Map function. The Map function outputs the input data with a probability of 1%. If the output size exceeds 100 MB, random sample selection is repeated until the result is less than 100 MB.

을 STR 알고리즘의 매개 변수(parameter) d(R-tree degree)로 설정하고 샘플을 대상으로 STR 알고리즘을 수행한다. STR 알고리즘은 MBR로 파티션의 경계를 정하기 때문에 각각의 파티션의 모양은 MBR이 되며, 이는 곧 R-트리 색인의 최하위 노드 MBR이 된다. 추후 입력 데이터 부하 분산을 위해 데이터들이 HDFS 블록 크기만큼 파티션에 포함되도록 경계를 정한다. 파티션의 경계가 결정되면, 각각의 파티션에 포함되는 데이터들을 그룹지어 재분배하는 물리적 파티션 분할(physical partitioning) 작업을 수행한다. 물리적 파티션 분할과 이후 진행되는 지역 색인 구축 단계는 하나의 맵리듀스 작업이다.Once the sample is selected,

Is set as a parameter d (R-tree degree) of the STR algorithm, and the STR algorithm is performed on the sample. Since the STR algorithm determines the boundaries of the partitions with the MBR, the shape of each partition becomes the MBR, which is the lowest node MBR of the R-tree index. For future input data load balancing, the data is bounded to include the HDFS block size in the partition. When the boundaries of the partitions are determined, the physical partitioning operation of redistributing the data included in each partition is performed. Physical partitioning and subsequent local index building steps are one mapping task.

First, the input data is split and passed to each Map function. The Map function determines which partition (= lowest node MBR) belongs to the received data, and performs a Reduce function by making the partition and its data into a key-value pair. The Reduce function shuffles the MapReduce, passes each partition as a key, and the data belonging to it as a list, and outputs one local index file (part-xxxx.rtree) for each partition. The local index files have information about the data belonging to the partition and the boundaries of this MBR. Finally, in the global index construction stage, a global index file (_master.rtree) containing meta information of all local indexes created is created and stored in the master node.

Figure 5 shows an example of a random sample selection operation. First, all the spatial data (geom_1 to geom_6) input to extract the representative sample data are divided into respective Map functions. The Map function randomly selects data based on a pre-determined ratio of 1% to the discretized data, and writes the selected data (geom_1, geom_4, and geom_6) as a result. If the result is larger than 100 MB, repeat the operation until the result is less than 100 MB.

Figure 6 shows an example of physical partitioning and local index building. First, if initial MBR (initial MBR) and all spatial data (geom_1 ~ geom_6) for grouping into MBR are inputted, input data is divided by each Map function. The Map function examines which MBR area the input data belongs to, and groups the data together with the corresponding MBR (data grouping) and sends it to the Reduce function. If the data does not belong to any MBR, it is considered to belong to the nearest MBR. The Reduce function uses the partition (mbr_1, mbr_2) passed through shuffling (MBR & data shuffling) to list the values with the same partition as a key and the list of data (list_1, list_2) (MBR & data writing) to a binary index of a local index file together with corresponding data per partition.

(2) Selection of work scope

When the first R-tree index is constructed, the data selection unit 220 selects the input data for the spatial knowledge extraction operation. Data selection is performed using spatial queries such as range queries, k-nearest neighbors. Using the locality of the R-tree index greatly reduces data access cost and processing cost in spatial query execution, which is very helpful for data selection. Hereinafter, an example of performing the data selection operation based on the range query will be described. The process of selecting data based on MapReduce is shown in FIG.

Referring to FIG. 7, in the data selection operation, the lowest node MBRs of the R-tree index and a range including a lot of spatial data that are the main object of interest are inputted. In particular, before the MBR distribution of the lowest node MBRs to the Map function, it is checked whether the input range and the lowest node MBR area intersect to reduce unnecessary data access costs. The lowest non-intersecting node MBR is pre-pruned (MBR pruning) so that it is not distributed to the Map function. Then, the lowest-level node MBRs that are not pruned are distributed to the Map function, and the spatial data belonging to each MBR is read from each Map function (data reading). Finally, if the read spatial data belongs to a range, it is output as a result.

FIG. 8 shows an example of the data selection job process described above. First, in order to perform the data selection operation, the MBRs (mbr_1 and mbr_2) of the R-tree index and the addresses (local address_1 and local address_2) and the range of the area index stored therein are received. Before delivering mbr_1 and mbr_2 to each Map function, check whether range and area intersect, distribute mbr_1 where area intersects with map function, and not otherwise, mbr_2 does not distribute to map function by pruning. In other words, data that is not related to the range is filtered in advance to avoid unnecessary data access and processing. The Map function reads all the data belonging to the mbr_1 passed to it, forms it into a list, checks whether the data belonging to the list actually belongs to the range, and outputs geom_1 and geom_2, which belong to the range.

(3) Extraction of phase relation knowledge

The process of extracting the phase relation performed by the phase relation knowledge extracting unit 241 every time the reference data is selected is shown in FIG. The phase relation extraction operation receives the lowest node MBRs of the second R-tree index, the address of the spatial data belonging thereto, and the reference data used as a reference for extracting the phase relation. In order to reduce the unnecessary phase relationship analysis cost, the phase relation extraction process checks whether the lowest-order node MBRs of the lowest-order node MBRs intersect with the lowest-order node MBR to which the reference data belongs before distributing the lowest-order node MBRs to Map functions. Cross-label or non-intersection labeling of node MBRs (MBR labeling). We then distribute the labeled lowest node MBRs to each Map function, and the Map function reads the spatial data belonging to each lowest node MBR (data reading). Before determining the phase relationship between the reference data and the read spatial data by modeling the phase relation, the lowest node MBR and phase relationship modeling that can determine the phase relation only by looking at the labels attached to the lowest node MBRs and topological relation modeling are omitted And classify the lowest node MBR that can identify the phase relation (MBR classification). That is, we classify the lowest node MBR labeled with a cross-label and the lowest node MBR labeled with a difference label. The phase relation between the reference data and the spatial data belonging to the least significant node MBR labeled with a difference label is determined by 'disjoint'. The phase relation between the reference data and the spatial data belonging to the lowest node MBR with the crossed label is determined by phase relationship modeling, and the phase relation can be determined by modeling using DE-9IM (Dimensionally Extended nine-Intersection Model) .

FIG. 10 shows an example of a process of extracting a phase relation. First, the MBRs (mbr_1 and mbr_2) of the second R-tree index and the addresses (local address_1 and local address_2) where the spatial data belonging thereto are located and the base geometry are input to proceed with the phase relation extraction operation . Before transferring mbr_1 and mbr_2 to each Map function, check whether the area intersects with the MBR of the reference data, add cross label to mbr_1 where the area intersects, and compare label to mbr_2 that does not map (MBR labeling) (MBR distribution). The map function reads mbr_1 and mbr_2 and converts mbr_1 and mbr_2 to DE-9IM modeling and mbr_2, which can distinguish phase relationship by omitting DE-9IM modeling (MBR classification. When the MBRs are classified, the data in mbr_1 is determined based on the DE-9IM modeling with the reference data, and the data in mbr_2 is omitted from the DE-9IM modeling and is "disjoint" . When the phase relation is determined, the phase relation knowledge is synthesized (knowledge synthesized) and outputted as a result (knowledge writing).

(4) Extraction of orientation relation knowledge

The processing of the directional relationship extracting operation performed by the directional relationship knowledge extracting unit 242 every time the reference data is selected is as shown in FIG. The direction relation extraction operation receives the lowest node MBRs of the second R-tree index, the address of the spatial data belonging thereto, and the reference data that is a reference of the direction relation extraction. In order to use a range query that requires less processing cost than to directly calculate a direction angle, the MBR of the reference data, that is, the MBR of the reference data itself is obtained, and is formed in the top node MBR of the second R-tree index based on the center point Directional area modeling. Here, the highest node MBR refers to all MBRs including the lowest node MBRs, and in FIG. 11, MBR for mbr_1, mbr_2, mbr_3, and mbr_4.

Then, the lowest node MBRs are distributed to the map function (MBR distribution), and the map function reads the spatial data belonging to each lowest node MBR (data reading). (MBR & centroid modeling) the MBR of the read spatial data itself and its center point before performing the range query on each direction domain. As a result, the directional relationship is determined according to which of the eight directional angles the center point of the MBR of the spatial data belongs to. If the center point lies on the boundary of the directional angle region and does not belong to any of the eight directional angle regions, the directional angle is actually calculated to determine the directional relationship.

Fig. 12 shows an example of a process of extracting a directional relationship. First, in order to perform the direction relation extraction operation, the lowest node MBRs (mbr_1, mbr_2) of the second R-tree index and the addresses (local address_1, local address_2) and the base geometry Receive input. Before transferring mbr_1 and mbr_2 to each Map function, the MBR of the reference data is obtained, and directional area models formed in the top node MBR are modeled based on the center point, and mbr_1 and mbr_2 are mapped to Map function (MBR distribution). The Map function reads all the data belonging to the mbr_1 and mbr_2 that have been passed into the list (list_1, list_2). The data in the list are sequentially modeled (MBR & centroid modeling) by their MBR and center point, and the range query is performed on each of the 8 directional areas to determine the directional relationship according to which area the center point belongs to. When the direction relation is determined, knowledge synthesis is performed on the direction relation knowledge and the result is outputted as knowledge writing.

13 is a block diagram of a spatial knowledge extraction system based on Hadoop MapReduce according to an embodiment. The master node 100 located at the front end of the spatial knowledge extraction system includes a web interface 110, an email notifier 120, a download tool 130 ). First, a user designates a knowledge extraction area (for example, designates a rectangular work area) on a map through the web interface 110 and inputs parameters for knowledge extraction extraction parameter to request a MapReduce operation. Selecting a knowledge extraction area is a necessary process because of unnecessary knowledge extraction and expansion of knowledge capacity extracted by it. When the requested quantitative reasoning work is performed in the back-end slave nodes 200, the user is informed of the work progress status from the email notification unit 120. When the work is completed, the user can download the download tool 130 The extracted knowledge stored in the back-end can be received as an N-Triple text file.

The back-end of the spatial knowledge extraction system extracts the spatial knowledge by distributing and parallel processing the tasks requested from the user by the processors of the respective slave nodes 200. The slave nodes 200 include an index construction unit 210, a data selection unit 220, and a spatial knowledge extraction unit 240. First, the index construction unit 210 constructs an R-tree index from spatial data (USGS, OpenStreetMap, OSOpen Data) including geographical information such as city, road, Based on the MBRs of the lowest MBRs in the group. Then, when a knowledge extracting operation is requested, a data selector receives the knowledge extraction area of the rectangle inputted by the user and performs a range query. The selected data is constructed again by R-tree index construction and data redistribution by the index builder, and the knowledge extraction is performed by processing the knowledge extraction work based on the parameters set by the user.

The present invention has been described with reference to the preferred embodiments. It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. Therefore, the disclosed embodiments should be considered in an illustrative rather than a restrictive sense. The scope of the present invention is defined by the appended claims rather than by the foregoing description, and all differences within the scope of equivalents thereof should be construed as being included in the present invention.

100: master node 200: slave node
210: Index construction unit 211: First index construction unit
212: second index construction unit 220: data selection unit
230: Reference line selection unit 240: Spatial knowledge extraction unit
241: phase relation knowledge extracting unit 242: direction relation knowledge extracting unit

Claims (17)

Tree index on the entire spatial data distributed on the Hadoop Distributed File System (HDFS), and the entire spatial data is divided into a minimum bounding rectangle to constitute the lowermost nodes of the R-tree, Rectangle, and MBR), constructing a local index for each redistributed least significant node MBR, and constructing a global index containing meta information about the constructed regional indexes;
A data selection unit for selecting some spatial data for spatial knowledge extraction through a range query among spatial data belonging to the lowest-order node MBRs indexed by the first index construction unit by R-tree indexing;
Tree index is constructed for some spatial data selected by the data selection unit, and some spatial data is grouped into MBR units constituting the lowest nodes of the R-tree and redistributed, and a local index A second index construction unit for constructing a global index including meta information about the constructed local indexes; And
And a spatial knowledge extraction unit for extracting spatial knowledge by determining a spatial relationship of some spatial data selected by the data selection unit based on the R-tree index constructed by the second index construction unit,
The data selection unit checks whether the areas of the lowest node MBRs indexed by the R-tree indexed by the input range and the first index construction unit intersect each other. Only the MBRs having the area intersecting the input range are distributed to Map functions Hadoop MapReduce - based spatial knowledge extraction system that performs range query.
The method according to claim 1,
A spatial knowledge extraction system based on Hadoop MapReduce having a predetermined MBR unit having an HDFS block size.
delete delete 3. The apparatus according to claim 1 or 2, wherein the spatial knowledge extracting unit comprises:
A phase relation extracting unit for extracting a phase relation knowledge between the reference data and spatial data belonging to the lowest-order node MBRs indexed by the second index construction unit; And
And a direction relation knowledge extracting unit for extracting a direction relation knowledge between the reference data and the spatial data belonging to the lowest node MBRs indexed by the R-tree by the second index building unit,
The spatial data extraction system based on Hadoop MapReduce is selected by the data selection unit as the reference data once.
6. The method of claim 5,
The phase relation knowledge extraction unit checks whether the lowest-order node MBRs indexed by the second index construction unit intersect with the lowest-order node MBR to which the reference data belongs. If the intersection labels or the cross- The comparative label is labeled, and the phase relation between the reference data and the spatial data belonging to the lowest node MBR labeled with the difference label is identified as 'disjoint', extracted as the phase relation knowledge, and the reference data and the crossing label are labeled Hadoop mapper-based spatial knowledge extraction system that identifies the phase relationship between the spatial data belonging to the lowest-level node MBR and the topological relation through modeling for phase relation determination.
The method according to claim 6,
The spatial relation knowledge extraction unit extracts the spatial relationship between the reference data and spatial data belonging to the lowest node MBR labeled with a cross-label by using DE-9IM (Dimensionally Extended nine-Intersection Model).
6. The method of claim 5,
The direction relation knowledge extraction unit models the areas of the direction angle based on the center point of the lowest node MBR to which the reference data belongs, reads the spatial data belonging to each lowest node MBR using the Map function, And extracts the direction relation between the reference data and the spatial data according to the direction angles to which the MBR center point of the modeled spatial data belongs.
9. The method of claim 8,
The directional relationship knowledge extractor is a Hadoop mapper-based spatial knowledge extraction system that calculates orientation angles when the MBR center point of the modeled spatial data does not belong to any one of the regions of directional angles.
delete delete A method for extracting spatial knowledge based on Hadoop MapReduce performed by one or more processors,
The first index construction unit of the processor constructs an R-tree index on the entire spatial data distributed on the Hadoop Distributed File System (HDFS), and constructs the lowest spatial nodes of the R- A first index for grouping and redistributing a minimum boundary rectangle (MBR) unit, building a local index for each redistributed lowest node MBR, and building a global index containing meta information about the constructed regional indexes Building phase;
The data selection unit of the processor may include a data selection step of selecting some spatial data for spatial knowledge extraction through a range query among spatial data belonging to the lowest-order node MBRs indexed by R-trees in the first index construction step;
The second index construction unit of the processor constructs an R-tree index for some spatial data, groups and redirects some spatial data into MBR units constituting the lowermost nodes of the R-tree, A second index construction step of constructing a global index including meta information about the constructed regional indexes; And
The spatial knowledge extracting unit of the processor includes a spatial knowledge extracting step of extracting spatial knowledge by determining a spatial relation of some spatial data based on the R-tree index constructed by the second index building step,
In the data selection step, it is checked whether the input range and the area of each of the lowest-order node MBRs indexed by the first index construction unit intersect each other. Only MBRs having an area intersecting the input range are distributed A spatial knowledge extraction method based on Hadoop MapReduce that performs range queries.
13. The method of claim 12, wherein extracting the spatial knowledge comprises:
Extracting a phase relation knowledge between the reference data and spatial data belonging to the lowest node MBRs indexed by the R-tree by the second index construction step; And
Extracting a phase relation knowledge between the reference data and spatial data belonging to the lowest-level node MBRs indexed by the R-tree by the second index construction step;
A spatial data extraction method based on Hadoop MapReduce based on spatial data selected in the data selection stage.
14. The method of claim 13,
In the phase relation knowledge extraction step, it is checked whether the lowest-order node MBRs indexed by the R-tree index intersect with the lowest node MBR belonging to the reference data by the second index construction step, and cross- Or comparison label is labeled and the phase relationship between the reference data and the spatial data belonging to the lowest node MBR labeled with the difference label is identified as 'disjoint', extracted as the phase relation knowledge, and the reference data and cross- A spatial knowledge extraction method based on Hadoop MapReduce that identifies the phase relation between spatial data belonging to the labeled lowest node MBR by modeling for topological relationship and extracts it as phase relation knowledge.
15. The method of claim 14,
The phase relation knowledge step is a Hadoop mapper-based spatial knowledge extraction method for determining the phase relation between the reference data and the spatial data belonging to the lowest node MBR labeled with a cross-label using a dimensionally extended nine-intersection model (DE-9IM).
14. The method of claim 13,
In the step of extracting the directional relationship knowledge, the direction angle areas are modeled based on the center point of the lowest node MBR to which the reference data belongs, the spatial data belonging to each lowest node MBR is read using the Map function, Hadoop Mapper-based Spatial Knowledge Extraction Method that models the MBR and its center point and extracts direction relations between the reference data and the spatial data according to the direction angles of the MBR center point of the modeled spatial data. .
17. The method of claim 16,
The method of extracting the directional relationship knowledge extracts a spatial knowledge based on Hadoop MapReduce based on determining directional angles when the MBR center point of the modeled spatial data does not belong to any one of the directional angle regions.

Images