JP2012058836A - Distributed processing system and distributed processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a distributed processing system and a distributed processing method capable of performing distributed processing scalable in units of mesh optimal for distributed processing without being conscious of the position or scale of a map element.SOLUTION: A distributed processing system (1) includes: zone acquisition means (22) for reading range designation information indicating a closed zone occupied by a zone map element on a map; point element acquisition means (32) for reading zone designation information indicating a point occupied by a point map element on the map; mesh specification means (23, 33) for specifying a mesh, of plural meshes, the mesh including at least a part of zone designation information of the zone map element and point map element; data generation means (24, 34) for generating the same number of KEY-VALUE data pieces (where KEY is a specified mesh, and VALUE is read zone designation information) as the number of acquired meshes; and execution processing distribution means (25, 36) for causing plural servers to execute processing on the generated KEY-VALUE data in units of mesh.

Description

  The present invention relates to a distributed processing system and a distributed processing method for performing distributed processing of map data constituting a map that can be partitioned into a plurality of predetermined meshes.

  With the explosion of web pages in recent years, the amount of information existing on the network has become enormous. MapReduce is known as a software framework for performing processing for such an enormous amount of information on a network (for example, Non-Patent Document 1).

Keisuke Nishida (Author) “Technology that Supports Google” World inside a giant system, published by Technical Review Co., Ltd., April 25, 2008, Chapter 4

Here, MapReduce is a framework for performing parallel processing using a computer cluster that is a set of many computers. In order to perform predetermined processing, MapReduce divides input data to be processed into a plurality of fragments and assigns them to a plurality of computers. In each computer, processing according to the Map definition is performed in parallel to perform KEY-VALUE. A Map task that generates type data as an intermediate file, sorts the KEY-VALUE type data, bundles them with the same KEY, and assigns a computer that processes the intermediate file for each KEY, and each assigned computer, It consists of a Reduce task that performs processing according to the Reduce definition in parallel, aggregates the processing results, and obtains the result of the predetermined processing. According to MapReduce, distributed processing for an enormous amount of information is realized by dividing it into a Map task and a Reduce task.
In recent years, Hadoop is known as open source software that implements MapReduce, and anyone can easily perform large-scale distributed processing by using Hadoop.

  Incidentally, in recent years, map information has been actively distributed on the Internet. For example, when a store name or the like is searched on a map, a service that displays a map to which the store belongs is known. Such a service is realized by appropriately generating index data for forming a map. Here, the map can divide the map into latitude (N) longitude (E), and the map on the Internet is formed of a plurality of meshes divided into latitude and longitude. For this reason, in order to distribute the index generation among a plurality of computers (servers), it is preferable to generate the index data for each mesh (mesh unit).

  However, map elements with a certain range on the map (for example, buildings, natural objects, regions, etc.) may span multiple meshes. In such a case, it is appropriate to simply distribute each mesh. Index data cannot be generated.

  The present invention performs processing for map data including map elements that span a plurality of meshes, and is capable of scaling in units of meshes that are optimal for distributed processing, without regard to the position and size of the map elements. An object of the present invention is to provide a distributed processing system and a distributed processing method.

  The present inventors have completed the present invention by newly developing a method for generating a KEY-VALUE type data set suitable for performing distributed processing for each mesh unit. In the following, “shuffle” includes processing of organizing the generated KEY-VALUE type data set and sorting the data set by KEY.

  (1) Parallel processing of map data constituting a map that can be partitioned into a plurality of predetermined meshes by assigning one of a plurality of servers connected via a network to each predetermined plurality of meshes by shuffling In the distributed processing system to be executed, at least a part of the plurality of servers includes the map data, mesh identification data for identifying each of the predetermined plurality of meshes, and a position indicating the position of the mesh on the map Storage control means for associating and storing information in the storage means, area acquisition means for reading out the range designation information representing the closed area occupied by the area map element included in the map on the map, and the map Point element acquisition means for reading range designation information representing points where the included point map elements are located on the map from the storage means; and The mesh identification data and the position information are read out from the storage unit, and the mesh including at least a part of the closed region represented by the range designation information read out by the region acquisition unit out of the plurality of meshes and the point element acquisition KEY-VALUE type data in which mesh specifying means for specifying a mesh including the point represented by the range designation information read by the means, mesh identification data for identifying the identified mesh is set to KEY, and the range designation information is set to VALUE. Using the data generation means for generating the number of meshes specified by the mesh specification means, and the generated KEY-VALUE type data, the processing relating to the positional relationship between the area map elements and the point map elements is performed with the mesh specification. For each mesh specified by the means, one of the plurality of servers is shuffled. A distributed processing system comprising: an execution processing distribution unit that allocates and executes in parallel.

  According to the distributed processing system of (1), the mesh specifying unit specifies a mesh including at least a part of the range specifying information of the closed region to be distributed and a mesh including the point range specifying information. The data generation means generates KEY-VALUE type data sets in which the identification data of the identified mesh is KEY and the range designation information is VALUE, as many as the number of meshes specified by the mesh specification means. Then, the execution processing distribution unit sorts the generated KEY-VALUE type data set by KEY by shuffling, and performs processing on the KEY-VALUE type data set to a plurality of servers for each mesh specified by the mesh specifying unit. Assign and execute.

  As a result, the KEY-VALUE type data set, in particular, the range specification information of the closed region to be distributed is provided to all of the plurality of servers to which the processing is distributed, so distributed processing is performed for each mesh unit. Therefore, a suitable KEY-VALUE type data set can be generated. As a result, even when the closed region extends over a plurality of meshes, it is possible to perform distributed processing that can be scaled in units of meshes that are optimal for distributed processing, without being aware of the position and size of the closed region. Since the KEY-VALUE type data set is sorted by KEY by shuffle, the distributed processing system performs processing related to the positional relationship between the area map element and the point map element on the same server for the data set having the same KEY. Can be executed in parallel.

  (2) The mesh specifying unit includes a base point mesh specifying unit that specifies a base point mesh to which arbitrary point information included in the range specifying information belongs, and an adjacent mesh adjacent to the base point mesh specified by the base point mesh specifying unit. And an inclusion determining unit that determines whether or not at least part of the closed region indicated by the range designation information is included in the adjacent mesh specified by the adjacent mesh specifying unit. The distributed processing system according to 1).

  According to the distributed processing system of (2), after specifying a base point mesh to which arbitrary point information of the range designation information belongs, a mesh including at least a part of the range designation information among meshes adjacent to the base point mesh is included. The mesh is determined to include at least a part of the range designation information. Thereby, even when the closed region extends over a plurality of meshes, the plurality of meshes can be appropriately acquired. Note that “adjacent” includes not only 8 meshes around the mesh but also meshes of 24, 49, etc., depending on the size of the range designation information.

  (3) Parallel processing for distributing map data constituting a map that can be partitioned into a plurality of predetermined meshes by assigning one of a plurality of servers connected via a network to each predetermined plurality of meshes by shuffling A distributed processing method to be executed, wherein at least a part of the plurality of servers includes the map data, mesh identification data for respectively identifying the predetermined plurality of meshes, and a position indicating the position of the mesh on the map A storage control step for associating and storing information in the storage means, an area acquisition step for reading out range designation information representing the closed area occupied by the area map element included in the map on the map from the storage means, and the map A point element acquisition step of reading out range designation information representing a point where the included point map element is located on the map from the storage means The mesh identification data and the position information are read from the storage means, and the mesh including at least a part of the closed region represented by the range designation information read in the region acquisition step among the plurality of meshes and the point KEY-VALUE that specifies a mesh including the point represented by the range designation information read in the element acquisition step, mesh identification data for identifying the identified mesh is KEY, and the range designation information is VALUE. Using the data generation step for generating the type data as many as the number of meshes specified by the mesh specification step, and the generated KEY-VALUE type data, the processing relating to the positional relationship between the area map element and the point map element is performed as described above. For each mesh identified by the mesh identification step A distributed processing system comprising: an execution processing distribution step in which one of the plurality of servers is allocated by shuffle and executed in parallel.

  According to the distributed processing method of (3), the same effects as those of the distributed processing system of (1) are obtained.

  According to the present invention, it is possible to perform distributed processing that can be scaled in units of meshes optimal for distributed processing, without being aware of the position and size of map elements.

It is a figure which shows the outline | summary of the distributed processing system which concerns on an example of embodiment of this invention. It is a figure which shows the function structure of the distributed processing system which concerns on an example of embodiment of this invention. It is a figure which shows the memory | storage means which concerns on an example of embodiment of this invention. It is a conceptual diagram which shows the flow of a process of the mesh specification means which concerns on an example of embodiment of this invention. It is a flowchart which shows the flow of a process of the distributed processing system which concerns on an example of embodiment of this invention. It is a conceptual diagram which shows the flow of a process of the distributed processing system which concerns on an example of embodiment of this invention. It is a figure which shows index DB produced | generated by the distributed processing system which concerns on an example of embodiment of this invention.

  Embodiments of the present invention will be described below with reference to the drawings.

[Overview of Distributed Processing System 1]
First, an overview of the distributed processing system 1 of the present embodiment will be described with reference to FIG.

As shown in FIG. 1A, the distributed processing system 1 includes a master server M and a plurality of worker servers W. The distributed processing system 1 can be realized using Hadoop, which is open source software that implements the above-described MapReduce.
The master server M makes a request to the worker server W to divide input data to be subjected to predetermined processing into a large number of fragments, assign the worker server W, and perform processing for each. The worker server W receives a request from the master server M, generates a KEY-VALUE type data set from the input data as an intermediate file according to the Map definition, and sorts the generated KEY-VALUE type data set for each KEY. The completion of the process is notified to the master server M. Then, the master server M requests the worker server W to allocate the worker server W for each KEY and perform processing for each. The worker server W receives a request from the master server M, performs processing for each KEY-VALUE type data set according to the Reduce definition, aggregates the results, and performs predetermined processing relating to the positional relationship, for example, index data. The result of the generation process is obtained.

  In the present embodiment, the processing for the area map element 100 of the map A (including meshes A1, A2, A3, A4, A5, and A6) shown in FIG. The relationship between the area map element 100 and the point map element 110 is determined. Here, the area map element 100 is a map element having a certain range such as a building or an area within a predetermined distance from the station, and the point map element 110 is treated as a point on a map such as a restaurant. Map element. In the present embodiment, a process for determining whether or not the point map element 110 is included in the area map element 100, for example, whether or not a certain restaurant exists in a certain building is performed by a plurality of worker servers W. Distributed.

[Functional configuration of distributed processing system 1]
Next, the functional configuration of the distributed processing system 1 will be described with reference to FIG.

  The distributed processing system 1 generates an area map element process 2 and a point map element process 3 (Map task) that generate a KEY-VALUE type data set with a mesh ID (mesh identification data) for identifying a mesh as KEY. A region inside / outside determination process 4 (Reduce task) for performing processing on a KEY-VALUE type data set for each mesh (on a mesh basis) is executed. The area map element process 2, the point map element process 3 and the area inside / outside determination process 4 are executed by the worker server W, but each worker server W has the area map element process 2, the point map element process 3 and the area inside / outside area. It can function as a server that executes any one of the determination processes 4, and the master server M distributes it as necessary. Therefore, one worker server W may execute not only the area map element process 2 but also the point map element process 3, and may execute the area inside / outside determination process 4. Further, the master server M itself can function as the worker server W.

  In order to execute the area map element processing 2, the distributed processing system 1 includes a storage control means 21, an area acquisition means 22, a mesh identification means 23, a data generation means 24, and an execution processing distribution means 25. Composed.

The storage control unit 21 controls the storage unit 5 provided at an arbitrary place in the distributed processing system 1 and stores predetermined information in the storage unit 5.
Here, the memory | storage means 5 is comprised including mesh DB51 and map element DB52. As shown in FIG. 3A, the mesh DB 51 stores map information including position information indicating the position of the mesh on the map in association with the mesh ID for identifying the mesh. The mesh is a map divided into latitude and longitude. As shown in FIG. 3B, the map element DB 52 stores map element information including range designation information and map element names in association with map element IDs. The map element may be a point element, a map element having a rectangular range, or a map element having a polygonal range. Although the designation of the map element range designation information can be arbitrarily performed, in this embodiment, the map element range designation information is designated by designating the latitude and longitude of the vertex of the map element. .
The storage control unit 21 stores, for example, the mesh ID and the position information of the mesh ID in the mesh DB 51 of the storage unit 5 in association with each other.
Note that the worker server W itself does not need to include the storage unit 5 as long as the worker server W can specify a destination from which various information is read. The storage unit 5 can be provided at any location accessible by the worker server W. Of course, in terms of data locality, the worker server W executing the area map element process 2 and the point map element process 3 for performing the process of accessing the storage means 5 reads the data from the storage means 5 of the own node and performs the process. It can be said that it is preferable.

  The area acquisition unit 22 reads out range designation information representing the closed area occupied by the area map element to be processed on the map from the map element DB 52 provided at an arbitrary location.

The mesh specifying unit 23 reads the mesh ID and position information from the mesh DB 51 and specifies a mesh including at least a part of the closed region represented by the range designation information read by the region acquiring unit 22.
Here, since the closed region indicated by the range specifying information acquired by the region acquiring unit 22 is a closed region expressed by the range specifying information related to the region map element having a certain range, there are a plurality of meshes specified by the mesh specifying unit 23. there's a possibility that. Therefore, in the present embodiment, the mesh specifying unit 23 includes a base point determining unit 231, a base point mesh specifying unit 232, an adjacent mesh specifying unit 233, and an inclusion determining unit 234.

  The base point determination unit 231 determines arbitrary point information included in the closed region represented by the range designation information. As the arbitrary point information, for example, the center point of the area map element, the vertex of the area map element, and the like can be considered. In this embodiment, the center point of the area map element is used as the arbitrary point information. Therefore, the base point determination unit 231 calculates the center point of the area map element. The calculation of the center point can be performed by an arbitrary method, and may be calculated according to a predetermined algorithm or may be stored in advance in the map element DB 52.

  The base point mesh specifying unit 232 specifies a mesh to which arbitrary point information (center point) included in the closed region represented by the range designation information belongs as a base point mesh.

  The adjacent mesh specifying means 233 specifies a mesh adjacent to the base point mesh as an adjacent mesh. The adjacent mesh specifying means 233 is not limited to eight meshes (adjacent) around the base mesh, but depending on the size of the area map element, the base mesh such as 24 meshes around the base mesh, 49 meshes, etc. It is good also as acquiring the surrounding mesh.

  The inclusion determination unit 234 determines whether or not the adjacent mesh includes at least a part of the closed region represented by the range designation information.

  Here, with reference to FIG. 4, the flow of processing in which the mesh specifying unit 23 acquires a mesh including at least a part of the closed region represented by the range designation information will be described.

First, the mesh identification unit 23 (base point determination unit 231) determines a center point 100A, which is arbitrary point information of the area map element 100 (FIG. 4 (1)).
Subsequently, the mesh specifying unit 23 (base mesh specifying unit 232) specifies the mesh A5 to which the center point 100A belongs (FIG. 4 (2)).
Subsequently, the mesh specifying means 23 (adjacent mesh specifying means 233) acquires adjacent meshes A1, A2, A3, A4, A6,... That are meshes around the mesh A5 (FIG. 4 (3)).
Finally, the mesh specifying unit 23 (inclusion determining unit 234) determines whether or not at least part of the closed region indicated by the range designation information is included for each of the adjacent meshes A1, A2, A3, A4, A6. And a mesh including at least a part of the closed region represented by the range designation information is specified (FIG. 4 (4)). In FIG. 4 (4), meshes A2, A3, A5, and A6 are specified as meshes including at least a part of the closed region represented by the range designation information of the region map element 100.

  Returning to FIG. 2, the data generation unit 24 generates a KEY-VALUE type data set in which the mesh acquired by the mesh specifying unit 23 is KEY and the range designation information read by the region acquiring unit 22 is VALUE. Generate as many meshes as you have. For example, the data generation means 24 may include “KEY: Mesh A2, VALUE: Range designation information of area map element 100”, “KEY: Mesh A3, VALUE: Range designation information of area map element 100”, “KEY: Mesh A5, A KEY-VALUE type data set such as “VALUE: range designation information of area map element 100” and “KEY: mesh A6, VALUE: range designation information of area map element 100” is generated.

  The execution process distribution unit 25 shuffles the KEY-VALUE type data set, that is, sorts the KEY-VALUE type data set by the KEY and performs the inside / outside determination process for each mesh (for each mesh acquired by the mesh specifying unit 23). By distributing to the worker server W that executes 4, the processing for the generated KEY-VALUE type data set is distributed in mesh units, and the inside / outside determination processing 4 is executed.

  Subsequently, in order to execute the point map element process 3, the distributed processing system 1 includes a storage control unit 31, a point element acquisition unit 32, a mesh identification unit 33, a data generation unit 34, and an execution process distribution unit 35. , Including. In addition, since each means for performing the point map element process 3 is basically the same as each means for executing the area map element process 2, detailed description is omitted.

  The storage control unit 31 controls the storage unit 5 provided at an arbitrary location in the distributed processing system 1 and stores predetermined information in the storage unit 5. The point element acquisition means 32 reads the point map element range designation information (point information) from the map element DB 52. The mesh specifying unit 33 specifies a mesh including the range designation information read by the point element acquisition unit 32. Here, since the range designation information of the point map element is point information having no range, it is sufficient for the mesh specifying means 33 to specify the mesh to which the point information belongs. Of course, when the point map element has a certain range, the same processing as that of the mesh specifying means 23 may be performed.

  The data generation unit 34 generates a KEY-VALUE type data set in which the mesh specified by the mesh specification unit 33 is KEY and the range designation information read by the point element acquisition unit 32 is VALUE. For example, when determining whether or not a certain restaurant (restaurant 1 to N) is present in a certain building, the data generation means 34 is “KEY: mesh to which restaurant 1 belongs, VALUE: range designation information of restaurant 1”, Generate a KEY-VALUE-type data set such as “KEY: mesh to which restaurant 2 belongs, VALUE: range designation information for restaurant 2”..., “KEY: mesh to which restaurant N belongs, VALUE: range designation information for restaurant N” .

  The execution process distribution unit 35 shuffles the KEY-VALUE type data set, that is, sorts the KEY-VALUE type data set by KEY, and distributes the data set to the worker server W that executes the area inside / outside determination process 4 in mesh units. The processing for the generated KEY-VALUE type data set is distributed in units of meshes, and the region inside / outside determination processing 4 is executed. Thereby, the worker server W that executes the area inside / outside determination process 4 can obtain all the range designation information included in the predetermined mesh indicated by the mesh ID assigned as KEY.

  Thus, since the KEY-VALUE type data set sorted by the KEY is provided to the worker server W that executes the area inside / outside determination process 4, the distributed processing system 1 uses the data set having the same KEY. The processing related to the positional relationship between the area map element and the point map element can be executed in parallel on the same worker server W.

Subsequently, the distributed processing system 1 is configured to include processing execution means 41 for executing processing related to the positional relationship on the map in units of meshes in order to execute the region inside / outside determination processing 4.
For example, for the worker server W that executes the area inside / outside determination process 4, “KEY: Mesh A2, VALUE: Area map element 100 range designation information” and “KEY: Mesh A2, VALUE: Mesh A2 range designation. KEY-VALUE-type data set such as “information” is given, and the worker server W determines whether or not the restaurant exists in the area map element 100 as an example of processing related to the positional relationship on the map. . The determination (spatial operation) of whether or not a restaurant exists in the area map element 100 can be realized by using, for example, JTS that is open source software.

  The worker server W to which the process is distributed by the execution process distribution unit 25 (35) executes the area inside / outside determination process 4. Depending on the distribution, the area map element process 2 or the point map element process 3 is executed. The worker server W itself can be the worker server W that executes the area inside / outside determination process 4.

[Hardware configuration of each server constituting the distributed processing system 1]
The hardware of each server configuring the distributed processing system 1 described above can be configured by a general computer. For example, a general computer includes a central processing unit (CPU) as a control unit, and a memory (RAM, ROM), a hard disk (HDD), and an optical disk (CD, DVD, etc.) as a storage unit, and a network communication device. Various wired and wireless LAN devices as display devices, for example, various displays such as liquid crystal displays and plasma displays, and as input devices, for example, keyboards and pointing devices (mouse, tracking balls, etc.) Are connected by a bus line. In such a general computer, the CPU controls each server constituting the distributed processing system 1 in an integrated manner, reads and executes various programs as appropriate, and cooperates with the hardware described above. Various functions related to are realized.

[Processing of distributed processing system 1]
Next, processing of the distributed processing system 1 will be described with reference to FIG.

[Area map element processing 2]
S201: The area acquisition unit 22 reads from the map element DB 52 range designation information that represents a closed area that the area map element occupies on the map.
For example, when determining whether or not the point map elements 111, 112, 113, 114, 115, 116 exist in the area map element 100 (see FIG. 6), the area acquisition unit 22 uses the range designation information of the area map element 100. Are read from the map element DB 52.

S202: The mesh specifying unit 23 reads the mesh ID and position information from the mesh DB 51, and specifies a mesh including at least a part of the closed region represented by the range designation information read in S201. Specifically, the mesh specifying unit 23 determines the center point of the area map element 100 and specifies the base point mesh to which the center point belongs. Then, the mesh specifying unit 23 acquires an adjacent mesh adjacent to the base mesh, and specifies a mesh including at least a part of the closed region represented by the range designation information among the adjacent mesh. Further, the mesh specifying unit 23 acquires an adjacent mesh further adjacent to the adjacent mesh until the above specification is completed for all the parts of the closed region, and repeats the same processing.
In FIG. 6, meshes A <b> 2, A <b> 3, A <b> 5, A <b> 6 are acquired as meshes including at least a part of the closed region represented by the range designation information of the region map element 100.

S203: The data generation unit 24 generates KEY-VALUE type data sets for which the number of meshes specified in S202 is KEY and the range specification information read in S201 is VALUE.
In FIG. 6, “KEY: Mesh A2, VALUE: Range specification information of area map element 100”, “KEY: Mesh A3, VALUE: Range specification information of area map element 100”, “KEY: Mesh A5, VALUE: Area map. A KEY-VALUE type data set such as “range specification information of element 100” and “KEY: mesh A6, VALUE: range specification information of area map element 100” is generated.

S204: The execution process distribution unit 25 distributes the KEY-VALUE type data set to the worker server W that executes the area inside / outside determination process 4 in units of meshes.
In FIG. 6, “KEY: Mesh A2, VALUE: Range specification information of area map element 100” is distributed to a worker server W (or a plurality, the same below) of the worker servers W that execute the area inside / outside determination process 4, "KEY: Mesh A3, VALUE: Range specification information of area map element 100" is distributed to another worker server W among the worker servers W that execute the area inside / outside determination process 4, and "KEY: Mesh A5, VALUE: Area map" The “range designation information of the element 100” is distributed to another worker server W among the worker servers W that execute the area inside / outside determination processing 4, and “KEY: mesh A6, VALUE: range designation information of the area map element 100” is designated as the area. The worker server W that executes the inside / outside determination process 4 is assigned to another worker server W.

[Point map element processing 3]
S301: The point element acquisition means 32 reads the range specification information (point information) of the point map element from the map element DB 52.
In FIG. 6, the range designation information of the point map elements 111, 112, 113, 114, 115, and 116 for determining whether or not the area map element 100 exists is read from the map element DB 52.

S302: The mesh specifying unit 33 reads the mesh ID and the position information from the mesh DB 51, and specifies the mesh including the range designation information read in S301.
In FIG. 6, the mesh A2 is acquired as a mesh including the point map element 111, the mesh A5 is acquired as the mesh including the point map element 112, the mesh A5 is acquired as the mesh including the point map element 113, and the point map element 114 is acquired. The mesh A3 is acquired as the mesh including the point map, the mesh A3 is acquired as the mesh including the point map element 115, and the mesh A6 is acquired as the mesh including the point map element 116.

S303: The data generation unit 34 generates a KEY-VALUE type data set in which the mesh specified in S302 is KEY and the range designation information read in S301 is VALUE.
In FIG. 6, “KEY: Mesh A2, VALUE: Range designation information of point map element 111”, “KEY: Mesh A3, VALUE: Range designation information of point map element 114”, “KEY: Mesh A3, VALUE: Point map. “Range specification information of element 115”, “KEY: Mesh A5, VALUE: Range specification information of point map element 112”, “KEY: Mesh A5, VALUE: Range specification information of point map element 113”, “KEY: Mesh A6, A KEY-VALUE type data set such as “VALUE: range designation information of point map element 116” is generated.

S304: The execution process distribution unit 35 distributes the KEY-VALUE type data set to the worker server W that executes the area inside / outside determination process 4 in units of meshes.
In FIG. 6, “KEY: Mesh A2, VALUE: Range specification information of the point map element 111” is distributed to a worker server W among the worker servers W that execute the area inside / outside determination processing 4, and “KEY: Mesh A3, “VALUE: Range designation information of point map element 114” and “KEY: Mesh A3, VALUE: Range designation information of point map element 115” are sent to another worker server W among the worker servers W that execute the area inside / outside determination processing 4. Sorting, “KEY: Mesh A5, VALUE: Range designation information of point map element 112” and “KEY: Mesh A5, VALUE: Range designation information of point map element 113” of worker server W executing area inside / outside determination processing 4 Assign to another worker server W, “KEY: Mesh A6, VALUE: Point map required 116 range specification information "of sorts to yet another worker server W of the worker server W to perform an area outside judgment process 4.

[Inside / outside area determination processing 4]
S401: Each of the worker servers W that executes the area inside / outside determination process 4 acquires the KEY-VALUE type data set distributed in S204 and S304 in units of meshes. In addition, the worker server W that executes the area inside / outside determination process 4 uses the Hadoop MultipleInputs class to simultaneously acquire and read the area designation information of the area map element 100 and the area designation information of the point map elements 111 to 116. Can do.
In FIG. 6, the worker server W that performs processing of the mesh A <b> 2 has “KEY: mesh A <b> 2, VALUE: range specification information of the area map element 100” and “KEY: mesh A <b> 2, VALUE: range specification information of the point map element 111”. The worker server W that performs processing of the mesh A3 obtains “KEY: mesh A3, VALUE: range designation information of the area map element 100”, “KEY: mesh A3, VALUE: range designation information of the point map element 114”. And “KEY: Mesh A3, VALUE: Range designation information of point map element 115” and worker server W that performs processing of mesh A5, “KEY: Mesh A5, VALUE: Range designation information of area map element 100” , “KEY: Mesh A5, VALUE: Range designation information of point map element 112” and “KE : Mesh A5, VALUE: Range specification information of point map element 113 ", and worker server W that performs processing of mesh A6 performs" KEY: Mesh A6, VALUE: Range specification information of area map element 100 "and" KEY. : Mesh A6, VALUE: Range specification information of point map element 116 ”.

S402: The process execution means 41 executes a process related to the positional relationship on the map, for example, a process of determining whether or not the point map elements 111 to 116 are present in the area map element 100. Note that the processing related to the positional relationship on the map can be realized by using, for example, JTS which is open source software.
In FIG. 6, the point map elements 112 and 115 are determined to exist in the area map element 100, and the point map elements 111, 113, 114, and 116 are determined not to exist in the area map element 100.

[Effect of distributed processing system 1]
In the distributed processing system 1 of the present embodiment, when the processing related to the area map element 100 that spans a plurality of meshes is performed in a distributed manner in units of meshes, the range designation information of the area map element 100 is applied to all the meshes that the area map element 100 spans. It is supposed to have. Thereby, regardless of the position and size of the area map element, it is possible to perform distributed processing that can be scaled in units of meshes that are optimal for distributed processing.
As an example, as a result of the distributed processing by the distributed processing system 1, an index DB as shown in FIG. 7 can be generated. For example, index data indicating that a certain restaurant exists in a certain building can be generated. In addition, for example, if a certain range along Ring 7 (Tokyo Metropolitan Route 318 Ring 7) is set as an area map element, index data indicating that a ramen shop is along Ring 7 is generated. You can also. If a range within a 5-minute walk from the station is set as an area map element, index data indicating that a certain shop exists within a 5-minute walk from a certain station can be generated.

  As mentioned above, although embodiment of this invention was described, this invention is not restricted to embodiment mentioned above. The effects described in the embodiments of the present invention are only the most preferable effects resulting from the present invention, and the effects of the present invention are limited to those described in the embodiments of the present invention. is not.

DESCRIPTION OF SYMBOLS 1 Distributed processing system 2 Area | region map element processing 21 Storage control means 22 Area | region acquisition means 23 Mesh specification means 24 Data generation means 25 Execution processing distribution means 3 Point map element processing 31 Storage control means 32 Point element acquisition means 33 Mesh specification means 34 Data Generation means 35 Execution process distribution means 4 Area inside / outside determination process 41 Process execution means 5 Storage means 51 Mesh DB
52 Map Element DB
M Master server W Worker server

Claims (3)

Distribution of map data constituting a map that can be partitioned into a plurality of predetermined meshes is performed in parallel by assigning one of a plurality of servers connected via a network by shuffle to each of the predetermined plurality of meshes. A processing system,
At least a part of the plurality of servers stores the map data and mesh identification data for identifying the predetermined plurality of meshes in association with position information indicating the positions of the meshes on the map in the storage unit. Storage control means for
A region acquisition unit that reads out from the storage unit range designation information that represents a closed region occupied by the region map element included in the map on the map;
Point element acquisition means for reading out range designation information representing points where point map elements included in the map are located on the map;
The mesh identification data and the position information are read from the storage unit, and the mesh including at least a part of the closed region represented by the range designation information read by the region acquisition unit among the plurality of meshes and the point element Mesh specifying means for specifying a mesh including the point represented by the range specifying information read by the acquiring means;
Data generating means for generating KEY-VALUE type data having the mesh identification data for identifying the identified mesh as KEY and the range specifying information as VALUE as many as the number of meshes specified by the mesh specifying means;
Using the generated KEY-VALUE type data, one of the servers is shuffled for each mesh specified by the mesh specifying means for processing related to the positional relationship between the area map element and the point map element. Execution processing distribution means for executing in parallel,
A distributed processing system. The mesh specifying means includes
Base point mesh specifying means for specifying a base point mesh to which any point information included in the range designation information belongs;
An adjacent mesh specifying means for specifying an adjacent mesh adjacent to the base mesh specified by the base mesh specifying means;
Inclusion determination means for determining whether or not at least a part of the closed region indicated by the range designation information is included in the adjacent mesh specified by the adjacent mesh specifying means;
The distributed processing system according to claim 1. Distribution of map data constituting a map that can be partitioned into a plurality of predetermined meshes is performed in parallel by assigning one of a plurality of servers connected via a network by shuffle to each of the predetermined plurality of meshes. A processing method,
At least a part of the plurality of servers stores the map data and mesh identification data for identifying the predetermined plurality of meshes in association with position information indicating the positions of the meshes on the map in the storage unit. A storage control step to
An area acquisition step of reading out range designation information representing a closed area occupied on the map by an area map element included in the map;
A point element acquisition step of reading out range designation information representing points where the point map elements included in the map are located on the map;
The mesh identification data and the position information are read from the storage means, and the mesh including at least a part of the closed region represented by the range designation information read in the region acquisition step among the plurality of meshes and the point element A mesh specifying step for specifying a mesh including the point represented by the range designation information read in the obtaining step;
A data generation step for generating KEY-VALUE type data having the mesh identification data for identifying the identified mesh as KEY and the range designation information as VALUE for the number of meshes identified by the mesh identification step;
Using the generated KEY-VALUE type data, one of the servers is shuffled for each mesh identified by the mesh identifying step, with respect to the positional relationship between the area map element and the point map element. Execution process distribution step to be executed in parallel,
A distributed processing system.

Images