WO2019069506A1

WO2019069506A1 - Feature value generation device, feature value generation method, and feature value generation program

Info

Publication number: WO2019069506A1
Application number: PCT/JP2018/022428
Authority: WO
Inventors: テイ陳; 幸貴楠村; 遼平藤巻; 和世成田; 理人浅原; 優輔村岡
Original assignee: 日本電気株式会社
Priority date: 2017-10-05
Filing date: 2018-06-12
Publication date: 2019-04-11

Abstract

A table acquisition means 281 acquires a first table including prediction objects and first geographical attributes, and a second table including second geographical attributes. If the values of the second geographical attributes satisfy a prescribed condition with respect to the values of the first geographical attributes, an attribute adding means 282 adds, to the attributes of the first table, as feature values, i.e. variables which may affect the prediction objects, statistical values of distances calculated on the basis of the values of the first geographical attributes and the values of the second geographical attributes which satisfy the condition.

Description

Feature quantity generation device, feature quantity generation method and feature quantity generation program

The present invention relates to a feature quantity generation device, a feature quantity generation method, and a feature quantity generation program that combine a plurality of tables to generate a feature quantity.

Data mining is a technology for finding useful knowledge that has been unknown so far from a large amount of information. In order to find useful findings that are unknown, it is important to generate more attribute candidates. Specifically, it is important to generate candidates for many attributes (explanatory variables) that can affect variables (target variables) to be predicted. By generating such a large number of candidates, it is possible to increase the possibility that an attribute useful for prediction is included in the candidates.

For example, Patent Document 1 describes that candidates for feature quantities used in machine learning processing are generated by combining a target table including an object variable with a source table not including an object variable. In the method described in Patent Document 1, the process of generating candidate feature quantities is defined by a combination of three conditions of Filter conditions, map conditions, and reduce conditions, and thus the number of analysts who generate candidate feature quantities. To reduce

Further, Patent Document 2 describes a demand forecasting device that predicts the number of demands for a dispatch service of vehicles such as taxis in a forecast target area by regression analysis. The demand prediction device described in Patent Document 2 acquires estimated population information in a predetermined area, and uses the acquired estimated population information as an explanatory variable of regression analysis.

International Publication No. 2017/090475 JP, 2011-113141, A

The present inventor has received the idea that prediction accuracy is improved by utilizing various information sources when predicting any target in a predetermined area. That is, it may be preferable to combine multiple related information sources to obtain information.

For example, Patent Document 1 exemplifies using a customer ID commonly included in a target table and a source table as a joining condition (that is, map condition) between the target table and the source table. Further, in Patent Document 2, the same reference (area ID, area polygon, and the like) is the prediction target area, which is a unit when predicting the number of demand for service, and the predetermined area, which is a unit of estimated population information used as an explanatory variable. It is stated that it is defined by).

However, when various information sources are used for prediction, the method of defining geographical information included in each information source may be different from the method of defining geographical information at the time of prediction. The person found it. For example, in the case of geographical information, it is possible to specify by latitude and longitude, or to specify by municipality name. Furthermore, in this case, the present inventor has found that the task of generating candidate feature quantities for predicting a prediction target from each information source can be complicated.

That is, in Patent Document 1 and Patent Document 2, it is assumed that each information source is associated with a customer ID or the same standard. However, even if it is assumed that geographical information is used to associate each information source, such geographical information is not necessarily defined on the same basis. Therefore, it is difficult to simply associate these information sources, and there is a problem that data analysis using such information requires a large number of man-hours.

Then, this invention aims at providing the feature-value production | generation apparatus, the feature-value production | generation method, and the feature-value production | generation program which can produce | generate a feature-value efficiently from several information sources which have geographical information.

A feature quantity generation device according to the present invention comprises a table acquisition unit for acquiring a first table including a prediction target and a first geographical attribute, and a second table including a second geographical attribute, Calculated based on the value of the first geographical attribute and the value of the second geographical attribute satisfying the condition when the value of the second geographical attribute with respect to the value of the geographical attribute satisfies a predetermined condition An attribute adding means is provided for adding the statistics of the distance to the attribute of the first table as a feature that is a variable that can affect the prediction target.

The feature quantity generation method according to the present invention obtains a first table including a prediction target and a first geographical attribute, and a second table including a second geographical attribute, and generates the first geographical attribute. A distance statistic calculated based on the value of the first geographic attribute and the value of the second geographic attribute that satisfies the condition, when the value of the second geographic attribute to the value satisfies a predetermined condition Is added to the attribute of the first table as a feature that is a variable that can affect the prediction target.

A feature amount generation program according to the present invention includes a table acquisition process for acquiring, on a computer, a first table including a prediction target and a first geographical attribute, and a second table including a second geographical attribute. , Based on the value of the first geographical attribute and the value of the second geographical attribute satisfying the condition, when the value of the second geographical attribute with respect to the value of the first geographical attribute satisfies a predetermined condition It is characterized in that an attribute addition process is performed in which the statistical value of the distance calculated as described above is added to the attribute of the first table as a feature quantity that is a variable that can affect the prediction target.

According to the present invention, it is possible to efficiently generate feature quantities from a plurality of information sources having geographical information by the above-mentioned technical means.

FIG. 1 is a block diagram illustrating an embodiment of an information processing system according to the present invention. It is an explanatory view showing an example of a configuration file. It is explanatory drawing which shows the example of the process which converts data. It is explanatory drawing which shows the example of the relationship between each parameter, a 1st table, and a 2nd table. It is explanatory drawing which shows the example of the process which produces | generates a map parameter based on distance. It is explanatory drawing which shows the example of the other process which produces | generates a map parameter based on distance. It is explanatory drawing which shows the example of the method of judging whether it is contained in the same area. It is explanatory drawing which shows the example of the process which produces | generates a map parameter based on whether it is a common area. It is an explanatory view showing an example of processing which generates map parameters based on an inclusion relation. It is explanatory drawing which shows the example of the process which produces | generates a map parameter based on the difference of time. It is explanatory drawing which shows the example of the process which produces | generates a map parameter based on the similarity of text. It is explanatory drawing which shows the example of the process which produces | generates a map parameter based on the similarity of a structure. It is explanatory drawing which shows the example of the produced | generated map parameter. It is explanatory drawing which shows the example of the process which produces | generates the intensive parameter for calculating the statistics value of distance. It is explanatory drawing which shows the example of the process which produces | generates the intensive parameter for calculating the statistics value of an area | region. It is explanatory drawing which shows the example of the produced | generated aggregation parameter. It is explanatory drawing which shows the example which couple | bonded map parameters. It is explanatory drawing which shows the example of the method of combining a parameter and producing | generating a feature-value production | generation function. It is a flow chart which shows an example of processing which generates joint conditions. It is a flow chart which shows other examples of processing which generates joint conditions. It is a flowchart which shows the example of the process which produces | generates a feature-value. It is a flowchart which shows the other example of the process which produces | generates a feature-value. It is a block diagram which shows the outline | summary of the feature-value production | generation apparatus by this invention. It is a schematic block diagram showing composition of a computer concerning at least one embodiment.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

The information processing system according to the present embodiment includes a table (hereinafter also referred to as a first table) including variables to be predicted (for example, target variables) and a table different from the first table (hereinafter referred to as a second table). It may be described as a table of). In the following description, the first table may be referred to as a target table, and the second table may be referred to as a source table. Also, the first table and the second table may each include a set of data.

In the present embodiment, the first table and the second table each include an attribute common to the viewpoint. Common viewpoint means that the semantic content of the data of the attribute is common. Note that the method of representing data may be common or different. Hereinafter, the attribute included in the first table is described as a first attribute, and the attribute included in the second table is described as a second attribute.

For example, as an attribute having a common viewpoint, a geographical viewpoint, a temporal viewpoint, and the like can be mentioned. For example, the value of the attribute of geographical viewpoint can be classified into the following four types of geographical data types. Note that the description below the colon in the heading indicates the syntax for the data.
(1) Point P (Point): p = (x, y) ∈ P
The point P is represented as coordinates (longitude, latitude).
(2) the polygon G (Polygon): g = ( b 1, b 2, ..., b n) ∈G
The polygon G is defined by one outer boundary b ₁ and an inner boundary (b ₂ ,..., B _n ) of 0 or more. Here, b ₁ = (p ₁ , p ₂ ,..., P _n ) (where p ₁ , p ₂ ,..., P _n ∈ P) is a closed order defined as an order of three or more points Of the ring.
(3) a plurality polygons M (MultiPoligon): m = ( g 1, g 2, ..., g n) ∈M, g 1, g 2, ..., g n ∈G
The multi-polygon M is composed of one or more polygons.
(4) String S (String): s ∈ S
It is an address represented by a character string.

Also, as semantic information related to data analysis, an analysis data type may be defined in association with a data type. For example, in the case of the geographical point of view described above, polygon G and polygon M may be defined as an analysis data type for area and point P may be defined as an analysis data type for point . Also, the character string related to the address may be defined as, for example, an analysis data type related to a country, a city, a town, a landmark, a street or a point. Hereinafter, an analysis data type representing geographical information may be referred to as a geographical data type.

Also, for example, the type (temporal data type) of a temporal viewpoint attribute can be defined as a TimeStamp type.

Hereinafter, when the attribute having a common viewpoint is a geographical attribute, the attribute included in the first table is referred to as a first geographical attribute, and the attribute included in the second table is referred to as a second geographical attribute. Similarly, when the attribute having a common viewpoint is a temporal attribute, the attribute included in the first table is referred to as a first temporal attribute, and the attribute included in the second table is referred to as a second temporal attribute. The same shall apply to the other attributes. The first geographical attribute may be the primary key of the first table.

In the above, the common attribute shows the example of the geographical point of view and the time point of view, but the common attribute is not limited to the geographical point of view and the time point of view. Other examples of common attributes include the character string aspect and the structural aspect. The value of the attribute of the character string viewpoint is, for example, an address or the like. Further, the value of the attribute of the structural viewpoint is, for example, a URL (Uniform Resource Locator), a tree structure path, or the like. Hereinafter, in order to facilitate the description, mainly the geographical attribute and the temporal attribute will be mainly described as the attributes common to the viewpoints.

FIG. 1 is a block diagram showing an embodiment of an information processing system according to the present invention. The information processing system 100 of the present embodiment includes an input unit 10, a geocoder (Geo-Coder) 20, a map parameter generator (Map Parameter Generator) 30, a filter parameter generator (Filter Parameter Generator) 50, and aggregation parameters. A generator (Reduce Parameter Generator) 60, a storage unit 80, a feature quantity generation function generator (Feature Descriptor Generator) 81, a feature quantity generator (Feature Generator) 82, a feature quantity selector (Feature Selector) 83, and , An output unit 90, a learning unit 91, and a prediction unit 92.

The input unit 10 acquires a first table and a second table. Since the input unit 10 acquires each table, the input unit 10 can be referred to as table acquisition means. The input unit 10 may acquire a plurality of second tables. For example, when the storage unit 80 stores the first table and the second table, the input unit 10 may acquire the first table and the second table from the storage unit 80. Also, the input unit 10 may obtain the first table and the second table from another system or storage unit via a communication network (not shown).

For example, when geographical viewpoints are common, the input unit 10 acquires a first table including the prediction target and the first geographical attribute, and a second table including the second geographical attribute. It is also good. Also, for example, when the temporal viewpoint is common, the input unit 10 acquires the first table including the prediction target and the first temporal attribute, and the second table including the second temporal attribute. You may In addition, the input unit 10 may acquire a first table including a prediction target and a first character string attribute, and a second table including a second character string attribute. A first table containing the first structural attribute and a second table containing the second structural attribute may be obtained. The structural attributes will be described later.

Furthermore, the input unit 10 calculates a similarity between the first attribute and the second attribute (hereinafter referred to as a similarity function) and the degree of similarity with respect to the function of the first attribute. A condition for determining whether the value and the value of the second attribute are similar (hereinafter, also referred to as a condition for the degree of similarity) is accepted. The similarity function may be represented by a mathematical expression or may be represented as a parameter. Further, the condition for the similarity may be represented by a threshold (hereinafter simply referred to as a threshold for similarity) for determining the presence or absence of the similarity based on the degree of the relationship, and the similarity according to the parameter etc. It may be expressed by an expression that outputs whether or not.

For example, when geographical viewpoints are common, the input unit 10 may receive a geographical relationship as a similarity function, and receive a threshold of similarity indicating the degree of the geographical relationship as a condition. That is, when the first attribute and the second attribute are geographical attributes, for example, the similarity function is defined as a function that calculates the similarity higher as the distance is closer.

Also, for example, when the temporal viewpoints are common, the input unit 10 may receive the temporal relationship as the similarity function, and may receive the threshold of the similarity indicating the degree of the temporal relationship as the condition. That is, when the first attribute and the second attribute are temporal attributes, the similarity function is defined as, for example, a function that calculates the similarity higher as the difference in time is smaller.

In addition, when the viewpoint of the character string is common, the input unit 10 may receive the relationship of the character string as the similarity function, and may receive the threshold value of the similarity indicating the degree of the relationship of the character string as a condition. Specifically, when the first attribute and the second attribute are character string attributes, for example, the similarity function is defined as a function that calculates the degree of similarity higher as the degree of matching between the two texts is higher. The text similarity includes, for example, morpheme Simpson coefficients.

Define morph (a) as a set of morphemes contained in the text string a. For example, the following four text strings indicating addresses are represented by morphological analysis as a set of forms as follows, respectively.
・ Morph ('Kawasaki City Nakahara Ward') = {'Kawasaki', 'City', 'Nakahara', 'ku'}
・ Morph ('Kanagawa Kawasaki City Nakahara Ward') = {'Kanagawa', 'Kenken', 'Kawasaki', 'City', 'Nakahara', 'Ku'}
・ Morph ('Kawasaki Kawasaki City's Yuki Ward') = {'Kanagawa', 'Prefecture', 'Kawasaki', 'city', 'Yuki', 'Ku'}
・ Morph ('Konan Yokohama-shi Konan-ku') = {'Kanagawa', 'Kenken', 'Yokohama', 'City', 'Konan', 'Ku'}

The function textSim (a, b) for calculating the degree of similarity between the text string a and the text string b can be defined by Equation 1 shown below.

In this case, the similarity between the text strings of the addresses exemplified above is calculated as follows.

・ TextSim ('Kawasaki City Nakahara Ward', 'Kanagawa Prefecture Kawasaki City Nakahara Ward') = 4/4 = 1.0
・ TextSim ('Kawasaki City Nakahara Ward', 'Kanagawa Prefecture Kawasaki City Kawasaki Ward') = 3/4 = 0.75
・ TextSim ('Kawasaki City Nakahara Ward', 'Konan Yokohama City Yokohama Konan Ward') = 2/4 = 0.5

In addition, when structural viewpoints are in common, the input unit 10 may receive a structural relationship as a similarity function, and may receive a threshold of similarity indicating the degree of the structural relationship as a condition. Hereinafter, a character string in which information of the tree structure, such as an address and a directory structure of a file, is expressed by "/" is defined as a path character string. For example, the address "Kanagawa Prefecture Kawasaki City" is expressed as "/ Kanagawa Prefecture / Kawasaki City" in the pass string. Also, for example, the directory structure “news → economy → bigdata” is expressed as “news / economy / bigdata” in the path string.

When the first attribute and the second attribute are structural attributes defined by the above-described path string, the similarity function is, for example, a function that calculates the higher the degree of similarity as the distance between the two path strings is closer. It is defined. As a distance function of the path string, for example, the minimum value of the distance to the lowest common ancestor (LCA) can be mentioned.

The lowest common ancestor node is the same node that appears first when traversing from the lowermost node represented by the two paths to the upper (ancestor) direction. Also, the distance to the lowest common ancestor node is the number of nodes when the lowest node is followed from the lowest common ancestor node.

For example, assume that there are two path strings '/ a / b / c' and '/ a / b / z'. In this case, the lowest common ancestor node of the two paths is '/ a / b'. Also, the distance from '/ a / b / c' to '/ a / b' is 1, and the distance from '/ a / b / z' to '/ a / b' is also 1.

Also, for example, it is assumed that there are two path strings '/ a / b / c' and '/ a / d / e / z'. In this case, the lowest common ancestor node of the two paths is '/ a'. Also, the distance from '/ a / b / c' to '/ a' is 2, and the distance from '/ a / d / e / z' to '/ a' is 3.

Assuming that the function representing the distance of the path character string is pathDis (x, y), the distance of the above-described path character string is calculated as follows.

PathDis ('/ a / b / c', '/ a / b / z') = 1
PathDis ('/ a / b / c', '/ a / d / e / z') = 2

FIG. 2 is an explanatory view showing an example of a configuration file (hereinafter referred to as a configuration file). The example shown in FIG. 2 indicates that the condition for the similarity function and the similarity is set in a configuration file (hereinafter referred to as a configuration file). The input unit 10 may receive this configuration file.

The C1 portion of the configuration file illustrated in FIG. 2 indicates conditions for the similarity function and the similarity. The C2 to C4 portions of the configuration file will be described later. In the C1 part, the former part (before the colon) has the data type of the first attribute (more specifically, the analysis data type) and the data type of the second attribute (more specifically, the analysis data type) Show the correspondence with. Also, the latter part (after the colon) indicates the similarity function and the condition (threshold of similarity). Each content will be described in detail below.

The "Point-Point" row in the C1 portion defines a geographical relationship representing the distance between the first geographical attribute represented by the point and the second geographical attribute represented by the point.

“DistanceMap” is a map function that defines the degree of geographical relationship, and includes a distance threshold as a parameter. The three parameters in the DistanceMap function indicate “start value”, “end value”, and “interval” (of the threshold applied from the start value to the end value) in order. Assuming that the unit of distance is km (“DistanceMap”, 1, 3, 1) illustrated in FIG. 2, three threshold values “distance within 1 km”, “distance within 2 km”, “distance within 3 km” Indicates to apply to the function.

“KNearestMap” is a map function that defines the degree of geographical relationship, and includes, as a parameter, a threshold of the number of pieces of geographical information in proximity. Similarly, the three parameters in the KNearestMap function indicate “start value”, “end value”, and “interval” (of the threshold applied from the start value to the end value). The number of pieces of geographical information adjacent to each other as illustrated in FIG. 2 (“KNearest Map”, 3, 5, 1) functions as three thresholds “within three”, “within four”, and “within five”. Indicates that it applies to

“SameCityMap” is a map function that defines the degree of geographical relationship, and is a function that determines whether two points are included in the same area. Although the SameCityMap function does not include parameters, it is determined whether it is included in the same area based on the area information defining the area. Area information is predefined.

The "Point-Area" row in the C1 portion defines a geographical relationship that represents an inclusive relationship between the first geographic attribute represented by the point and the second geographic attribute represented by the region.

"InclusionMap" is a map function that defines the degree of geographical relationship, and determines whether the first geographical attribute represented by a point is included in the second geographical attribute represented by a region. It is a function. Note that InclusionMap does not include parameters.

Also, in the "Point-Area" line, "KNearestMap" is defined. The content of the KNearestMap function is similar to the KNearestMap function in "Point-Point".

The "Area-Area" row in the C1 portion defines a geographical relationship that represents the cross-relationship between the first geographic attribute represented by the region and the second geographic attribute represented by the region.

“IntersectMap” is a map function that defines the degree of geographical relationship, and determines whether the first geographical attribute represented by the area intersects with the second geographical attribute represented by the area It is a function. Note that IntersectMap does not include parameters.

As described above, the first geographical data type and the second geographical data type may be the same geographical data type as each other, or may be different geographical data types. Also, the first geographical data type is a type of data that can identify geography with point information, and the second geographical data type is a type of data that can identify geography with range information. It is also good.

The line "TimeStamp-TimeStamp" in the C1 section defines a temporal relationship that represents the difference between the first temporal attribute and the second temporal attribute.

“TimeDiffMap” is a map function that defines the degree of temporal relationship, and includes a threshold of time difference as a parameter. Similarly, the three parameters in the TimeDiffMap function indicate “start value”, “end value”, and “interval” (of the threshold applied from the start value to the end value). In the example illustrated in FIG. 2 (“TimeDiffMap”, 30, 60, 30), when the unit of time is a minute, two threshold values “a time difference is within 30 minutes” and “a time difference is within 60 minutes” Indicates to apply to the function.

The line "Text-Text" in the C1 portion defines the correspondence between the first attribute representing a character string and the second attribute representing a character string. “ExactMap” is a function that determines whether the attribute represented by the character string matches.

Also, in the “Text-Text” line, a similar relationship between a first attribute representing a character string and a second attribute representing a character string may be defined. Specifically, a map function "textSimMap" that defines the degree of relation of character strings may be set in the "Text-Text" line. “TextSimMap” is a map function that defines the degree of relation of character strings, and includes a threshold of similarity as a parameter. The textSimMap function, like the DistanceMap function, has three parameters, which respectively indicate "start value", "end value", and "interval" (of the threshold applied from the start value to the end value).

For example, it is assumed that the textSimMap function is used to define [(“textSimMap”, 0.8, 1.0, 0.1], which means that “the similarity is 0.8 or more”, “the similarity is 0. 9 shows that three threshold values of “9 or more” and “similarity is 1.0 (or more)” are applied to the function.

In addition, the setting method of the similarity function and the threshold value of similarity is not limited to the content illustrated to C1 part of FIG. In the configuration file, for example, a structural relationship "Path-Path" representing a distance between a first structural attribute represented by a path string and a second structural attribute represented by a path string is defined. It is also good.

Specifically, in the “Path-Path” line, a map function “pathDisMap” may be set which defines the degree of structural relationship. “PathDisMap” is a map function that defines the degree of structural relationship, and includes a distance threshold as a parameter. Like the DistanceMap function, the pathDisMap function has three parameters, which respectively indicate "start value", "end value", and "interval" (of the threshold applied from the start value to the end value).

For example, it is assumed that the pathDisMap function is used to define [(“pathDisMap”, 1, 3, 1]. This means that “distance is 1 or less”, “distance is 2 or less” and “distance is 3 or less” It shows applying three threshold values to a function.

When the input unit 10 receives the configuration file illustrated in FIG. 2, the map parameter generator 30, which will be described later, is a combination for combining a record included in the first table and a record included in the second table. Generate conditions (map parameters).

The input unit 10 may also receive the attribute of the data indicated by each column of the table.

The geocoder 20 converts data of an attribute represented by a character string. For example, when the data of the geographical attribute is represented by a character string, the geocoder 20 converts the character string into data of point, polygon or multi-polygon. Note that when there is no need to convert data, the information processing system 100 may not include the geocoder 20.

FIG. 3 is an explanatory view showing an example of processing for converting data. In the example illustrated in FIG. 3, it is assumed that a table adt1 in which an analysis data type for each column is defined and a table adt2 in which correspondence to convert an analysis data type to a data type is defined are acquired in advance.

In this situation, it is assumed that the input unit 10 acquires the target table T, the source table S1 and the source table S2 illustrated in FIG. 3. The analysis data type of the "Pickup_location" column of the source table S2 is Point when referring to the table adt1, and there is no need for conversion. On the other hand, the analysis data type of the "community" column of the source table S1 is "TownAddress" when referring to the table adt1, and when referring to the table adt2, it is necessary to convert it to the data type Polygon. Therefore, the geocoder 20 converts the data included in the "community" column of the source table S1 so as to be represented by a polygon area. For example, area information capable of specifying an area as a polygon is predetermined according to the contents of "community", and the geocoder 20 converts data so that the data type becomes Polygon based on the area information. It is also good.

The map parameter generator 30, the filter parameter generator 50, and the aggregation parameter generator 60 are features for generating a feature that is a variable that can be influenced by the feature quantity generation function generator 81 described later. Generates parameters to be used when generating a quantity generation function.

In the following description, the feature amount means the content of the feature itself (for example, "population", "position", etc.). In addition, feature quantity vectors (in the case of a plurality of cases) in which specific data are fitted to feature quantities (for example, population = "8112", position = "(-73.965, 40.724)", etc.) It is described as a feature amount table).

Further, the feature quantities generated by the feature quantity generator 82 described later become candidates for explanatory variables when generating a model using machine learning. In other words, by using the feature quantity generation function generated in the present embodiment, it is possible to automatically generate candidate explanatory variables when generating a model using machine learning.

FIG. 4 is an explanatory view showing an example of the relationship between each parameter and the first table and the second table.

The parameters generated by the filter parameter generator 50 are parameters representing extraction conditions of the rows included in the second table. Hereinafter, this parameter may be referred to as a filter parameter, and a process of extracting a row from the second table based on the filter parameter may be described as “filter”. Also, this list of extraction conditions may be described as "F list". The extraction condition is arbitrary, and for example, a condition to judge whether it is the same (large or small) as the value of the designated column.

The parameters generated by the aggregation parameter generator 60 are parameters representing an aggregation method of aggregating data of each row included in the second table for each objective variable. Generally, the rows in the first table correspond to the rows in the second table in many cases, so the rows are aggregated as a result. Aggregation information may be defined as an aggregation function for columns of the source table (second table).

The aggregation method is optional, and includes, for example, the total number of columns, maximum value, minimum value, average value, median value, variance, and the like. Also, the total number of columns may be aggregated in terms of excluding duplicate data or not excluding duplicate data.

Hereinafter, this parameter may be described as an aggregation parameter, and a process of aggregating data of each column may be described as “reduce” by a method indicated by the aggregation parameter. In particular, the process of aggregating geographical information may be described as "Geo-reduce". Also, the list of aggregation processing may be described as "R list". The details of the process of aggregating geographical information will be described later.

The parameters generated by the map parameter generator 30 are parameters representing the corresponding conditions of the first table and the columns of the second table. Hereinafter, this parameter may be referred to as a map parameter, and the process of associating the columns of each table based on the map parameter may be referred to as “map”. Also, the list of correspondence conditions may be described as "M list". In particular, the process of associating geographical information may be described as "Geo-map". Also, the mapping of the columns of each table by map can be said to be a join of a plurality of tables into one table in the mapped columns. The details of the process of associating geographical information are also described later.

The map parameter generator 30 includes a geomap generator (GeoMap Generator) 40, a time difference map generator (TimeDiff Map Generator) 31, a map generator (Exact Map Generator) 32, and an attribute specifying unit 33. The map parameter generator 30 (more specifically, each generator included in the map parameter generator 30) sets the condition calculated by the similarity calculated by the value of the first attribute and the value of the second attribute. A join condition is generated to combine the record of the first table including the value of the first attribute that satisfies the condition and the record of the second table including the value of the second attribute. To satisfy the condition means, for example, that the similarity is equal to or less than or equal to a threshold, or included in a predetermined range.

The geomap generator 40 generates a parameter representing a correspondence condition between columns including geographical attributes of the first table and the second table. The geomap generator 40 includes a distance map generator (distance map generator) 41, an inclusion map generator (inclusion map generator) 42, an overlap map generator (overlap map generator) 43, and the same area map generator (SameArea). Map Generator (44).

The geomap generator 40 (more specifically, each generator included in the geomap generator 40) determines that the relationship between the value of the first geographical attribute and the value of the second geographical attribute is geographically A join condition (map parameter) for joining the record included in the first table and the record included in the second table, which satisfies the degree of the relationship, is generated. The processing of each generator will be described in detail below.

The distance map generator 41 generates map parameters when it receives a similarity function and a condition (for example, a threshold of similarity) for associating the first table with the second table based on the closeness of the distance. Do. The example shown in FIG. 2 corresponds to the case where at least one of the DistanceMap function and the KNearestMap function is set in the configuration file.

The distance map generator 41 includes the records included in the first table and the second table such that the distance between the value of the first geographical attribute and the value of the second geographical attribute is within a threshold. Generate map parameters to combine with the record to be recorded.

FIG. 5 is an explanatory view showing an example of processing for generating map parameters based on distances. The example shown in FIG. 5 shows the case where one target table T and one source table S2 are acquired. The target table T illustrated in FIG. 5 is a table including data representing the number of passengers (pickup_number) at five locations on January 8, 2015 at 22:00. Moreover, source table S2 illustrated in FIG. 5 is a table which matches and records the number of passengers, the movement distance, and the landing position of a passenger in each time.

For example, in the case of the DistanceMap function illustrated in FIG. 2, the distance map generator 41 has a distance between the position indicated by the value of the first geographical attribute and the position indicated by the value of the second geographical attribute within 1 km. A parameter that associates each record of the target table T with the record of the source table S2 is generated. Furthermore, the distance map generator 41 targets the records of the source table S2 in which the distance between the position indicated by the value of the first geographical attribute and the position indicated by the value of the second geographical attribute is within 2 km and 3 km. The parameter which matches each record of table T is generated, respectively.

In the example shown in FIG. 5, the attribute of the "target_location" column of the target table T is a first geographical attribute, and the attribute of the "Pickup_location" column of the source table S2 is a second geographical attribute. These two columns are associated. A row to be associated with the first table and the second table may be specified in advance, or may be specified by the attribute specifying unit 33 described later.

As a result, the parameter P11 illustrated in FIG. 5 is generated. As illustrated in FIG. 5, map parameters are generated based on the geographical analysis data type, and one map processing is defined based on one map parameter. The map data M11 illustrated in FIG. 5 indicates the result of associating each record of the target table T with the record of the source table S2 having a distance of 1 km or less. For example, only one record from the source table is associated with the first record of the target table. Also, for example, two records from the source table are associated with the second record of the target table.

FIG. 6 is an explanatory view showing an example of another process of generating map parameters based on distances. The target table T and source table S2 illustrated in FIG. 6 are similar to the target table T and source table S2 illustrated in FIG.

For example, in the case of the KNearestMap function illustrated in FIG. 2, the distance map generator 41 sequentially operates in the order from the closest distance between the position indicated by the value of the first geographical attribute and the position indicated by the value of the second geographical attribute. A parameter is generated which associates each record of the target table T with the record of the source table S2 within two or less. Furthermore, the distance map generator 41 sets the target table T in the records of the source table S2 in order from the closest distance between the position indicated by the value of the first geographical attribute and the position indicated by the value of the second geographical attribute. Generate a parameter that associates each record with 3 or less and 4 or less.

In the example shown in FIG. 6, the attribute of the "target_location" column of the target table T is the first geographical attribute, and the attribute of the "Pickup_location" column of the source table S2 is the second geographical attribute. These two columns are associated. A row to be associated with the first table and the second table may be specified in advance, or may be specified by the attribute specifying unit 33 described later.

As a result, the parameter P12 illustrated in FIG. 6 is generated. As illustrated in FIG. 6, map parameters are generated based on the geographical analysis data type, and one map processing is defined based on one map parameter. The map data M12 illustrated in FIG. 6 indicates the result of associating two records of the target table T with the records of the source table S2 in order of closeness. For example, for each record of the target table, the two closest records from the source table are associated.

When the area map generator 44 receives a similarity function for associating the first table with the second table based on whether the area is included in the same area, the area map generator 44 generates map parameters. The example shown in FIG. 2 corresponds to the case where the SameCityMap function is set in the configuration file.

In the same area map generator 44, the records included in the first table are included in the same area such that the position indicated by the value of the first geographical attribute and the position indicated by the value of the second geographical attribute are included in the same area. Generate map parameters to combine the records contained in the second table.

FIG. 7 is an explanatory view showing an example of a method of determining whether or not it is included in the same area. In the example shown in FIG. 7, the common area table CAT in which each area and the area of the area specified by the polygon are associated is defined in advance. Examples of common areas include countries, states, cities, autonomous regions, and cities. The common area is defined as a common area that does not overlap each other, and represents boundary information on the map. The common area table CAT may be stored, for example, in the storage unit 80.

First, based on the common area table CAT, it is determined whether two positions exist in the same area. Specifically, the area indicated by the position of the record t1 in the target table T is specified, and it is determined whether the position of the record s1 in the source table S is within the area. Hereinafter, the same processing is performed on all the records of the target table T and the source table S.

FIG. 8 is an explanatory view showing an example of processing of generating map parameters based on whether or not it is a common area. The target table T and source table S2 illustrated in FIG. 8 are similar to the target table T and source table S2 illustrated in FIG.

For example, in the case of the SameCityMap function illustrated in FIG. 2, the same area map generator 44 includes the position indicated by the value of the first geographical attribute and the position indicated by the value of the second geographical attribute in the same area. A parameter that associates the record of the source table S2 with each record of the target table T is generated.

In the example shown in FIG. 8, the attribute of the "target_location" column of the target table T is a first geographical attribute, and the attribute of the "Pickup_location" column of the source table S2 is a second geographical attribute. These two columns are associated. A row to be associated with the first table and the second table may be specified in advance, or may be specified by the attribute specifying unit 33 described later.

As a result, the parameter P13 illustrated in FIG. 8 is generated. The map data M13 illustrated in FIG. 8 indicates the result of associating the records of the source table S2 having the geographical attribute determined to be the same area with the records of the target table T. In addition, the map data M13 illustrated in FIG. 8 shows the example matched on the assumption that the point whose distance is less than 1 km is located in the same city.

The inclusion map generator 42 generates map parameters when it receives a similarity function for associating the first table with the second table based on the inclusion relation. The example shown in FIG. 2 corresponds to the case where the InclusionMap function is set in the configuration file.

The inclusion map generator 42 is configured to record the second table and the records included in the first table such that the position indicated by the value of the first geographical attribute is included in the area indicated by the value of the second geographical attribute. Generate map parameters to combine records contained in the table.

FIG. 9 is an explanatory view showing an example of processing of generating map parameters based on the inclusive relation. The target table T illustrated in FIG. 9 is similar to the target table T illustrated in FIG. Further, the source table S1 illustrated in FIG. 9 is a table that associates and records the population in each area, the number of males, and the number of people from 20 to 40 years old.

For example, in the case of the InclusionMap function illustrated in FIG. 2, the inclusion map generator 42 records the source table S1 included in the area indicated by the value of the second geographical attribute at the position indicated by the value of the first geographical attribute. Generate a parameter that associates each record of the target table T with.

In the example shown in FIG. 9, the attribute of the "target_location" column of the target table T is the first geographical attribute, and the attribute of the "community" column of the source table S1 is the second geographical attribute. These two columns are associated. A row to be associated with the first table and the second table may be specified in advance, or may be specified by the attribute specifying unit 33 described later.

As a result, the parameter P14 illustrated in FIG. 9 is generated. The map data M14 illustrated in FIG. 9 indicates the result of associating each record of the target table with the record of the source table S1 existing in the same area.

The overlap map generator 43 generates map parameters when it receives a similarity function for associating the first table with the second table based on the overlapping area. The example shown in FIG. 2 corresponds to the case where the IntersectMap function is set in the configuration file.

The overlapping map generator 43 sets the second table and the records included in the first table such that the area indicated by the value of the first geographical attribute and the area indicated by the value of the second geographical attribute overlap. Generate map parameters to combine with included records.

The time difference map generator 31 generates map parameters when it receives a similarity function and a condition (for example, a threshold of similarity) for associating the first table with the second table based on the difference in time. Do. The example shown in FIG. 2 corresponds to the case where the TimeDiffMap function is set in the configuration file.

The temporal difference map generator 31 determines whether the relationship between the value of the first temporal attribute and the value of the second temporal attribute satisfies the degree of the temporal relationship, the records included in the first table and the second Create join conditions to join records contained in the table of. In the present embodiment, the time difference map generator 31 sets the records included in the first table such that the difference between the value of the first temporal attribute and the value of the second temporal attribute is within the threshold. Generate map parameters to combine the records contained in the second table.

FIG. 10 is an explanatory drawing showing an example of processing for generating map parameters based on the difference in time. The target table T and the source table S2 illustrated in FIG. 10 are similar to the target table T and the source table S2 illustrated in FIG.

For example, in the case of the TimeDiffMap function illustrated in FIG. 2, the time difference map generator 31 determines that the difference between the value of the first temporal attribute and the value of the second geographical attribute is within 30 minutes. Generate a parameter that associates each record of the target table T with the record. Furthermore, the time difference map generator 31 associates each record of the target table T with the record of the source table S2 in which the difference between the value of the first temporal attribute and the value of the second temporal attribute is within 60 minutes. Generate parameters.

In the example shown in FIG. 10, the attribute of the "time" column of the target table T is the first temporal attribute, and the attribute of the "pickup_time" column of the source table S2 is the second temporal attribute. These two columns are associated. A row to be associated with the first table and the second table may be specified in advance, or may be specified by the attribute specifying unit 33 described later.

As a result, the parameter P15 illustrated in FIG. 10 is generated. The map data M15 illustrated in FIG. 10 shows the result of associating each record of the target table T with the record of the source table S2 in which the time difference is within 30 minutes.

When the map generator 32 receives a similarity function for associating the first table with the second table, the map generator 32 generates map parameters. In this embodiment, based on the value of an attribute that is neither a geographical attribute nor a temporal attribute, a parameter that associates a record of the target table with a record of the source table is generated.

The example shown in FIG. 2 corresponds to the case where the ExactMap function is set in the configuration file. The map generator 32 is a map for combining a record included in the first table and a record included in the second table such that the value of the first attribute matches the value of the second attribute. Generate parameters.

FIG. 11 is an explanatory view showing an example of processing of generating map parameters based on text similarity. The target table T illustrated in FIG. 11 is a table including data representing the number of passengers (pickup_number) at a certain address. In addition, the source table S illustrated in FIG. 11 is a table for recording the income average in each area.

For example, in the case of the textSimMap function described above, the map generator 32 targets the records of the source table S whose similarity between the value of the first character string attribute and the value of the second character string attribute is 0.8 or more. Generate a parameter that associates each record of table T. Furthermore, the map generator 32 sets the target table T to records of the source table S in which the similarity between the value of the first character string attribute and the value of the second character string attribute is 0.9 or more and 1.0 or more. Generates parameters to associate each record of.

In the example shown in FIG. 11, the attribute of the "address" column of the target table T is registered as the first character string attribute, and the attribute of the "address" column of the source table S is registered as the second character string attribute. I assume. Then, these two columns are associated. As a result, the parameter P16 illustrated in FIG. 11 is generated.

The map data M illustrated in FIG. 11 indicates the result of associating each record of the target table T with the record of the source table S having a similarity of 0.8 or more. For example, only one record from the source table is associated with the first record of the target table.

FIG. 12 is an explanatory view showing an example of a process of generating map parameters based on the structural similarity. The target table T illustrated in FIG. 12 is a table including data representing the number of accesses (access_number) to the Web page identified by a certain URL. Also, the source table S illustrated in FIG. 12 is a table for recording the number of accesses (access_number) of the last month of the Web page identified by a certain URL.

For example, in the case of the pathDisMap function described above, the map generator 32 sets the target table T to a record of the source table S in which the distance between the value of the first structural attribute and the value of the second structural attribute is 1 or less. Generate a parameter that associates each record. Furthermore, the map generator 32 associates each record of the target table T with the record of the source table S in which the distance between the value of the first structural attribute and the value of the second structural attribute is 2 or less and 3 or less. Generate each parameter.

In the example shown in FIG. 12, the attribute of the "URL" column of the target table T is registered as the first structural attribute, and the attribute of the "URL" column of the source table S is registered as the second structural attribute. I assume. Then, these two columns are associated. As a result, the parameter P17 illustrated in FIG. 12 is generated.

The map data M illustrated in FIG. 12 indicates the result of associating each record of the target table T with the record of the source table S having a similarity of 1 or less. For example, only one record from the source table is associated with the first record of the target table.

The attribute specifying unit 33 specifies an attribute having a common viewpoint in the first table and the second table. Specifically, the attribute specifying unit 33 specifies the same attribute as the attribute of the data indicated by each column of the first table and the attribute of the data indicated by each column of the second table. For example, in the case of the geographical data type, the attribute specifying unit 33 specifies the first geographical attribute having the same data type as the first geographical data type from the first table, and the second geographical attribute. A second geographic attribute having the same data type as the information data type may be identified from the second table. By doing this, it is possible to identify columns having geographical data types from each table. In addition, the attribute specifying unit 33 may specify the attributes of the columns of the first table and the second table from the information of the attributes of the column input to the input unit 10.

The map parameter generator 30 (more specifically, each generator included in the map parameter generator 30) includes a first geographical (temporal) attribute which is a target of determination of the geographical (temporal) relationship. The storage unit 80 also stores parameters including the first table row and the second table row including the second geographical (temporal) attribute and the degree of the geographical (temporal) relationship Good. For example, the map parameter generator 30 may store the parameter P11 illustrated in FIG. 5 or the parameter P15 illustrated in FIG. 10 in the storage unit 80.

FIG. 13 is an explanatory view showing an example of the generated map parameter. As shown in the example described above, the input unit 10 receives the target table T, the source table S1 and the source table S2 illustrated in FIG. 13, and the C1 portion of the configuration file illustrated in FIG. The map parameter P16 has the attribute of the "target_location" column of the target table T as the first geographical attribute, and the attribute of the "community" column of the source table S1 as the second geographical attribute, based on the KNearestMap function. It is an example of the parameter generated. The map parameter generator 30 (more specifically, each generator included in the map parameter generator 30) generates 13 map parameters P11 to 16 illustrated in FIG. 13 from these pieces of information.

The filter parameter generator 50 includes a filter generator (Exact Filter Generator) 51. The filter generator 51 generates filter parameters in which the columns of the second table are associated with the extraction conditions applied to the columns.

The method of generating the filter parameters is arbitrary. The filter generator 51 may generate filter parameters based on, for example, the information defined in the C2 portion of the configuration file illustrated in FIG. Alternatively, the extraction condition may be stored in advance in the storage unit 80, and the filter generator 51 may read the extraction condition to generate a filter parameter.

Furthermore, the filter generator 51 may combine a plurality of extraction conditions to generate additional extraction conditions. Also, the number of combinations of extraction conditions is arbitrary. The input unit 10 may receive this combined maximum number, for example. For example, as illustrated in FIG. 2, a parameter (“max_combination_filter_length”) indicating the maximum number of combinations may be set in the C4 portion of the configuration file.

The aggregation parameter generator 60 (more specifically, each generator included in the aggregation parameter generator 60) generates a parameter representing a method of aggregating data of each row included in the second table. The aggregation parameter generator 60 includes a geo aggregation generator (GeoReduce Generator) 70 and a numerical aggregation generator (Numeric Reduce Generator) 61.

The geo-aggregate generator 70 (more specifically, each generator included in the geo-aggregate generator 70) is a method of aggregating data of each row by the value of the column including the geographical attribute included in the second table. Generate aggregate parameters to represent. Specifically, the geo aggregation generator 70 calculates the statistical value of the value of the geographical attribute based on the designated aggregation method.

The method of specifying the aggregation method is arbitrary. For example, the input unit 10 may receive designation of the aggregation method. Specifically, as exemplified in the C3 portion of the configuration file in FIG. 2, the aggregation method is defined according to the analysis data type of geographical attribute, and the aggregation parameter is generated according to the defined aggregation method. Good. Each content will be described in detail below.

The "Point" row in the C3 portion defines an aggregation method when the second geographical attribute (more specifically, the geographical data type) is represented by Point.

(“Sum”, “distance”) are a value of the first geographical attribute and a value of the second geographical attribute among the records of the second table associated with the records of the first table Define the aggregation method to calculate the sum of the distances calculated based on

("Avg", "distance") are a value of the first geographical attribute and a value of the second geographical attribute among the records of the second table associated with the records of the first table Define an aggregation method that calculates the average of distances calculated based on

("Count") defines an aggregation method for calculating, as a statistical value, the number of records of the second table associated with each record (that is, the target variable) of the first table.

The "Area" line in the C3 portion defines an aggregation method when the second geographical attribute (more specifically, the geographical data type) is represented by an area.

("Sum", "areaSize") is calculated as the total value of the size of the area of the second geographical attribute among the respective records of the second table associated with the records of the first table as a statistical value Define the method of aggregation.

("Avg", "areaSize") calculates, as a statistical value, the average of the size of the area of the second geographic attribute among the records of the second table associated with the records of the first table Define the method of aggregation.

The geo consolidation generator 70 includes a point consolidation generator (Point Reduce Generator) 71 and an area consolidation generator (Area Reduce Generator) 72.

The point aggregation generator 71 generates an aggregation parameter for calculating a distance statistic calculated based on the value of the first geographical attribute and the value of the second geographical attribute. Here, the records of the second table targeted here are records respectively associated with the records of the first table. In the case of geographical attributes, as described above, records that satisfy certain conditions, such as the value of the first geographical attribute and the value of the second geographical attribute either matching or within a certain range Are associated with each other. Thus, the point aggregation generator 71 determines that the value of the first geographic attribute and the second condition satisfy the condition when the value of the second geographic attribute with respect to the value of the first geographical attribute satisfies a predetermined condition. An aggregation parameter is generated to calculate distance statistics based on the value of the geographical attribute. The calculated statistical value is used as a feature value.

For example, in the point aggregation generator 71, at least one of (“sum”, “distance”), (“avg”, “distance”) and (“count”) illustrated in FIG. 2 is set in the configuration file. In some cases, aggregate parameters may be generated to calculate distance statistics.

FIG. 14 is an explanatory diagram of an example of a process of generating an aggregation parameter for calculating a distance statistic. In the example shown in FIG. 14, three types of consolidation methods are set in the configuration file. Therefore, the point aggregation generator 71 calculates an aggregation parameter that calculates the sum and average of distances between records of the source table and a record of the target table, and an aggregation parameter that calculates the number of records of the associated source table. calculate. For example, as in the aggregation list P21 illustrated in FIG. 14, the point aggregation generator 71 associates the column names of the source table to be aggregated, the column names of the target table to be associated, the aggregation content (distance), and the aggregation parameter May be generated.

Aggregated data R21 illustrated in FIG. 14 shows the result of aggregating map data M11 based on the aggregation parameter for calculating the sum of distances.

The area aggregation generator 72 generates an aggregation parameter for calculating the statistical value of the area calculated based on the value of the second geographical attribute. Similar to the point aggregation generator 71, the records in the second table targeted here are records respectively associated with the records in the first table.

For example, in the area aggregation generator 72, at least one of ("sum", "areaSize") and ("avg", "areaSize") and ("count") illustrated in FIG. 2 is set in the configuration file. In some cases, aggregation parameters may be generated to calculate region statistics.

FIG. 15 is an explanatory diagram of an example of a process of generating an aggregation parameter for calculating a region statistical value. In the example shown in FIG. 15, three types of consolidation methods are set in the configuration file. Therefore, the area aggregation generator 72 calculates an aggregation parameter for calculating the sum and average of the areas of the records of the source table associated with each record of the target table, and the aggregation for calculating the number of records of the associated source table. Calculate the parameters. The area aggregation generator 72 may generate an aggregation parameter in which the column name of the source table to be aggregated, the aggregation content (area), and the aggregation function are associated, for example, as in the aggregation list P22 illustrated in FIG.

Aggregated data R22 illustrated in FIG. 15 shows the result of aggregating the map data M14 based on the aggregation parameter for calculating the sum of the areas.

The numerical aggregation generator 61 generates an aggregation parameter representing a method of aggregating data of each row by a value of a column including an attribute (Nemuric) attribute (hereinafter referred to as a numerical attribute) included in the second table. . Specifically, the numerical aggregation generator 61 calculates statistical values of numerical values based on the designated aggregation method.

The method of specifying the aggregation method is arbitrary. Similar to the geo aggregation generator 70, for example, the input unit 10 may receive specification of the aggregation method. Specifically, as exemplified in the C3 portion of the configuration file of FIG. 2, an aggregation method for numerical attributes may be defined, and an aggregation parameter may be generated according to the defined aggregation method. In the example shown in FIG. 2, designation is made to generate an aggregation parameter for calculating the sum and average of the columns of numerical attributes.

The aggregation parameter generator 60 (more specifically, each generator included in the aggregation parameter generator 60) may store the generated aggregation parameter in the storage unit 80. FIG. 16 is an explanatory diagram of an example of the generated aggregation parameter. As shown in the example described above, the input unit 10 receives the target table T, the source table S1 and the source table S2 illustrated in FIG. 16, and the C3 portion of the configuration file illustrated in FIG.

The aggregation parameter P23 is an example of an aggregation parameter for a column of numerical attributes of the source table S2. Also, the aggregation parameter P24 is an example of an aggregation parameter for a column of numerical attributes of the source table S1. The aggregation parameter generator 60 (more specifically, each generator included in the aggregation parameter generator 60) generates 16 map parameters P21 to 24 illustrated in FIG. 16 from these pieces of information.

The feature quantity generation function generator 81 generates a feature quantity generation function for generating the above-mentioned feature quantity from the first table and the second table. Specifically, the feature quantity generation function generator 81 generates a feature quantity generation function using (combining) the combination condition (map parameter) and the aggregation condition (aggregation parameter) described above. Further, the feature quantity generation function generator 81 may generate a feature quantity generation function using (in combination with) the extraction condition (filter parameter) in addition to the combination condition and the aggregation condition.

Further, in the present embodiment, the feature quantity generation function generator 81 is a map in which a map parameter for geographical attribute and a map parameter for temporal attribute are combined in advance among combining conditions (map parameters). Parameters may be generated. The feature quantity generation function generator 81 is, for example, a map parameter for the geographical attribute when “True” is set to the parameter “time_spatial_map_combination” as shown in the C4 part of the configuration file illustrated in FIG. 2. It may be determined to combine with the map parameters for temporal attributes.

FIG. 17 is an explanatory view showing an example in which map parameters are combined. For example, it is assumed that there are six map parameters P11 and P12 for geographical attributes and two map parameters P15 for temporal attributes. At this time, the feature quantity generation function generator 81 may generate a new map parameter P31 by combining the map parameter for geographical attribute and the map parameter for temporal attribute one by one. . In the case of the example shown in FIG. 17, new 6 × 2 = 12 map parameters are generated.

Hereinafter, the procedure of the feature quantity generation function generator 81 generating a feature quantity generation function will be specifically described. Here, it is assumed that the target table T and source tables S1 and S2 illustrated in FIG. 13 are input. Further, the variable to be predicted (target variable) is a variable that represents the number of passengers (pickup_number) included in the target table T.

FIG. 18 is an explanatory view showing an example of a method of generating a feature quantity generation function by combining parameters. FIG. 18A shows a combination example of generating a feature quantity generation function for generating a feature quantity from the target table T and the source table S1. Further, FIG. 18B shows a combination example of generating a feature quantity generation function for generating a feature quantity from the target table T and the source table S2. In the example shown in FIG. 18 (b), it is assumed that map parameters in which map parameters for geographical attributes and map parameters for temporal attributes are combined are used.

In the example shown in FIG. 18A, map parameters of 4 and aggregation parameters of 9 are generated. The feature value generation function generator 81 selects one parameter from each of the map parameters and the aggregation parameter, and generates a combination of each parameter. In this example, 4 × 9 = 36 combinations are generated based on each parameter. When a filter parameter is generated, the feature value generation function generator 81 selects one parameter each from the map parameter, the filter parameter, and the aggregation parameter, and generates a combination of each parameter.

Also in the example shown in FIG. 18B, 14 map parameters and 7 aggregation parameters are generated. The feature value generation function generator 81 selects one parameter from each of the map parameters and the aggregation parameter, and generates a combination of each parameter. In this example, 14 × 7 = 94 combinations are generated based on each parameter. From the above, a total of 36 + 94 = 130 parameter combinations are generated.

Next, the feature quantity generation function generator 81 generates a feature quantity generation function based on the generated combination. Specifically, the feature quantity generation function generator 81 converts the parameters included in the generated combination into a form of a query language for performing manipulation and definition of table data. The feature value generation function generator 81 may use, for example, SQL as a query language.

At this time, the feature quantity generation function generator 81 may generate each feature quantity generation function by applying each parameter to a template for generating an SQL statement. Specifically, a template for generating an SQL statement by fitting each parameter is prepared in advance, and the feature quantity generation function generator 81 sequentially applies each parameter included in the generated combination to the template. You may generate SQL statements. In this case, the feature quantity generation function is defined as a SQL statement, and each selected parameter corresponds to a parameter for generating the SQL statement.

Defining feature quantities using combinations of these parameters makes it possible to express many types of feature quantity generation functions as simple element combinations. Therefore, multiple table data can be used to efficiently generate a large number of feature amount candidates. For example, in the case of the above-described example, 130 types of feature values can be easily generated simply by generating 4 map parameters and 9 aggregation parameters, 14 map parameters and 7 aggregation parameters. . Further, since the definition of each parameter once generated can be reused, the effect of reducing the number of man-hours for generating the feature quantity generation function can also be obtained.

The feature amount generator 82 generates a feature amount using a feature amount generation function. For example, it is assumed that the feature amount generation function includes a parameter for calculating the above-described distance statistical value. In this case, the feature amount generator 82 performs the operation of aggregating the records of the second table satisfying the predetermined condition for each record of the first geographical attribute based on the feature amount generation function, thereby obtaining the distance. The statistical value of may be calculated.

Specifically, the feature quantity generator 82 performs, as an operation of aggregating the records of the second table, the geographical attribute of the second table satisfying the predetermined condition with respect to each record of the first geographical attribute. The sum and / or the average of the distances may be calculated. Then, the feature quantity generator 82 may add at least one of the sum and the average of the calculated distances as the feature quantity to the attribute of the first table.

In addition, the feature quantity generator 82 is a record of the geographical attribute of the second table which satisfies a predetermined condition for each record of the first geographical attribute as an operation of aggregating the records of the second table. The number may be calculated. Then, the feature quantity generator 82 may add the calculated number of records as the feature quantity to the attribute of the first table.

As described above, since the feature quantity generator 82 also performs processing for adding the generated feature quantity to the attribute of the first table, the feature quantity generator 82 can be called attribute addition means. In addition, the feature quantities generated by the feature quantity generator 82 can also be said to be candidates for feature quantities because they become candidates when the feature quantity selector 83 described later selects feature quantities.

In the present embodiment, the case where the feature quantity generator 82 generates feature quantity candidates using the feature quantity generation function has been described. However, the feature amount generator 82 may directly generate feature amount candidates from the first table and the second table using the combination condition and the aggregation condition using the similarity function. As described above, the join condition is a record of the first table including the value of the first attribute in which the degree of similarity calculated by the value of the first attribute and the value of the second attribute satisfies the condition; It is a condition for combining the record of the second table including the value of the second attribute. Further, the aggregation condition is a condition represented by an aggregation method for a plurality of records in the second table and a column that is an object of the aggregation.

For example, when there are a plurality of combination conditions and aggregation conditions, the feature amount generator 82 may generate a number of feature amounts combining a plurality of combination conditions and a plurality of aggregation conditions. By combining the combination condition and the aggregation condition, the same effect as the process of generating the feature quantity generation function by the feature quantity generation function generator 81 described above can be obtained.

The feature amount selector 83 selects a feature amount optimal for prediction from the generated feature amounts. In addition, the method of feature-value selection is arbitrary. The feature quantity selector 83 may select feature quantities using, for example, L1 regularization. However, the algorithm used to select feature quantities is not limited to L1 regularization. The feature quantity selector 83 may select the feature quantity most suitable for prediction according to the algorithm used for selecting the feature quantity.

The output unit 90 outputs the generated feature amount. The output unit 90 may output only the feature amount selected by the feature amount selector 83, or may output all the feature amounts generated by the feature amount generator 82.

The learning unit 91 learns a prediction model using the generated feature amount. The learning unit 91 learns, for example, a prediction model using the added attribute as a feature amount. Specifically, the learning unit 91 applies the data of the first table and the second table to the generated feature amount to generate training data. Then, the learning unit 91 learns a model that predicts the value of the prediction target, using the generated feature quantity as an explanatory variable candidate. In addition, the learning method of a model is arbitrary.

The prediction unit 92 performs prediction using the model learned by the learning unit 91. Specifically, the prediction unit 92 applies the data of the first table and the second table to the generated feature amount to generate data for prediction. Then, the prediction unit 92 applies the generated data for prediction to the learned model to obtain a prediction result.

Input unit 10, Geocoder 20, Map parameter generator 30, Filter parameter generator 50, Aggregated parameter generator 60, Feature quantity generation function generator 81, Feature quantity generator 82, Feature quantity selector 83, an output unit 90, a learning unit 91, and a prediction unit 92, a processor (for example, a central processing unit (CPU), a graphics processing unit (GPU), an FPGA (for example) of a computer operating according to a program (information processing program)). field-programmable gate array)). The map parameter generator 30 more specifically includes the geomap generator 40 (more specifically, the distance map generator 41, the inclusion map generator 42, the overlap map generator 43, and the same area map generator 44). , A time difference map generator 31, a map generator 32, and an attribute specifying unit 33. Also, the aggregation parameter generator 60 is realized by the geo aggregation generator 70 (more specifically, the point aggregation generator 71 and the area aggregation generator 72) and the numerical aggregation generator 61.

For example, the program is stored in the storage unit 80, and the processor reads the program, and according to the program, the input unit 10, the geocoder 20, the map parameter generator 30, the filter parameter generator 50, the aggregation parameter generator 60, the feature value The generation function generator 81, the feature quantity generator 82, the feature quantity selector 83, the output unit 90, the learning unit 91, and the prediction unit 92 may operate. In addition, the functions of the information processing system may be provided in the form of Software as a Service (SaaS).

Input unit 10, Geocoder 20, Map parameter generator 30, Filter parameter generator 50, Aggregated parameter generator 60, Feature quantity generation function generator 81, Feature quantity generator 82, Feature quantity selector Each of 83, the output unit 90, the learning unit 91, and the prediction unit 92 may be realized by dedicated hardware. In addition, part or all of each component of each device may be realized by a general purpose or dedicated circuit, a processor, or the like, or a combination thereof. These may be configured by a single chip or may be configured by a plurality of chips connected via a bus. A part or all of each component of each device may be realized by a combination of the above-described circuits and the like and a program.

Further, in the case where a part or all of each component of each device is realized by a plurality of information processing devices, circuits, etc., the plurality of information processing devices, circuits, etc. may be arranged centrally. It may be done. For example, the information processing apparatus, the circuit, and the like may be realized as a form in which each is connected via a communication network, such as a client and server system, a cloud computing system, and the like. Further, the information processing system 100 of the present embodiment may be realized as a single information processing apparatus. In addition, a part or all of the information processing system 100 according to the present embodiment performs the process of generating the above-described feature quantity, and thus an apparatus including a function of performing the process of generating the feature quantity be able to.

Next, the operation of the information processing system 100 of the present embodiment will be described. FIG. 19 is a flowchart illustrating an example of a process of generating a combination condition.

The input unit 10 acquires a first table including a prediction target and a first geographical attribute, and a second table including a second geographical attribute (step S11). Further, the input unit 10 receives the geographical relationship and the degree of the geographical relationship (step S12). The map parameter generator 30 can set the second table and the records included in the first table such that the relation between the value of the first geographical attribute and the value of the second geographical attribute satisfies the degree of geographical relation. A join condition for joining records included in the table is generated (step S13).

FIG. 20 is a flowchart showing another example of the process of generating the combining condition. The input unit 10 acquires a first table including a prediction target and a first temporal attribute, and a second table including a second temporal attribute (step S21). Also, the input unit 10 receives a temporal relationship and a degree of the temporal relationship (step S22). The map parameter generator 30 is configured to record the second table and the records included in the first table such that the relationship between the value of the first temporal attribute and the value of the second temporal attribute satisfies the degree of the temporal relationship. A join condition for joining records included in the table is generated (step S23).

FIG. 21 is a flowchart illustrating an example of processing for generating a feature amount. The input unit 10 acquires a first table including a prediction target and a first geographical attribute, and a second table including a second geographical attribute (step S31). The feature quantity generator 82 calculates a distance statistic when the value of the second geographical attribute with respect to the value of the first geographical attribute satisfies a predetermined condition (step S32), and calculates the calculated statistic as a feature A quantity is added to the attribute of the first table (step S33).

FIG. 22 is a flowchart illustrating another example of the process of generating the feature amount. The input unit 10 acquires a first table including the prediction target and the first attribute and a second table including the second attribute (step S41). The input unit 10 also receives a similarity function used to calculate the similarity between the first attribute and the second attribute, and a condition for the similarity (for example, a threshold for the similarity) (step S42). The feature quantity generator 82 generates feature quantity candidates from the first table and the second table using the combination condition and the aggregation condition calculated using the similarity function (step S43). Then, the feature amount selector 83 selects a feature amount optimal for prediction from the feature amount candidates (step S44).

As described above, in the present embodiment, the input unit 10 acquires the first table including the prediction target and the first geographical attribute, and the second table including the second geographical attribute. In addition, the input unit 10 receives a geographical relationship and the degree of the geographical relationship. Then, the map parameter generator 30 may be configured to set the records included in the first table such that the relation between the value of the first geographical attribute and the value of the second geographical attribute satisfies the degree of geographical relation. Create a join condition for joining the records included in the second table. Similarly, in the present embodiment, the input unit 10 acquires a first table including a prediction target and a first temporal attribute, and a second table including a second temporal attribute. In addition, the input unit 10 receives a temporal relationship and a degree of the temporal relationship. Then, the records included in the first table and the map parameter generator 30 are such that the relationship between the value of the first temporal attribute and the value of the second temporal attribute satisfies the degree of the temporal relationship. Create a join condition for joining the records included in the second table. Thus, it is possible to reduce the number of steps of associating a plurality of pieces of information through geographical information or temporal information. As a result, it is possible to reduce the load on a computer that processes information represented by various expressions.

Further, in the present embodiment, the input unit 10 acquires a first table including a prediction target and a first geographical attribute, and a second table including a second geographical attribute. Then, when the value of the second geographical attribute with respect to the value of the first geographical attribute satisfies a predetermined condition, the feature quantity generator 82 determines that the value of the first geographical attribute and the second satisfying the condition. The statistics of the distance calculated based on the value of the geographical attribute is added to the attribute of the first table as a feature that is a variable that can affect the prediction target. Therefore, feature quantities can be efficiently generated from a plurality of information sources having geographical information.

Furthermore, in the present embodiment, the input unit 10 acquires a first table including a prediction target and a first attribute, and a second table including a second attribute. Further, the input unit 10 receives a similarity function used to calculate the similarity between the first attribute and the second attribute, and a condition for the similarity. Then, the feature quantity generator 82 generates candidate feature quantities from the first table and the second table using the combination condition and the aggregation condition calculated using the similarity function, and the feature quantity selector 83 selects a feature quantity optimal for prediction from the feature quantity candidates. Therefore, it is possible to reduce the number of analysts for generating the feature amount.

Next, an outline of the present invention will be described. FIG. 23 is a block diagram showing an outline of a feature quantity generation apparatus according to the present invention. The feature quantity generation device 280 according to the present invention, a first table (for example, a target table) including a prediction target and a first geographical attribute, and a second table (for example, a source table) including a second geographical attribute And a table acquisition unit 281 (for example, the input unit 10) for acquiring the first geographical attribute when the value of the second geographical attribute with respect to the value of the first geographical attribute satisfies a predetermined condition. An attribute of adding a distance statistic calculated based on the value of and the value of the second geographical attribute that satisfies the condition to the attribute of the first table as a feature that is a variable that can affect the prediction target And an addition unit 282 (for example, a feature quantity generator 82).

With such a configuration, it is possible to efficiently generate feature quantities from multiple information sources having geographical information.

In addition, the attribute adding unit 282 may calculate the distance statistical value by performing an operation of aggregating the records of the second table satisfying the predetermined condition for each record of the first geographical attribute.

Specifically, the attribute adding unit 282 calculates the records of the second table as the operation of aggregating the records of the first geographical attribute with the geographical attributes of the second table satisfying the predetermined condition. The sum of distances and / or the average may be calculated and added to the attributes of the first table.

In addition, the attribute adding unit 282 calculates the number of records of the geographical attribute of the second table which satisfies a predetermined condition for each record of the first geographical attribute as an operation of aggregating the records of the second table. May be calculated and added to the attributes of the first table.

In addition, the feature quantity generation device 280 may include a learning unit (for example, a learning unit 91) that learns a prediction model using the added attribute as a feature quantity.

FIG. 24 is a schematic block diagram showing the configuration of a computer according to at least one embodiment. The computer 1000 includes a processor 1001, a main storage 1002, an auxiliary storage 1003, and an interface 1004.

The above-described information processing system is implemented in a computer 1000. The operation of each processing unit described above is stored in the auxiliary storage device 1003 in the form of a program (combination condition generation program). The processor 1001 reads a program from the auxiliary storage device 1003 and expands it in the main storage device 1002, and executes the above processing according to the program.

In at least one embodiment, the auxiliary storage device 1003 is an example of a non-temporary tangible medium. Other examples of non-transitory tangible media include magnetic disks connected via an interface 1004, magneto-optical disks, CD-ROMs, DVD-ROMs, semiconductor memories, and the like. Further, when this program is distributed to the computer 1000 by a communication line, the distributed computer 1000 may expand the program in the main storage unit 1002 and execute the above processing.

Further, the program may be for realizing a part of the functions described above. Furthermore, the program may be a so-called difference file (difference program) that realizes the above-described function in combination with other programs already stored in the auxiliary storage device 1003.

As mentioned above, although this invention was demonstrated with reference to embodiment and an Example, this invention is not limited to the said embodiment and Example. The configurations and details of the present invention can be modified in various ways that those skilled in the art can understand within the scope of the present invention.

This application claims priority based on US Provisional Application No. 62 / 568,448 filed Oct. 5, 2017, the entire disclosure of which is incorporated herein.

10 input unit 20 geocoder 30 map parameter generator 31 time difference map generator 32 map generator 33 attribute specifying unit 40 geomap generator 41 distance map generator 42 inclusion map generator 43 overlap map generator 44 same area map generator 50 filter parameter generator 51 filter generator 60 aggregation parameter generator 61 numerical aggregation generator 70 geo-aggregate generator 71 point aggregation generator 72 area aggregation generator 80 storage unit 81 feature quantity generation function generator 82 feature quantity generator 83 feature quantity selector 90 output unit 91 learning unit 92 prediction unit

Claims

Table acquisition means for acquiring a first table including a prediction target and a first geographical attribute, and a second table including a second geographical attribute;
The value of the first geographical attribute and the value of the second geographical attribute satisfying the condition when the value of the second geographical attribute with respect to the value of the first geographical attribute satisfies a predetermined condition And attribute adding means for adding the statistical value of the distance calculated on the basis of the distance to the attribute of the first table as a feature that is a variable that can affect the prediction target. Quantity generator.
The attribute addition means calculates a distance statistical value by performing an operation of aggregating records of a second table satisfying a predetermined condition for each record of the first geographical attribute. Generator.
The attribute adding means is an operation of aggregating the records of the second table, for each record of the first geographical attribute, the sum and average of the distance to the geographical attribute of the second table satisfying the predetermined condition. The feature quantity generation device according to claim 1 or 2, wherein at least one is calculated and added to the attribute of the first table.
The attribute addition means calculates the number of records of the geographical attribute of the second table satisfying a predetermined condition for each record of the first geographical attribute as an operation of aggregating the records of the second table, The feature quantity generation device according to claim 1 or 2, which is added to the attribute of the table of 1.
The feature quantity generation device according to any one of claims 1 to 4, further comprising a learning unit that learns a prediction model using the added attribute as a feature quantity.
Obtaining a first table including the prediction target and the first geographical attribute, and a second table including the second geographical attribute;
The value of the first geographical attribute and the value of the second geographical attribute satisfying the condition when the value of the second geographical attribute with respect to the value of the first geographical attribute satisfies a predetermined condition And adding a statistical value of a distance calculated based on the feature to the attribute of the first table as a feature that is a variable that can affect the prediction target.
The feature quantity generation method according to claim 6, wherein a distance statistical value is calculated by performing an operation of aggregating records of a second table satisfying a predetermined condition for each record of the first geographical attribute.
On the computer
A table acquisition process for acquiring a first table including a prediction target and a first geographical attribute, and a second table including a second geographical attribute;
The value of the first geographical attribute and the value of the second geographical attribute satisfying the condition when the value of the second geographical attribute with respect to the value of the first geographical attribute satisfies a predetermined condition And a feature amount generating program for executing attribute addition processing for adding the statistical value of the distance calculated based on the feature to the attribute of the first table as a feature amount that is a variable that may affect the prediction target.
On the computer
9. The feature value according to claim 8, wherein in the attribute addition processing, a distance statistical value is calculated by performing an operation of aggregating records of a second table satisfying a predetermined condition for each record of the first geographical attribute. Generator.