US11514062B2 - Feature value generation device, feature value generation method, and feature value generation program - Google Patents

Feature value generation device, feature value generation method, and feature value generation program Download PDF

Info

Publication number
US11514062B2
US11514062B2 US16/753,757 US201816753757A US11514062B2 US 11514062 B2 US11514062 B2 US 11514062B2 US 201816753757 A US201816753757 A US 201816753757A US 11514062 B2 US11514062 B2 US 11514062B2
Authority
US
United States
Prior art keywords
attribute
similarity
feature
condition
attributes
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active, expires
Application number
US16/753,757
Other languages
English (en)
Other versions
US20200301921A1 (en
Inventor
Ting Chen
Yukitaka Kusumura
Ryohei Fujimaki
Kazuyo Narita
Masato Asahara
Yusuke Muraoka
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Dotdata Inc
Original Assignee
Dotdata Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Dotdata Inc filed Critical Dotdata Inc
Priority to US16/753,757 priority Critical patent/US11514062B2/en
Publication of US20200301921A1 publication Critical patent/US20200301921A1/en
Assigned to DOTDATA, INC. reassignment DOTDATA, INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: MURAOKA, YUSUKE, ASAHARA, MASATO, FUJIMAKI, RYOHEI, NARITA, KAZUYO, KUSUMURA, YUKITAKA, CHEN, TING
Application granted granted Critical
Publication of US11514062B2 publication Critical patent/US11514062B2/en
Active legal-status Critical Current
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
    • G06F16/2465—Query processing support for facilitating data mining operations in structured databases
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2455—Query execution
    • G06F16/24553—Query execution of query operations
    • G06F16/24558—Binary matching operations
    • G06F16/2456—Join operations
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/28—Databases characterised by their database models, e.g. relational or object models
    • G06F16/284—Relational databases
    • G06F16/285—Clustering or classification
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/30—Semantic analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/04—Inference or reasoning models

Definitions

  • the present invention relates to a feature generating device, a feature generating method, and a feature generating program for combining a plurality of tables to generate features.
  • Data mining is a technique in which useful knowledge not known before it is found in a large amount of data.
  • a large number of attribute candidates must be generated in order to find useful knowledge not known before.
  • a large number of candidates for attributes must be generated that can affect the variable being predicted (target variable).
  • Patent Document 1 describes the generation of feature candidates used in machine learning by combining target tables including a target variable with source tables not including the target variable.
  • the processing performed to generate feature candidates is defined using combinations of three conditions, namely, a filter condition, map condition, and reduction condition, to reduce the number of hours of labor that analysts must perform to generate feature candidates.
  • Patent Document 1 WO 2017/090475 A1
  • the present inventors came up with the idea that prediction accuracy could be improved by using a wide variety of information sources when predicting a target in a given area. In other words, they believed that information is preferably obtained by combining a plurality of related information sources.
  • Patent Document 1 uses customer IDs in a target table and source table in the combination conditions (that is, map conditions) for the target table and the source table.
  • the present inventors discovered that a plurality of candidates for map conditions could be generated depending on the target of analysis.
  • the processing is complicated. As a result, the amount of labor performed by the analyst is substantial.
  • An aspect of the present invention is a feature generating device comprising: a table acquiring means for acquiring a first table including a prediction target and a first attribute, and a second table including a second attribute; a receiving means for receiving a similarity function and a similarity condition used to calculate the degree of similarity between the first attribute and the second attribute; a feature generating means for generating feature candidates from the first table and the second table able to affect the prediction target using a combination condition for combining a record in the first table including a value for the first attribute and a record in the second table including a value for the second attribute whose degree of similarity calculated from the value of the first attribute and the value of the second attribute using the similarity function satisfies the condition, a reduction method for the plurality of records in the second table, and a reduction condition represented by a column to be reduced; and a feature selecting means for selecting the optimum feature for prediction from among the feature candidates.
  • Another aspect of the present invention is a feature generating method comprising: acquiring a first table including a prediction target and a first attribute, and a second table including a second attribute; receiving a similarity function and a similarity condition used to calculate the degree of similarity between the first attribute and the second attribute; generating feature candidates from the first table and the second table able to affect the prediction target using a combination condition for combining a record in the first table including a value for the first attribute and a record in the second table including a value for the second attribute whose degree of similarity calculated from the value of the first attribute and the value of the second attribute using the similarity function satisfies the condition, a reduction method for the plurality of records in the second table, and a reduction condition represented by a column to be reduced; and selecting the optimum feature for prediction from among the feature candidates.
  • Another aspect of the present invention is a feature generating program causing a computer to execute: a table acquiring process for acquiring a first table including a prediction target and a first attribute, and a second table including a second attribute; a receiving process for receiving a similarity function and a similarity condition used to calculate the degree of similarity between the first attribute and the second attribute; a feature generating process for generating feature candidates from the first table and the second table able to affect the prediction target using a combination condition for combining a record in the first table including a value for the first attribute and a record in the second table including a value for the second attribute whose degree of similarity calculated from the value of the first attribute and the value of the second attribute using the similarity function satisfies the condition, a reduction method for the plurality of records in the second table, and a reduction condition represented by a column to be reduced; and a feature selecting process for selecting the optimum feature for prediction from among the feature candidates.
  • the technical means of the present invention have the technical effect of being able to reduce the amount of labor performed by an analyst to generate features.
  • FIG. 1 is a block diagram of the information processing system in an embodiment of the present invention.
  • FIG. 2 is a diagram used to explain an example of a configuration file.
  • FIG. 3 is a diagram used to explain an example of data conversion processing.
  • FIG. 4 is a diagram used to explain an example of the relationship of each parameter with a first table and a second table.
  • FIG. 5 is a diagram used to explain an example of processing performed to generate map parameters based on distance.
  • FIG. 6 is a diagram used to explain another example of processing performed to generate map parameters based on distance.
  • FIG. 7 is a diagram used to explain an example of a method used to determine whether or not attributes are in the same area.
  • FIG. 8 is a diagram used to explain an example of processing performed to generate map parameters based on whether or not locations are in a common area.
  • FIG. 9 is a diagram used to explain an example of processing performed to generate map parameters based on an inclusion relationship.
  • FIG. 10 is a diagram used to explain an example of processing performed to generate map parameters based on time differences.
  • FIG. 11 is a diagram used to explain an example of processing performed to generate map parameters based on text similarities.
  • FIG. 12 is a diagram used to explain an example of processing performed to generate map parameters based on structural similarities.
  • FIG. 13 is a diagram used to explain an example of generated map parameters.
  • FIG. 14 is a diagram used to explain an example of processing performed to generate reduction parameters for calculating distance statistics.
  • FIG. 15 is a diagram used to explain an example of processing performed to generate reduction parameters for calculating area statistics.
  • FIG. 16 is a diagram used to explain an example of generated reduction parameters.
  • FIG. 17 is a diagram used to explain an example of combined map parameters.
  • FIG. 18 is a diagram used to explain an example of a method used to combine parameters and generate a feature descriptor.
  • FIG. 19 is a flowchart showing an example of processing performed to generate combination conditions.
  • FIG. 20 is a flowchart showing another example of processing performed to generate combination conditions.
  • FIG. 21 is a flowchart showing an example of processing performed to generate features.
  • FIG. 22 is a flowchart showing another example of processing performed to generate features.
  • FIG. 23 is a block diagram showing an overview of a feature generating device of the present invention.
  • FIG. 24 is a schematic block diagram showing the configuration of a computer related to at least one embodiment.
  • the information processing system in the present embodiment acquires a table including variables for a predicted target (such as target variables) (referred to as the first table below) and a table different from the first table (referred to as the second table below).
  • the first table is sometimes referred to as the target table and the second table is sometimes referred to as the source table.
  • the first table and the second table may also include sets of data.
  • the first table and the second table include attributes from a shared perspective.
  • a shared perspective means the semantic content of attribute data is the same.
  • the method used to express the data may be the same or different.
  • the attributes in the first table are referred to as first attributes and the attributes in the second table are referred to as second attributes.
  • the shared perspective may be a geographic perspective or a temporal perspective.
  • attribute values from a geographic perspective can be classified as being one of the following four types of geographic data.
  • the description following the colon in the header indicates the syntax of the data.
  • Point P is indicated as (longitude, latitude) coordinates.
  • Polygon G is defined by a single outer boundary b 1 and zero or more inner boundaries (b 2 , . . . , b n ).
  • b 1 (p 1 , p 2 , . . . , p n ) is a boundary of a closed ring defined as an order of three or more points (provided p 1 , p 2 , . . . , p n ⁇ P)
  • a multipolygon M consists of one or more polygons.
  • the analysis data type may be defined in association with a data type as semantic information related to data analysis.
  • polygons G and multipolygons M may be defined as analysis data types for areas (Area)
  • points P may be defined as an analysis data type related to points (Point).
  • a character string relating to an address may be defined as an analysis data type relating to, for example, a country, city, town, landmark, street, or point.
  • An analysis data type representing geographic information is sometimes referred to as a geographic data type below.
  • an attribute type from a time perspective can be defined as a time stamp (TimeStamp) type.
  • the attributes in the first table are referred to as first geographic attributes and the attributes in the second table are referred to as second geographic attributes.
  • the attributes in the first table are referred to as first temporal attributes and the attributes in the second table are referred to as second temporal attributes.
  • Other attributes are described in similar ways.
  • the first geographic attribute may be the primary key in the first table.
  • the attributes share either a geographic perspective or a temporal perspective. However, the attributes do not have to share a geographic perspective or a temporal perspective.
  • the attributes may share a textual perspective or a structural perspective.
  • the attribute value from a textual perspective may be an address.
  • the attribute value from a structural perspective may be a URL (Uniform Resource Locator) or tree structure path.
  • URL Uniform Resource Locator
  • the attributes with a shared perspective in the following explanation are primarily geographic attributes and temporal attributes.
  • FIG. 1 is a block diagram of the information processing system in an embodiment of the present invention.
  • the information processing system 100 in the present embodiment includes an input unit 10 , geo-coder 20 , map parameter generator 30 , filter parameter generator 50 , reduction parameter generator 60 , storage unit 80 , feature descriptor generator 81 , feature generator 82 , feature selector 83 , output unit 90 , learning unit 91 , and predicting unit 92 .
  • the input unit 10 acquires a first table and a second table. Because the input unit 10 acquires these tables, the input unit 10 can be referred to as the table acquiring means.
  • the input unit 10 may acquire a plurality of second tables. When the first table and the second table are stored by the storage unit 80 , the input unit 10 may acquire the first table and the second table from the storage unit 80 .
  • the input unit 10 may also acquire the first table and the second table from another system or storage unit via a communication network (not shown).
  • the input unit 10 may acquire a first table including prediction targets and first geographic attributes and a second table including second geographic attributes.
  • the input unit 10 may acquire a first table including prediction targets and first temporal attributes and a second table including second temporal attributes.
  • the input unit 10 may acquire a first table including prediction targets and first textual attributes and a second table including second textual attributes, or a first table including prediction targets and first structural attributes and a second table including second structural attributes. Structural attributes will be described later.
  • the input unit 10 also receives a function for calculating the degree of similarity between a first attribute and a second attribute (referred to below as the similarity function) and a condition for determining the similarity between the value of a first attribute and the value of a second attribute when there is a certain degree of similarity (referred to below as the similarity condition).
  • the similarity function may be expressed as an equation or as a parameter.
  • the similarity condition may be expressed as a threshold value for determining whether or not there is similarity based on the degree of similarity (referred to simply as the similarity threshold value below) or may be expressed as an equation for outputting whether or not there is a similarity based on a parameter, etc.
  • the input unit 10 receives the geographic relationship as a similarity function and receives a similarity threshold value indicating the degree of geographic relationship as a condition.
  • the similarity function can be defined as a function that calculates a higher degree of similarity when the distance is closer.
  • the input unit 10 receives the temporal relationship as a similarity function and receives a similarity threshold value indicating the degree of temporal relationship as a condition.
  • the similarity function can be defined as a function that calculates a higher degree of similarity when the time difference is smaller.
  • the input unit 10 receives the textual relationship as a similarity function and receives a similarity threshold value indicating the degree of textual relationship as a condition.
  • the similarity function can be defined as a function that calculates a higher degree of similarity when there is a greater match between the two texts.
  • the Simpson coefficient for morphemes can be used to determine the textual similarity.
  • morph (a) is defined as the set of morphemes in text string a. For example, the following four text strings indicating an address can be expressed as a set of morphemes.
  • textSim (a, b) used to calculate the degree of similarity between text string a and text string b can be defined using Equation 1 below.
  • textSim( a,b )
  • the degree of similarity between the text strings for the addresses in the examples provided above is calculated in the following way.
  • the input unit 10 receives the structural relationship as a similarity function and receives a similarity threshold value indicating the degree of structural relationship as a condition.
  • a character string in which tree structure information such as the directory structure for an address or file is expressed using forward slashes is defined as a path string below.
  • the address ‘Kanagawa-ken, Kawasaki-shi’ is expressed by the path string ‘Kanagawa-ken/Kawasaki-shi’.
  • the directory structure ‘news ⁇ economy ⁇ bigdata’ is expressed by the path string ‘news/economy/bigdata’.
  • the similarity function can be defined as a function that calculates a higher degree of similarity when there is a closer distance between the two path strings.
  • the distance coefficient for path strings can be the minimum value for the distance to the lowest common ancestor (LCA) node.
  • the lowest common ancestor node is the same node that first appears when tracing from the lowest node represented by each of two paths in the upper (ancestor) direction.
  • the distance to the lowest common ancestor node is the number of nodes when tracing from the lowest node to the lowest common ancestor node.
  • the two path character strings ‘/a/b/c’ and ‘/a/b/z’ Take, for example, the two path character strings ‘/a/b/c’ and ‘/a/b/z’.
  • the lowest common ancestor node of the two paths is ‘a/b’.
  • the distance from ‘/a/b/c’ to ‘/a/b’ is 1 and the distance from ‘/a/b/z’ to ‘/a/b’ is 1.
  • the two path character strings ‘/a/b/c’ and ‘/a/d/e/z’.
  • the lowest common ancestor node of the two paths is ‘/a’.
  • the distance from ‘/a/b/c’ to ‘/a’ is 2 and the distance from ‘/a/d/e/z’ to ‘/a’ is 3.
  • FIG. 2 is a diagram used to explain an example of a configuration file (referred to as a config file below).
  • a config file referred to as a config file below.
  • the similarity function and similarity condition are set in a configuration file (config file below).
  • the input unit 10 may receive the config file.
  • Portion C 1 in the config file shown in FIG. 2 shows the similarity function and similarity condition. Portions C 2 to C 4 in the config file are described later.
  • the first part shows the correspondence between the data type of the first attribute (more specifically, the analysis data type) and the data type of the second attribute (more specifically, the analysis data type).
  • the later part shows the similarity function and the condition (similarity threshold value). The contents are described in greater detail later.
  • the “Point-Point” line in portion C 1 defines the geographic relationship indicating the distance between a first geographic attribute represented by a point and a second geographic attribute represented by a point.
  • “DistanceMap” is a map function that defines the degree of the geographic relationship, and includes a distance threshold as a parameter.
  • the three parameters in the DistanceMap function indicate in successive order the “start value,” the “end value,” and the “interval” (the threshold value applied from the start value to the end value).
  • “DistanceMap,” 1, 3, 1) in FIG. 2 represent the three threshold values (“distance within 1 km,” “distance within 2 km,” and “distance within 3 km”) applied to the function.
  • KNearestMap is a map function that defines the degree of geographic relationship, and includes a threshold value for the number of nearby geographic information items as a parameter.
  • the three parameters in the KNearestMap function similarly indicate the “start value,” the “end value,” and the “interval” (the threshold value applied from the start value to the end value).
  • “KNearestMap,” 3, 5, 1) indicates that the number of nearby geographic information items applied to the function are the three threshold values “within 3,” “within 4,” and “within 5.”
  • “SameCityMap” is a map function that defines the degree of geographic relationship, and is a function that determines whether two points are included in the same area. While the SameCityMap function does not include a parameter, it determines whether or not the points are included in the same area based on area information defining the area. Area information is defined in advance.
  • the “Point-Area” line in portion C 1 defines the geographic relationship indicating the distance between a first geographic attribute represented by a point and a second geographic attribute represented by an area.
  • InclusionMap is a map function that defines the degree of geographic relationship, and determines whether the first geographic attribute represented by a point is included in the second geographic attribute represented by an area. InclusionMap does not include a parameter.
  • KNearestMap is also defined in the “Point-Area” line.
  • the content of the KNearestMap function is the same as the KNearestMap function in “Point-Point.”
  • the “Area-Area” line in portion C 1 defines the geographic relationship indicating the distance between a first geographic attribute represented by an area and a second geographic attribute represented by an area.
  • Intersect Map is a map function that defines the degree of geographic relationship, and determines whether the first geographic attribute represented by an area intersects with the second geographic attribute represented by an area. IntersectMap does not include a parameter.
  • the first geographic data type and the second geographic data type may be the same geographic data type or may be different geographic data types.
  • the first geographic data type may be a type of data able to specify geography using point information
  • the second geographic data type may be a type of data able to specify geography using range information.
  • the “TimeStamp-TimeStamp” line in portion C 1 defines the temporal relationship indicating the difference between a first temporal attribute and a second temporal attribute.
  • TimeDiffMap is a map function that defines the degree of temporal relationship, and includes a threshold value for time difference as a parameter.
  • the three parameters in the TimeDiffMap function indicate the “start value,” the “end value,” and the “interval” (the threshold value applied from the start value to the end value).
  • the unit of time is minutes
  • “TimeDiffMap,” 30, 60, 30) in FIG. 2 represent the two threshold values (“time difference within 30 minutes,” “time difference within 60 minutes”) applied to the function.
  • the “Text-Text” line in portion C 1 defines the matching relationship between a first attribute representing a character string and a second attribute representing a character string.
  • “ExactMap” is a function for determining whether or not the attributes represented by character strings match.
  • a similarity relationship between a first attribute representing a character string and a second attribute representing a character string may also be defined in the “Text-Text” line.
  • a map function “textSimMap” that defines the degree of the relationship between the character strings may be set in the “Text-Text” line.
  • “TextSimMap” is a map function that defines the degree of relationship between character strings, and includes a threshold value for similarity as a parameter. As in the DistanceMap function, the textSimMap function has three parameters indicating in successive order the “start value,” the “end value,” and the “interval” (the threshold value applied from the start value to the end value).
  • a structural relationship “Path-Path” may be defined in the configuration file that represents the distance between a first structural attribute represented by a path character string and a second structural attribute represented by a path character string.
  • map function “pathDisMap” that defines the degree of structural relationship may be set in the “Path-Path” line.
  • “pathDisMap” is a map function that defines the degree of structural relationship, and includes a distance threshold as a parameter. As in the DistanceMap function, the pathDisMap function has three parameters indicating in successive order the “start value,” the “end value,” and the “interval” (the threshold value applied from the start value to the end value).
  • the map parameter generator 30 described later When a config file shown in FIG. 2 is received by the input unit 10 , the map parameter generator 30 described later generates a combination condition (map parameter) for combining a record in the first table with a record in the second table.
  • the input unit 10 may also receive the attributes of the data in each column of the table.
  • the geo-coder 20 converts attribute data represented by a character string. For example, when geographic attribute data is represented by a character string, the geo-coder 20 converts the character string into point, polygon, or multipolygon data. When there is no need to convert data, the information processing system 100 does not require a geo-coder 20 .
  • FIG. 3 is a diagram used to explain an example of data conversion processing.
  • table adt 1 defining the analysis data type for each column and table adt 2 defining the corresponding data type for conversion from the analysis data type are acquired in advance.
  • the input unit 10 acquires target table T, source table S 1 , and source table S 2 shown in FIG. 3 .
  • the analysis data type for the “Pickup_location” column in source table S 2 is Point when referring to table adt 1 , and does not have to be converted.
  • the analysis data type for the “community” column in source table S 1 is “TownAddress” when referring to the table adt 1 , and has to be converted to the Polygon data type when referring to table adt 2 . Therefore, the geo-coder 20 converts the data in the “community” column of source table S 1 so that the data is represented by a polygonal area.
  • area information that can specify a region using a polygon may be determined in advance for the content of “community,” and the geo-coder 20 may convert data based on the area information so that the data type becomes a Polygon.
  • the map parameter generator 30 , the filter parameter generator 50 , and the reduction parameter generator 60 generate parameters to be used by the feature descriptor generator 81 described later to generate a feature descriptor for generating a feature serving as a variable that can affect a prediction target.
  • a feature refers to the content of the feature itself (such as “population” or “location”).
  • a feature generated by the feature generator 82 described later is a candidate for an explanatory variable when a model is generated using machine learning.
  • a feature descriptor generated in the present embodiment can be used to automatically generate candidates for explanatory variables when a model is generated using machine learning.
  • FIG. 4 is a diagram used to explain an example of the relationship of each parameter with a first table and a second table.
  • the parameter generated by the filter parameter generator 50 is a parameter representing an extraction condition for a row in the second table.
  • This parameter is referred to as a filter parameter below, and the process of extracting a row from the second table based on a filter parameter is sometimes called “filtering.”
  • a list of extraction conditions is sometimes called an “F list.”
  • An extraction condition can be used, including, for example, a condition for determining whether a value is the same as (or larger or smaller than) a value in the designated column.
  • the parameter generated by the reduction parameter generator 60 is a parameter indicating the reduction method used to reduce the data in each row of the second table by each target variable.
  • the rows in the first table and the rows in the second table often have a one-to-many correspondence. As a result, the rows are reduced.
  • the reduction information may be defined as a reduction function for columns in a source table (second table).
  • Any reduction method can be used. Examples include the total number of columns, the maximum value, the minimum value, the average value, the median value, and the distribution. The total of the total number of columns may be calculated from any perspective to include or exclude duplicate data.
  • This parameter is referred below to as the reduction parameter, and the process used to reduce data in each column using the method indicated by the reduction parameter is referred to as the reduction process.
  • the process used to reduce geographic information is a geo-reduction process.
  • the reduction processing list is sometimes referred to as the “R list.” The process of reducing geographic information will be described later in greater detail.
  • the parameter generated by the map parameter generator 30 is a parameter representing the condition for the correspondence between the columns of the first table and the columns of the second table.
  • This parameter is referred to as the map parameter below, and the process of associating columns in each table based on the map parameter is sometimes referred to as mapping.
  • the list of conditions for correspondence is sometimes referred to as the “M list.”
  • the process of associating geographic information is sometimes referred to as geo-mapping.
  • the association of the columns in each table by mapping can be said to entail combining (joining) a plurality of tables into a single table using associated columns. The process of associating geographic information will be described later in greater detail.
  • the map parameter generator 30 includes a geo-map generator 40 , TimeDiff map generator 31 , exact map generator 32 , and attribute specifying unit 33 .
  • the map parameter generator 30 (more specifically, each generator in the map parameter generator 30 ) generates the combination condition for combining records from a first table that contain the value of a first attribute with records from a second table that contain the value of a second attribute so that the similarity calculated using the value of the first attribute and the value of the second attribute satisfies the condition. Satisfying the condition means the similarity is at or below a threshold value or within a predetermined range.
  • the geo-map generator 40 generates a parameter indicating the condition for correspondence between columns of the first table and the second table including geographic attributes.
  • the geo-map generator 40 has a distance map generator 41 , an inclusion map generator 42 , an overlap map generator 43 , and a same area map generator 44 .
  • the geo-map generator 40 (more specifically, each generator in the geo-map generator 40 ) generates the combination condition (map parameter) for combining records contained in the first table with records contained in the second table so that the relationship between the value of a first geographic attribute and the value of a second geographic attribute satisfy the degree of geographic relationship.
  • the processing performed by each generator will be described below in greater detail.
  • the distance map generator 41 generates a map parameter when the similarity and a condition (such as a similarity threshold value) have been received for associating the first table and the second table based on proximity in distance. In the example shown in FIG. 2 , this corresponds to the DistanceMap function or the KNearestMap function being set in the config file.
  • the distance map generator 41 generates a map parameter for combining records contained in the first table with records contained in the second table so that the value of a first geographic attribute and the value of the second geographic attribute are at or below a threshold value.
  • FIG. 5 is a diagram used to explain an example of processing performed to generate map parameters based on distance.
  • the target table T and one of the source tables S 2 are acquired.
  • the target table T in FIG. 5 includes data representing the number of passengers picked up at five locations (pickup_number) at 22:00 on Jan. 8, 2015.
  • the source table S 2 in FIG. 5 is used to associate and record the number of passengers, distances traveled, and passenger drop-off locations at each time.
  • the distance map generator 41 In the case of the DistanceMap function shown in FIG. 2 , the distance map generator 41 generates a parameter associating each record in the target table T with each record in the source table S 2 in which the distance between the location indicated by the value of the first geographic attribute and the location indicated by the value of the second geographic attribute is within 1 km. The distance map generator 41 also generates a parameter associating each record in the target table T with each record in the source table S 2 in which the distance between the location indicated by the value of the first geographic attribute and the location indicated by the value of the second geographic attribute is within 2 km and within 3 km.
  • the attribute in the “target_location” column of the target table T is the first geographical attribute
  • the attribute in the “Pickup_location” column of the source table S 2 is the second geographical attribute. These two columns are associated.
  • the columns to be associated in the first table and the second table may be established in advance or specified by the attribute specifying unit 33 described later.
  • the attribute in the “target_location” column of the target table T is the first geographical attribute
  • the attribute in the “Pickup_location” column of the source table S 2 is the second geographical attribute. These two columns are associated.
  • the columns to be associated in the first table and the second table may be established in advance or specified by the attribute specifying unit 33 described later.
  • FIG. 6 is a diagram used to explain another example of processing performed to generate map parameters based on distance.
  • the target table T and the source table S 2 in FIG. 6 are the same as target table T and the source table S 2 in FIG. 5 .
  • the distance map generator 41 In the case of the KNearestMap function shown in FIG. 2 , the distance map generator 41 generates a parameter in which each record in the target table T is associated with the two closest records in the source table S 2 in ascending order in terms of the distance between the location indicated by the value of the first geographic attribute and the location indicated by the value of the second geographic attribute. The distance map generator 41 also generates parameters in which each record in the target table T is associated with the three closest and the four closest records in the source table S 2 in ascending order in terms of the distance between the location indicated by the value of the first geographic attribute and the location indicated by the value of the second geographic attribute.
  • the attribute in the “target_location” column of the target table T is the first geographical attribute
  • the attribute in the “Pickup_location” column of the source table S 2 is the second geographical attribute. These two columns are associated.
  • the columns to be associated in the first table and the second table may be established in advance or specified by the attribute specifying unit 33 described later.
  • the parameter P 12 shown in FIG. 6 is generated.
  • the map parameter is generated based on the geographic analysis data type, and a single map processing operation is defined based on a single map parameter.
  • the map data M 12 in FIG. 6 is the result of associating each record in the target table T with the two closest records in the source table S 2 in ascending order. In one example, each record in the source table is associated with the two closest records in the target table.
  • the same area map generator 44 generates a map parameter when a similarity function is received for associating records in the first table and the second table based on whether they are in the same area. In the example shown in FIG. 2 , this corresponds to the SameCityMap function being set in the config file.
  • the same area map generator 44 generates a map parameter for combining a record in the first table with a record in the second table when the location indicated by the value of the first geographic attribute and the location indicated by the value of the second geographic attribute are within the same area.
  • FIG. 7 is a diagram used to explain an example of a method used to determine whether or not attributes are in the same area.
  • a common area table CAT is defined beforehand for associating each area with areas specified using polygons. Examples of common areas include countries, provinces, cities, autonomous regions, and neighborhoods. Common areas are defined so as not to overlap and represent boundary information on a map.
  • the common area table CAT may be stored in the storage unit 80 .
  • FIG. 8 is a diagram used to explain an example of processing performed to generate map parameters based on whether or not locations are in a common area.
  • the target table T and the source table S 2 in FIG. 8 are the same as the target table T and the source table S 2 in FIG. 5 .
  • the same area map generator 44 generates a parameter associating each record in the target table T with each record in the source table S 2 in which the location indicated by the value of the first geographic attribute and the location indicated by the value of the second geographic attribute are within the same area.
  • the attribute in the “target_location” column of the target table T is the first geographical attribute
  • the attribute in the “Pickup_location” column of the source table S 2 is the second geographical attribute. These two columns are associated.
  • the columns to be associated in the first table and the second table may be established in advance or specified by the attribute specifying unit 33 described later.
  • parameter P 13 shown in FIG. 8 is generated.
  • the map data M 13 shown in FIG. 8 is the result of associating each record in the target table T with each record in the source table S 2 having geographic attributes determined to be in the same area. Note that the map data M 13 shown in FIG. 8 provisionally associates geographic points within a distance of 1 km as being located in the same municipality.
  • the inclusion map generator 42 generates a map parameter when a similarity function for associating a first table with a second table based on the inclusion relationship is received. In the example shown in FIG. 2 , this corresponds to the InclusionMap function being set in the config file.
  • the inclusion map generator 42 generates a map parameter for combining records contained in the first table with records contained in the second table when a location indicated by the value of a first geographic attribute is present in the area indicated by the value of the second geographic attribute.
  • FIG. 9 is a diagram used to explain an example of processing performed to generate map parameters based on an inclusion relationship.
  • the target table T in FIG. 9 is the same as the target table T in FIG. 5 .
  • the source table S 1 in FIG. 9 is used to associate and record the overall population, the number of males, and the number of people age 20 to 40 in each area.
  • the inclusion map generator 42 generates a parameter associating each record in the target table T with each record in the source table S 1 in which a location indicated by the value of the first geographic attribute is within the area indicated by the value of the second geographic attribute.
  • the attribute in the “target_location” column of the target table T is the first geographical attribute
  • the attribute in the “community” column of the source table S 1 is the second geographical attribute. These two columns are associated.
  • the columns to be associated in the first table and the second table may be established in advance or specified by the attribute specifying unit 33 described later.
  • parameter P 14 shown in FIG. 9 is generated.
  • the map data M 14 in FIG. 9 shows the results of associating each record in the target table with the records in the source table S 1 that are in the same area.
  • the overlap map generator 43 generates a map parameter when a similarity function for associating a first table and a second table based on overlapping areas is received. In the example shown in FIG. 2 , this corresponds to the IntersectMap function being set in the config file.
  • the overlap map generator 43 generates a map parameter for combining records contained in the first table with records contained in the second table when an area indicated by the value of a first geographic attribute overlaps with an area indicated by the value of the second geographic attribute.
  • the time difference map generator 31 generates a map parameter when a similarity function and condition (such as a similarity threshold value) for associating a first table and a second table based on a time difference is received. In the example shown in FIG. 2 , this corresponds to the TimeDiffMap function being set in the config file.
  • the time difference map generator 31 generates a combination condition for combining a record in a first table with a record in a second table so that the relationship between the value of a first temporal attribute and the value of a second temporal attribute satisfy a degree of temporal relationship.
  • the time difference map generator 31 generates a parameter for combining a record in a first table with a record in a second table when the difference between the value of a first temporal attribute and the value of a second temporal attribute is at or below a threshold value.
  • FIG. 10 is a diagram used to explain an example of processing performed to generate map parameters based on time differences.
  • the target table T and source table S 2 in FIG. 10 is the same as the target table T and source table S 2 in FIG. 5 .
  • the time difference map generator 31 In the case of the TimeDiffMap function shown in FIG. 2 , the time difference map generator 31 generates a parameter for associating each record in target table T with records in source table S 2 in which the difference between the value of a first temporal attribute and the value of a second temporal attribute is at or below 30 minutes. The time difference map generator 31 generates a parameter for associating each record in target table T with records in source table S 2 in which the difference between the value of a first temporal attribute and the value of a second temporal attribute is at or below 60 minutes.
  • the attribute in the “time” column of the target table T is the first geographical attribute
  • the attribute in the “pickup_time” column of the source table S 2 is the second geographical attribute. These two columns are associated.
  • the columns to be associated in the first table and the second table may be established in advance or specified by the attribute specifying unit 33 described later.
  • parameter P 15 shown in FIG. 10 is generated.
  • the map data M 15 in FIG. 10 shows the results of associating each record in the target table T with the records in the source table S 2 with a time difference at or below 30 minutes.
  • the exact map generator 32 generates a map parameter when a similarity function for associating a first table with a second table has been received.
  • a parameter is generated for associating records in the target table with records in a source table based on the value of an attribute that is neither a geographic attribute nor a temporal attribute.
  • this corresponds to the ExactMap function being set in the config file.
  • the exact map generator 32 generates a map parameter for combining a record in the first table with a record in the second table when the value of the first geographic attribute and the value of the second geographic attribute match.
  • FIG. 11 is a diagram used to explain an example of processing performed to generate map parameters based on text similarities.
  • the target table T in FIG. 11 is a table including data indicating the number of passengers at a given location (pickup_number).
  • the source table S in FIG. 11 is a table for recording the average receipt in each area.
  • the exact map generator 32 generates a parameter for associating each record in the target table T with records in the source table S when the degree of similarity between the value of the first character string attribute and the value of the second character string attribute is 0.8 or more.
  • the exact map generator 32 generates a parameter for associating each record in the target table T with records in the source table S when the degree of similarity between the value of the first character string attribute and the value of the second character string attribute is 0.9 or more or 1.0 or more.
  • an “address” string in target table T is recorded as the first string attribute and an “address” string in the source table S is recorded as the second string attribute. Therefore, these two strings are associated. As a result, the parameter P 16 shown in FIG. 11 is generated.
  • the map data M in FIG. 11 shows the results of associating each record in the target table T with records in the source table S having a degree of similarity of 0.8 or more. In one example, only one record from the source table is associated with the first record in the target table.
  • FIG. 12 is a diagram used to explain an example of processing performed to generate map parameters based on structural similarities.
  • the target table T in FIG. 12 includes data indicating the number of times a web page identified by a certain URL has been accessed (access_number).
  • the source table S in FIG. 12 records the number of times the web page identified by the URL was accessed in the previous month (access_number).
  • the exact map generator 32 generates a parameter for associating each record in the target table T with records in the source table S when the distance between the value of the first structural attribute and the value of the second structural attribute is 1 or less.
  • the exact map generator 32 generates a parameter for associating each record in the target table T with records in the source table S when the distance between the value of the first structural attribute and the value of the second structural attribute is 2 or less or 3 or less.
  • a “URL” string in target table T is recorded as the first string attribute and a “URL” string in the source table S is recorded as the second string attribute. Therefore, these two strings are associated. As a result, the parameter P 17 shown in FIG. 12 is generated.
  • the map data M in FIG. 12 shows the results of associating each record in the target table T with records in the source table S having a degree of similarity of 1 or less. In one example, only one record from the source table is associated with the first record in the target table.
  • the attribute specifying unit 33 specifies attributes with a shared perspective in the first table and the second table. Specifically, the attribute specifying unit 33 specifies the attribute of data indicated by each string in the first table and the attribute of data indicated by each string in the second table as the same attribute. For example, in the case of the geographic data type, the attribute specifying unit 33 specifies first geographic attributes having the same data type as the first geographic data type in the first table and second geographic attributes having the same data type as the second geographic data type in the second table. In this way, strings having a geographic data type can be specified in each table. The attribute specifying unit 33 may specify the attribute of strings in the first table and the second table from string attribute information inputted to the input unit 10 .
  • the map parameter generator 30 (more specifically, each generator in the map parameter generator 30 ) may store in the storage unit 80 parameters including the degree of geographic (or temporal) relationship between strings in the first table including a first geographic (or temporal) attribute whose geographic (or temporal) relationship is to be determined and strings in the second table including a second geographic (or temporal) attributes.
  • the map parameter generator 30 may store in the storage unit 80 parameter P 11 in FIG. 5 or parameter P 15 in FIG. 10 .
  • FIG. 13 is a diagram used to explain an example of generated map parameters.
  • the input unit 10 receives target table T, source table S 1 and source table S 2 shown in FIG. 13 , and portion C 1 of the config file shown in FIG. 2 .
  • map parameter P 16 is generated based on the KNearestMap function using the attribute in the “target_location” string in target table T as the first geographic attribute, the attribute in the “community” string in source table S 1 as the second geographic attribute.
  • the map parameter generator 30 (more specifically, each generator in the map parameter generator 30 ) generates the thirteen map parameters P 11 - 16 shown in FIG. 13 from this information.
  • the filter parameter generator 50 includes exact filter generator 51 .
  • the exact filter generator 51 generates a filter parameter in which a column in the second table is associated with an extraction condition applied to the column.
  • the exact filter generator 51 may generate a filter parameter based, for example, on the information defined in portion C 2 of the config file shown in FIG. 2 . Extraction conditions may be stored beforehand in the storage unit 80 and the exact filter generator 51 may retrieve an extraction condition to generate a filter parameter.
  • the exact filter generator 51 may also combine multiple extraction conditions to generate an extraction condition. Any number of extraction conditions may be combined.
  • the input unit 10 may, for example, receive the maximum number for such combinations. For example, as shown in FIG. 2 , a parameter indicating the maximum number of combinations (“max_combination_filter_length”) may be set in the C 4 portion of the config file.
  • the reduction parameter generator 60 (more specifically, each generator in the reduction parameter generator 60 ) generates a parameter indicating the method used to reduce the data in each row of the second table.
  • the reduction parameter generator 60 includes a geo-reduce generator 70 and a numerical reduce generator 61 .
  • the geo-reduce generator 70 (more specifically, each generator in the geo-reduce generator 70 ) generates a reduction parameter indicating the method used to reduce data in each row using values in a column including geographic attributes in the second table. Specifically, the geo-reduce generator 70 calculates the statistical value of the geographic attribute based on the indicated reduction method.
  • the input unit 10 may receive the indicated reduction method.
  • the reduction method may be defined based on geographic attribute analysis data type as indicated in portion C 3 of the config file in FIG. 2 and the reduction parameter may be generated based on the defined reduction method. The content is described below in detail.
  • the “Point” line in portion C 3 defines the reduction method when the second geographic attribute (more specifically, the geographic data type) is expressed by a point (Point).
  • sum (“sum,” “distance”) defines a reduction method in which the total distance based on a first geographic attribute value and a second geographic attribute value among records in the second table associated with records in the first table is calculated as a statistical value.
  • avg “distance”
  • distance defines a reduction method in which the average distance based on a first geographic attribute value and a second geographic attribute value among records in the second table associated with records in the first table is calculated as a statistical value.
  • count defines a reduction method in which the number of records in the second table associated with each record in the first table (that is, target variables) is calculated as a statistical value.
  • the “Area” line in portion C 3 defines the reduction method when the second geographic attribute (more specifically, the geographic data type) is expressed by an area (Area).
  • sum (“sum,” “areaSize”) defines a reduction method in which the total size of the area in the second geographic attribute value among records in the second table associated with records in the first table is calculated as a statistical value.
  • count defines a reduction method in which the number of records in the second table associated with each record in the first table (that is, target variables) is calculated as a statistical value.
  • the geo-reduce generator 70 has a point reduce generator 71 and an area reduce generator 72 .
  • the point reduce generator 71 generates a reduction parameter for calculating the distance based on the value of the first geographic attribute and the value of the second geographic attribute as a statistical value.
  • the records in the second table to be processed are each associated with a record in the first table.
  • records are associated with each other that satisfy a certain condition such as the value of the first geographic attribute and the value of the second geographic attribute matching or falling within a certain range.
  • the point reduce generator 71 When the value of the first geographic attribute and the value of the second geographic attribute satisfy a predetermined condition, the point reduce generator 71 generates a reduction parameter for calculating the distance as a statistical value based on the value of the first geographic attribute and the value of the second geographic attribute satisfying the condition.
  • the calculated statistical value is used as a feature.
  • the point reduce generator 71 When at least one of (“sum,” “distance”), (“avg,” “distance”) and (“count”) in FIG. 2 has been set in the config file, the point reduce generator 71 generates a reduction parameter for calculating the statistical value of the distance.
  • FIG. 14 is a diagram used to explain an example of processing performed to generate reduction parameters for calculating distance statistics.
  • three types of reduction method are set in the config file. Therefore, the point reduce generator 71 calculates a reduction parameter for calculating the total and average distance between a record in the source table and a record in the target table and a reduction parameter for calculating the number of records in the associated source table.
  • the point reduce generator 71 may generate a reduction parameter in which the column name in the source table to be reduced, the column name in the target table to be associated, the reduction content (distance), and the reduce function are associated.
  • the reduce list R 21 shown in FIG. 14 shows the result of reducing map data M 11 based on the reduction parameter used to calculate the distance totals.
  • the area reduce generator 72 generates a reduction parameter for calculating the statistical value of an area based on the value of the second geographic attribute.
  • the records in the second table to be processed are each associated with a record in the first table.
  • the area reduce generator 72 When at least one of (“sum,” “areaSize”), (“avg,” “areaSize”) and (“count”) in FIG. 2 has been set in the config file, the area reduce generator 72 generates a reduction parameter for calculating the statistical value of the area.
  • FIG. 15 is a diagram used to explain an example of processing performed to generate reduction parameters for calculating area statistics.
  • three types of reduction method are set in the config file. Therefore, the area reduce generator 72 calculates a reduction parameter for calculating the total and average area of the records in the source table associated with each of the records in the target table, and a reduction parameter for calculating the number of records in the associated source table.
  • the area reduce generator 72 may generate a reduction parameter in which the column name in the source table to be reduced, the reduction content (area), and the reduce function are associated.
  • the reduce list R 22 shown in FIG. 15 shows the result of reducing map data M 14 based on the reduction parameter used to calculate the area totals.
  • the numerical reduce generator 61 generates a reduction parameter indicating the method used to reduce the data in each line using a value including attributes with a numerical value (numerical attribute below) in the second table. Specifically, the numerical reduce generator 61 calculates numerical statistics based on the indicated reduction method.
  • the input unit 10 may receive the indicated reduction method.
  • the reduction method for the numerical attributes may be defined as indicated in portion C 3 of the config file in FIG. 2 , and a reduction parameter generated based on the defined reduction method.
  • a reduction parameter for calculating the total and average for the columns with numerical attributes has been indicated.
  • the reduction parameter generator 60 (more specifically, the generators in the reduction parameter generator 60 ) may store the generated reduction parameter in the storage unit 80 .
  • FIG. 16 is a diagram used to explain an example of generated reduction parameters.
  • the input unit 10 receives target table T, source table S 1 and source table S 2 in FIG. 16 and portion C 3 in the config file shown in FIG. 2 .
  • Reduction parameter P 23 is a reduction parameter for numerical attribute columns in source table S 2 .
  • Reduction parameter P 24 is a reduction parameter for numerical attribute columns in source table S 1 .
  • the reduction parameter generator 60 (more specifically, the generators in the reduction parameter generator 60 ) generates the sixteen map parameters P 21 - 24 in FIG. 16 from this information.
  • the feature descriptor generator 81 generates a feature descriptor generator for generating the features described above from the first table and the second table. Specifically, the feature descriptor generator 81 generates a feature descriptor using (combining) the combination condition (map parameter) and reduction condition (reduction parameter) described above. The feature descriptor generator 81 may generate a feature descriptor using (combining) an extraction condition (filter parameter) in addition to the combination condition and reduction condition.
  • the feature descriptor generator 81 may generate a map parameter previously combining a map parameter for geographic attributes and a map parameter for temporal attributes among the combination conditions (map parameters). For example, when “True” has been set in the parameter “time_spatial_map_combination” as in portion C 4 of the config file shown in FIG. 2 , the feature descriptor generator 81 may determine that a map parameter for geographic attributes is to be combined with a map parameter for temporal attributes.
  • target table T and source tables S 1 and S 2 in FIG. 13 are inputted.
  • the variable (target variable) for the prediction target is a variable indicating the number of passengers picked up in target table T (pickup_number).
  • FIG. 18 is a diagram used to explain an example of a method used to combine parameters and generate a feature descriptor.
  • FIG. 18 ( a ) shows a combination example used to generate a feature descriptor for generating a feature from target table T and source table S 1 .
  • FIG. 18 ( b ) shows a combination example used to generate a feature descriptor for generating a feature from target table T and source table S 2 .
  • a map parameter is used that combines a map parameter for a geographic attribute and a map parameter for a temporal attribute.
  • the feature descriptor generator 81 selects one each from the map parameters, filter parameters, and reduction parameters to generate a combination of the parameters.
  • map parameters and seven reduction parameters are generated.
  • the feature descriptor generator 81 selects one parameter each from the map parameters and the reduction parameters and generates a combination of the parameters.
  • the feature descriptor generator 81 generates a feature descriptor based on the generated combination. More specifically, the feature descriptor generator 81 converts the parameters in the generated combination into the format of the query language for operating and defining table data. For example, the feature descriptor generator 81 may use SQL as the query language.
  • the feature descriptor generator 81 may apply the parameters to a template for producing an SQL statement to generate a feature descriptor.
  • the template for generating an SQL statement may be prepared for each parameter in advance, and the feature descriptor generator 81 may apply each parameter in the generated combination to the template in successive order to generate an SQL statement.
  • the feature descriptor is defined as an SQL statement and each of the selected parameters corresponds to a parameter for generating an SQL statement.
  • various feature descriptors can be expressed as combinations of simple elements. Therefore, various feature candidates can be efficiently generated using table data. For example, in the example described above, 130 different features can be easily generated by generating four map parameters and nine reduction parameters and by generating 14 map parameters and seven reduction parameters. Because the definitions of each parameter generated can be reused, the labor required to generate feature descriptors can be reduced.
  • the feature generator 82 generates features using feature descriptors.
  • feature descriptors may include parameters for calculating distances as statistical values as described above.
  • the feature generator 82 may calculate distances as statistical values by reducing the records in the second table meeting a predetermined condition by each record with a first geographic attribute based on a feature descriptor.
  • the feature generator 82 may calculate the total or average for the distance in second table geographic attributes satisfying a predetermined condition with each record having a first table geographic attribute to reduce the records in the second table. The feature generator 82 may then add the calculated total or average for the distance as a feature to an attribute in the first table.
  • the feature generator 82 may calculate the number of records with geographic attributes satisfying a predetermined condition in the second table with each record having a geographic attribute in the first table to reduce the records in the second table. The feature generator 82 may then add the calculated number of records as a feature to an attribute in the first table.
  • the feature generator 82 can add generated features to attributes in the first table, the feature generator 82 can be said to be an attribute adding means. Because features generated by the feature generator 82 are candidates for the feature selector 83 to select as described later, the features can also be referred to as feature candidates.
  • the feature generator 82 generates feature candidates using feature descriptors.
  • feature candidates may also be generated directly by the feature generator 82 from the first table and the second table using a similarity function, a combination condition, and a reduction condition.
  • the degree of similarity calculated from the value of a first attribute and the value of a second attribute is a combination condition used to combine records in the first table including values for first attributes and records in the second table including values for second attributes that satisfy the condition.
  • a reduction condition is expressed as a reduction method for records in the second table and columns to be reduced.
  • the feature generator 82 may generate features by combining combination conditions with reduction conditions. By combining combination conditions and reduction conditions, the same effect can be achieved as the feature descriptor generator 81 generating feature descriptors.
  • the feature selector 83 selects the optimum feature for a prediction from among the generated features. Any feature selecting method may be used.
  • the feature selector 83 may select a feature using, for example, L1 regularization.
  • the algorithm used to select a feature is not limited to L1 regularization.
  • the feature selector 83 may select the optimum feature for a prediction based on the algorithm used to select the feature.
  • the output unit 90 outputs the generated feature.
  • the output unit 90 may output only the feature selected by the feature selector 83 or may output all of the features generated by the feature generator 82 .
  • the learning unit 91 learns a prediction model using the generated feature.
  • the learning unit 91 learns prediction models using added attributes as features. Specifically, the learning unit 91 applies data from the first table and the second table to the generated feature to produce training data.
  • the learning unit 91 uses generated features as candidates for explanatory variables to learn a model that predicts the values to be predicted. Any model learning method can be used.
  • the predicting unit 92 makes predictions using the model learned by the learning unit 91 . Specifically, the predicting unit 92 applies data from the first table and the second table to a generated feature to generate prediction data. The predicting unit 92 applies the generated prediction data to the learned model and obtains prediction results.
  • the input unit 10 , geo-coder 20 , map parameter generator 30 , filter parameter generator 50 , reduction parameter generator 60 , feature descriptor generator 81 , feature generator 82 , feature selector 83 , output unit 90 , learning unit 91 , and predicting unit 92 are realized by a computer processor that operates according to a program (information processing program) such as a central processing unit (CPU), graphics processing unit (GPU), or field-programmable gate array (FPGA).
  • the map parameter generator 30 is realized by the geo-map generator 40 (distance map generator 41 , inclusion map generator 42 , overlap map generator 43 , same area map generator 44 ), time difference map generator 31 , exact map generator 32 , and attribute specifying unit 33 .
  • the reduction parameter generator 60 is realized by the geo-reduce generator 70 (point reduce generator 71 , area reduce generator 72 ) and the numerical reduce generator 61 .
  • the input unit 10 , geo-coder 20 , map parameter generator 30 , filter parameter generator 50 , reduction parameter generator 60 , feature descriptor generator 81 , feature generator 82 , feature selector 83 , output unit 90 , learning unit 91 , and predicting unit 92 may be operated in accordance with a program stored in the storage unit 80 and retrieved by a processor.
  • the functions of the information processing system may be provided in the SaaS (software as a service) format.
  • the input unit 10 , geo-coder 20 , map parameter generator 30 , filter parameter generator 50 , reduction parameter generator 60 , feature descriptor generator 81 , feature generator 82 , feature selector 83 , output unit 90 , learning unit 91 , and predicting unit 92 may also be realized by dedicated hardware. Some or all of the components in these devices may be realized by a combination of general or dedicated circuitry and processors. These may be mounted in a single chip or across multiple chips connected via a bus. Some or all of the components in these devices may be realized by a combination of the circuitry and processors described above.
  • the plurality of information processing devices and circuits may be arranged centrally or may be distributed.
  • the information processing devices and the circuits may be realized in a form connected via a communication network, such as in a client and server system or in a cloud computing system.
  • the information processing system 100 in the present embodiment may be realized as a single information processing device. Because some or all of the information processing system 100 in the present embodiment is used to generate features, the device including the function of producing a feature can be referred to as the feature generating device.
  • FIG. 19 is a flowchart showing an example of processing performed to generate combination conditions.
  • the input unit 10 acquires a first table including a prediction target and first geographic attributes and a second table including second geographic attributes (Step S 11 ).
  • the input unit 10 also receives a geographic relation and the degree of geographic relation (Step S 12 ).
  • the map parameter generator 30 generates a combination condition for combining records in the first table with records in the second table so that the relationship between the value of the first geographic attribute and the value of the second geographic attribute satisfy the degree of geographic relationship (Step S 13 ).
  • FIG. 20 is a flowchart showing another example of processing performed to generate combination conditions.
  • the input unit 10 acquires a first table including a prediction target and first temporal attributes and a second table including second temporal attributes (Step S 21 ).
  • the input unit 10 also receives a temporal relation and the degree of temporal relation (Step S 22 ).
  • the map parameter generator 30 generates a combination condition for combining records in the first table with records in the second table so that the relationship between the value of the first temporal attribute and the value of the second temporal attribute satisfy the degree of temporal relationship (Step S 23 ).
  • FIG. 21 is a flowchart showing an example of processing performed to generate features.
  • the input unit 10 acquires a first table including a prediction target and first geographic attributes and a second table including second geographic attributes (Step S 31 ).
  • the feature generator 82 calculates the statistical value of the distance when the value of the second geographic attribute satisfies a predetermined condition relative to the value of the first geographic attribute (Step S 32 ), and the calculated statistical value is added to an attribute of the first table as a feature (Step S 33 ).
  • FIG. 22 is a flowchart showing another example of processing performed to generate features.
  • the input unit 10 acquires a first table including a prediction target and first attributes and a second table including second attributes (Step S 41 ).
  • the input unit 10 also receives a similarity function used to calculate the degree of similarity between a first attribute and a second attribute and a similarity condition (such as a similarity threshold value) (Step S 42 ).
  • Feature candidates are generated from the first table and the second table using a combination condition and reduction condition in accordance with the similarity function (Step S 43 ).
  • the feature selector 83 selects the most appropriate feature for a prediction from the feature candidates (Step S 44 ).
  • the input unit 10 acquires a first table including a prediction target and first geographic attributes and a second table including second geographic attributes.
  • the input unit 10 also receives a geographic relation and the degree of geographic relation.
  • the map parameter generator 30 generates a combination condition for combining records in the first table with records in the second table so that the relationship between the value of the first geographic attribute and the value of the second geographic attribute satisfy the degree of geographic relationship.
  • the input unit 10 acquires a first table including a prediction target and first temporal attributes and a second table including second temporal attributes.
  • the input unit 10 also receives a temporal relation and the degree of temporal relation.
  • the map parameter generator 30 generates a combination condition for combining records in the first table with records in the second table so that the relationship between the value of the first temporal attribute and the value of the second temporal attribute satisfy the degree of temporal relationship. In this way, the amount of labor required to associate information via geographic information or temporal information can be reduced. As a result, the burden on a computer to process information expressed using a variety of expressions can be reduced.
  • the input unit 10 acquires a first table including a prediction target and first geographic attributes and a second table including second geographic attributes.
  • the feature generator 82 calculates the statistical value of the distance when the value of the second geographic attribute satisfies a predetermined condition relative to the value of the first geographic attribute, and the calculated statistical value is added to an attribute of the first table as a feature. In this way, features can be generated efficiently from information sources having geographic information.
  • the input unit 10 acquires a first table including a prediction target and first attributes and a second table including second attributes.
  • the input unit 10 also receives a similarity function used to calculate the degree of similarity between a first attribute and a second attribute and a similarity condition.
  • Feature candidates are generated from the first table and the second table using a combination condition and reduction condition in accordance with the similarity function.
  • the feature selector 83 selects the most appropriate feature for a prediction from the feature candidates. In this way, the labor required for analysts to generate features can be reduced.
  • FIG. 23 is a block diagram showing an overview of a feature generating device of the present invention.
  • a feature generating device 380 in the present invention comprises: a table acquiring means 381 (input unit 10 ) for acquiring a first table (target table) including a prediction target and a first attribute, and a second table (source table) including a second attribute; a receiving means 382 (input unit 10 ) for receiving a similarity function and a similarity condition used to calculate the degree of similarity between the first attribute and the second attribute; a feature generating means 383 (feature generator 82 ) for generating feature candidates from the first table and the second table able to affect the prediction target using a combination condition (map parameter) for combining a record in the first table including a value for the first attribute and a record in the second table including a value for the second attribute whose degree of similarity calculated from the value of the first attribute and the value of the second attribute using the similarity function satisfies the condition (similarity
  • This configuration can reduce the amount of labor performed by an analyst to generate features.
  • the feature generating means 383 may generate a number of features combining a combination condition and a reduction condition using a plurality of combination conditions and a plurality of reduction conditions.
  • the first attribute and the second attribute may be geographic attributes, and the similarity function may be defined as a function calculating a higher degree of similarity for closer distances.
  • first attribute and the second attribute may be temporal attributes
  • the similarity function may be defined as a function calculating a higher degree of similarity for smaller time differences.
  • the first attribute and the second attribute may be string attributes
  • the similarity function may be defined as a function calculating a higher degree of similarity for a higher match between text in the strings.
  • the first attribute and the second attribute may be structural attributes, and the similarity function may be defined as a function calculating a higher degree of similarity for closer distances to a shared node.
  • the feature generating device 380 may further comprise a function generating means (feature descriptor generator 81 ) for generating a feature descriptor using a combination condition, a reduction condition, and an extraction condition (filter parameter, etc.) for extracting a record satisfying the condition from the second table.
  • a function generating means feature descriptor generator 81
  • filter parameter filter parameter, etc.
  • FIG. 24 is a schematic block diagram showing the configuration of a computer related to at least one embodiment.
  • the computer 1000 includes a processor 1001 , a main storage device 1002 , an auxiliary storage device 1003 , and an interface 1004 .
  • This information processing system may be installed in a computer 1000 .
  • the operations performed by each processing unit may be stored in an auxiliary storage device 1003 in the format of a program (combination condition generating program).
  • the processor 1001 may retrieve the program from the auxiliary storage device 1003 and load the program in the main storage device 1002 to execute processing in accordance with the program.
  • the auxiliary storage device 1003 in at least one embodiment is a non-temporary physical medium.
  • An example of a non-temporary physical medium is a magnetic disk, a magneto-optical disk, a CD-ROM, a DVD-ROM, or a semiconductor memory connected via the interface 1004 .
  • the computer 1000 receiving the program may load the program in the main storage device 1002 and execute the processing described above.
  • the program may realize some of the functions described above.
  • the program may also combine these functions with those of another program already stored in the auxiliary storage device in the form of a so-called difference file (difference program).
  • a feature generating device comprising: a table acquiring means for acquiring a first table including a prediction target and a first attribute, and a second table including a second attribute; a receiving means for receiving a similarity function and a similarity condition used to calculate the degree of similarity between the first attribute and the second attribute; a feature generating means for generating feature candidates from the first table and the second table able to affect the prediction target using a combination condition for combining a record in the first table including a value for the first attribute and a record in the second table including a value for the second attribute whose degree of similarity calculated from the value of the first attribute and the value of the second attribute using the similarity function satisfies the condition, a reduction method for the plurality of records in the second table, and a reduction condition represented by a column to be reduced; and a feature selecting means for selecting the optimum feature for prediction from among the feature candidates.
  • a feature generating device according to addendum 1, wherein the feature generating means generates a number of features combining a combination condition and a reduction condition using a plurality of combination conditions and a plurality of reduction conditions.
  • a feature generating device according to addendum 1 or addendum 2, wherein the first attribute and the second attribute are geographic attributes, and the similarity function is defined as a function calculating a higher degree of similarity for closer distances.
  • a feature generating device according to addendum 1 or addendum 2, wherein the first attribute and the second attribute are temporal attributes, and the similarity function is defined as a function calculating a higher degree of similarity for smaller time differences.
  • a feature generating device according to addendum 1 or addendum 2, wherein the first attribute and the second attribute are string attributes, and the similarity function is defined as a function calculating a higher degree of similarity for a higher match between text in the strings.
  • a feature generating device according to addendum 1 or addendum 2, wherein the first attribute and the second attribute are structural attributes, and the similarity function is defined as a function calculating a higher degree of similarity for closer distances to a shared node.
  • a feature generating device according to any one of addendum 1 to addendum 6, further comprising a function generating means for generating a feature descriptor using a combination condition, a reduction condition, and an extraction condition for extracting a record satisfying the condition from the second table.
  • a feature generating method comprising: acquiring a first table including a prediction target and a first attribute, and a second table including a second attribute; receiving a similarity function and a similarity condition used to calculate the degree of similarity between the first attribute and the second attribute; generating feature candidates from the first table and the second table able to affect the prediction target using a combination condition for combining a record in the first table including a value for the first attribute and a record in the second table including a value for the second attribute whose degree of similarity calculated from the value of the first attribute and the value of the second attribute using the similarity function satisfies the condition, a reduction method for the plurality of records in the second table, and a reduction condition represented by a column to be reduced; and selecting the optimum feature for prediction from among the feature candidates.
  • a feature generating method wherein a number of features combining a combination condition and a reduction condition are generated using a plurality of combination conditions and a plurality of reduction conditions.
  • a feature generating program causing a computer to execute: a table acquiring process for acquiring a first table including a prediction target and a first attribute, and a second table including a second attribute; a receiving process for receiving a similarity function and a similarity condition used to calculate the degree of similarity between the first attribute and the second attribute; a feature generating process for generating feature candidates from the first table and the second table able to affect the prediction target using a combination condition for combining a record in the first table including a value for the first attribute and a record in the second table including a value for the second attribute whose degree of similarity calculated from the value of the first attribute and the value of the second attribute using the similarity function satisfies the condition, a reduction method for the plurality of records in the second table, and a reduction condition represented by a column to be reduced; and a feature selecting process for selecting the optimum feature for prediction from among the feature candidates.
  • a feature generating program according to addendum 10, wherein the computer is caused in the feature generating process to generate a number of features combining a combination condition and a reduction condition using a plurality of combination conditions and a plurality of reduction conditions.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Databases & Information Systems (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Software Systems (AREA)
  • Computational Linguistics (AREA)
  • Mathematical Physics (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Evolutionary Computation (AREA)
  • Probability & Statistics with Applications (AREA)
  • Fuzzy Systems (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • General Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
US16/753,757 2017-10-05 2018-06-12 Feature value generation device, feature value generation method, and feature value generation program Active 2038-11-27 US11514062B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US16/753,757 US11514062B2 (en) 2017-10-05 2018-06-12 Feature value generation device, feature value generation method, and feature value generation program

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US201762568397P 2017-10-05 2017-10-05
US16/753,757 US11514062B2 (en) 2017-10-05 2018-06-12 Feature value generation device, feature value generation method, and feature value generation program
PCT/JP2018/022429 WO2019069507A1 (ja) 2017-10-05 2018-06-12 特徴量生成装置、特徴量生成方法および特徴量生成プログラム

Publications (2)

Publication Number Publication Date
US20200301921A1 US20200301921A1 (en) 2020-09-24
US11514062B2 true US11514062B2 (en) 2022-11-29

Family

ID=65994215

Family Applications (1)

Application Number Title Priority Date Filing Date
US16/753,757 Active 2038-11-27 US11514062B2 (en) 2017-10-05 2018-06-12 Feature value generation device, feature value generation method, and feature value generation program

Country Status (5)

Country Link
US (1) US11514062B2 (ja)
EP (1) EP3696686A4 (ja)
JP (1) JPWO2019069507A1 (ja)
SG (1) SG11202003814TA (ja)
WO (1) WO2019069507A1 (ja)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7098327B2 (ja) 2015-11-25 2022-07-11 ドットデータ インコーポレイテッド 情報処理システム、関数作成方法および関数作成プログラム
EP3605363A4 (en) 2017-03-30 2020-02-26 Nec Corporation INFORMATION PROCESSING SYSTEM, CHARACTERISTICS, AND CHARACTERISTICS PROGRAM
CN114943607A (zh) * 2022-06-02 2022-08-26 支付宝(杭州)信息技术有限公司 特征发现方法、属性预测方法和装置

Citations (76)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5873088A (en) 1990-08-31 1999-02-16 Fujitsu Limited Derived data base processing system enabling one program to access a plurality of data basis
JPH11219367A (ja) 1998-02-03 1999-08-10 Nippon Telegr & Teleph Corp <Ntt> 住所情報による異種データの結合処理方法および装置
WO2000065484A2 (en) 1999-04-28 2000-11-02 Arena Pharmaceuticals, Inc. System and method for database similarity join
JP2002007435A (ja) 2000-06-20 2002-01-11 Nec Corp 対話的分析データベースシステム及び対話的分析プログラムを記録した記録媒体
JP2002109150A (ja) 2000-09-28 2002-04-12 Fuji Electric Co Ltd 時系列データの適応的予測方法
US20020147599A1 (en) 2001-04-05 2002-10-10 International Business Machines Corporation Method and system for simplifying the use of data mining in domain-specific analytic applications by packaging predefined data mining models
US20020198889A1 (en) 2001-04-26 2002-12-26 International Business Machines Corporation Method and system for data mining automation in domain-specific analytic applications
US6718346B1 (en) 2000-08-17 2004-04-06 3Com Corporation Generating summary data for a requested time period having a requested start time and end time a plurality of data records
US20050102303A1 (en) 2003-11-12 2005-05-12 International Business Machines Corporation Computer-implemented method, system and program product for mapping a user data schema to a mining model schema
JP2005302040A (ja) 2004-04-14 2005-10-27 Microsoft Corp 目標変数の自動データパースペクティブ生成
US6973459B1 (en) 2002-05-10 2005-12-06 Oracle International Corporation Adaptive Bayes Network data mining modeling
US20060173873A1 (en) 2000-03-03 2006-08-03 Michel Prompt System and method for providing access to databases via directories and other hierarchical structures and interfaces
US20060218132A1 (en) 2005-03-25 2006-09-28 Oracle International Corporation Predictive data mining SQL functions (operators)
US20070011134A1 (en) 2005-07-05 2007-01-11 Justin Langseth System and method of making unstructured data available to structured data analysis tools
US20070136346A1 (en) 2004-02-03 2007-06-14 Morris John M Executing a join plan using data compression
US20070185868A1 (en) 2006-02-08 2007-08-09 Roth Mary A Method and apparatus for semantic search of schema repositories
US20070203893A1 (en) 2006-02-27 2007-08-30 Business Objects, S.A. Apparatus and method for federated querying of unstructured data
JP2008102736A (ja) 2006-10-19 2008-05-01 Toshiba Corp 検索装置および検索方法
US20080263093A1 (en) 2007-04-20 2008-10-23 International Business Machines Corporation Generation of a set of pre-filters from a set of event subscriptions to more efficiently select events of interest
US7546312B1 (en) 2005-09-23 2009-06-09 Emc Corporation System and methods for modeling a report query database
US20090164943A1 (en) 2007-12-19 2009-06-25 Teradata Corporation Open model ingestion for Master Data Management
US20090162824A1 (en) 2007-12-21 2009-06-25 Heck Larry P Automated learning from a question and answering network of humans
US20100082601A1 (en) 2008-09-29 2010-04-01 Bhashyam Ramesh Method, database system and computer program for joining temporal database tables
US20100106747A1 (en) 2008-10-23 2010-04-29 Benjamin Honzal Dynamically building and populating data marts with data stored in repositories
US7991583B2 (en) 2006-04-13 2011-08-02 Dspace Digital Signal Processing And Control Engineering Gmbh Diagnosis in automotive applications
JP2011242851A (ja) 2010-05-14 2011-12-01 Nippon Telegr & Teleph Corp <Ntt> 和集合集約処理と等結合処理の組み合わせ方法及びデータベースシステム及びプログラム
US20110302187A1 (en) 2010-06-04 2011-12-08 Fujitsu Limited Schema definition generating device and schema definition generating method
WO2012025493A1 (en) 2010-08-25 2012-03-01 International Business Machines Corporation Geospatial database integration
JP2012059173A (ja) 2010-09-13 2012-03-22 Fujitsu Marketing Ltd レポート作成装置およびそのプログラム
US20120173226A1 (en) 2010-12-30 2012-07-05 International Business Machines Corporation Table merging with row data reduction
WO2012128207A1 (ja) 2011-03-18 2012-09-27 日本電気株式会社 多変量データの混合モデル推定装置、混合モデル推定方法および混合モデル推定プログラム
CN102714767A (zh) 2010-01-18 2012-10-03 日本电气株式会社 信息终端装置、信息终端装置的操作方法及其程序
US20120330931A1 (en) 2010-04-09 2012-12-27 Sadaki Nakano Database management method, computer, sensor network system and database search program
JP2013152656A (ja) 2012-01-26 2013-08-08 Fujitsu Ltd 説明変数の決定のための情報処理装置、情報処理方法及びプログラム
JP2013164724A (ja) 2012-02-10 2013-08-22 Nippon Telegr & Teleph Corp <Ntt> 情報取引システム
JP2013182498A (ja) 2012-03-02 2013-09-12 Hitachi Solutions East Japan Ltd 需要予測装置、需要予測プログラムおよび需要予測方法
US20130246996A1 (en) 2012-03-19 2013-09-19 Enterpriseweb Llc Declarative Software Application Meta-Model and System for Self-Modification
US8620934B2 (en) 2010-05-05 2013-12-31 University Of Washington Through Its Center For Commercialization Systems and methods for selecting data elements, such as population members, from a data source
US20140136471A1 (en) 2012-11-13 2014-05-15 International Business Machines Corporation Rapid Provisioning of Information for Business Analytics
US20140188918A1 (en) 2012-04-03 2014-07-03 Sas Institute Inc. Techniques to perform in-database computational programming
US20140201194A1 (en) 2013-01-17 2014-07-17 Vidyasagar REDDY Systems and methods for searching data structures of a database
US20140223284A1 (en) 2013-02-01 2014-08-07 Brokersavant, Inc. Machine learning data annotation apparatuses, methods and systems
US20140279074A1 (en) 2013-03-15 2014-09-18 Turn Inc. Data management platform for digital advertising
WO2014208205A1 (ja) 2013-06-26 2014-12-31 前田建設工業株式会社 表形式データ処理プログラム、方法及び装置
CN104408149A (zh) 2014-12-04 2015-03-11 威海北洋电气集团股份有限公司 基于社交网络分析的犯罪嫌疑人挖掘关联方法及系统
WO2015045318A1 (ja) 2013-09-27 2015-04-02 日本電気株式会社 情報処理システム、情報処理方法およびプログラムを記憶する記録媒体
JP2015075970A (ja) 2013-10-09 2015-04-20 前田建設工業株式会社 表形式データ処理プログラム、方法、及び装置
WO2015085916A1 (zh) 2013-12-10 2015-06-18 中国银联股份有限公司 数据挖掘方法
CN104881424A (zh) 2015-03-13 2015-09-02 国家电网公司 一种基于正则表达式的电力大数据采集、存储及分析方法
US9130832B1 (en) 2014-10-09 2015-09-08 Splunk, Inc. Creating entity definition from a file
US9146984B1 (en) 2013-03-15 2015-09-29 Google Inc. Enhancing queries for data tables with nested fields
US20150310082A1 (en) 2014-04-24 2015-10-29 Luke Qing Han Hadoop olap engine
US20150309990A1 (en) 2014-04-25 2015-10-29 International Business Machines Corporation Producing Insight Information from Tables Using Natural Language Processing
US20150356123A1 (en) 2014-06-04 2015-12-10 Waterline Data Science, Inc. Systems and methods for management of data platforms
WO2015186278A1 (ja) 2014-06-03 2015-12-10 日本電気株式会社 属性列挙システム、属性列挙方法および属性列挙プログラム
US20150379428A1 (en) 2014-06-30 2015-12-31 Amazon Technologies, Inc. Concurrent binning of machine learning data
US20150379430A1 (en) 2014-06-30 2015-12-31 Amazon Technologies, Inc. Efficient duplicate detection for machine learning data sets
WO2016017086A1 (ja) 2014-07-31 2016-02-04 日本電気株式会社 行動特性予測システム、行動特性予測器、方法およびプログラム
US20160042039A1 (en) 2014-08-06 2016-02-11 Martin Kaufmann Timeline index for partitioned temporal database tables
US20160103897A1 (en) 2014-10-10 2016-04-14 Fmr, Llc Database, Data Structure and Framework Transformer Apparatuses, Methods and Systems
US20160173122A1 (en) 2013-08-21 2016-06-16 Hitachi, Ltd. System That Reconfigures Usage of a Storage Device and Method Thereof
US20160342606A1 (en) 2015-05-21 2016-11-24 Oracle International Corporation Declarative mapreduce using regular expressions
WO2017090475A1 (ja) 2015-11-25 2017-06-01 日本電気株式会社 情報処理システム、関数作成方法および関数作成プログラム
US9934266B2 (en) 2015-05-14 2018-04-03 Walleye Software, LLC Memory-efficient computer system for dynamic updating of join processing
US20180095952A1 (en) 2016-09-15 2018-04-05 Gb Gas Holdings Limited System for data management in a large scale data repository
US20180121442A1 (en) 2016-11-02 2018-05-03 Sap Se Hierarchical map visualizations of geo-enriched data
US20190043127A1 (en) * 2017-08-04 2019-02-07 Airbnb, Inc. Verification model using neural networks
US10452632B1 (en) 2013-06-29 2019-10-22 Teradata Us, Inc. Multi-input SQL-MR
US20200057948A1 (en) 2016-10-31 2020-02-20 Nec Corporation Automatic prediction system, automatic prediction method and automatic prediction program
US10713589B1 (en) 2016-03-03 2020-07-14 Amazon Technologies, Inc. Consistent sort-based record-level shuffling of machine learning data
US20200334246A1 (en) * 2017-10-05 2020-10-22 Dotdata, Inc. Information processing device, combination condition generation method, and combination condition generation program
US20200387664A1 (en) * 2017-03-30 2020-12-10 Dotdata, Inc. Information processing system, feature description method and feature description program
US20200387505A1 (en) 2017-03-30 2020-12-10 Dotdata, Inc. Information processing system, feature description method and feature description program
US20210342341A1 (en) 2017-12-22 2021-11-04 Dotdata, Inc. Data analysis assistance device, data analysis assistance method, and data analysis assistance program
US20210357372A1 (en) 2017-12-22 2021-11-18 Dotdata, Inc. Data analysis assistance device, data analysis assistance method, and data analysis assistance program
US11182691B1 (en) 2014-08-14 2021-11-23 Amazon Technologies, Inc. Category-based sampling of machine learning data

Patent Citations (89)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5873088A (en) 1990-08-31 1999-02-16 Fujitsu Limited Derived data base processing system enabling one program to access a plurality of data basis
JPH11219367A (ja) 1998-02-03 1999-08-10 Nippon Telegr & Teleph Corp <Ntt> 住所情報による異種データの結合処理方法および装置
WO2000065484A2 (en) 1999-04-28 2000-11-02 Arena Pharmaceuticals, Inc. System and method for database similarity join
US20040153250A1 (en) 1999-04-28 2004-08-05 Hurst John R. System and method for database similarity join
US20060173873A1 (en) 2000-03-03 2006-08-03 Michel Prompt System and method for providing access to databases via directories and other hierarchical structures and interfaces
JP2002007435A (ja) 2000-06-20 2002-01-11 Nec Corp 対話的分析データベースシステム及び対話的分析プログラムを記録した記録媒体
US6718346B1 (en) 2000-08-17 2004-04-06 3Com Corporation Generating summary data for a requested time period having a requested start time and end time a plurality of data records
JP2002109150A (ja) 2000-09-28 2002-04-12 Fuji Electric Co Ltd 時系列データの適応的予測方法
US20020147599A1 (en) 2001-04-05 2002-10-10 International Business Machines Corporation Method and system for simplifying the use of data mining in domain-specific analytic applications by packaging predefined data mining models
US6636860B2 (en) 2001-04-26 2003-10-21 International Business Machines Corporation Method and system for data mining automation in domain-specific analytic applications
US20040010505A1 (en) 2001-04-26 2004-01-15 Vishnubhotla Prasad R. Method and system for data mining automation in domain-specific analytic applications
US20020198889A1 (en) 2001-04-26 2002-12-26 International Business Machines Corporation Method and system for data mining automation in domain-specific analytic applications
US6973459B1 (en) 2002-05-10 2005-12-06 Oracle International Corporation Adaptive Bayes Network data mining modeling
US20050102303A1 (en) 2003-11-12 2005-05-12 International Business Machines Corporation Computer-implemented method, system and program product for mapping a user data schema to a mining model schema
US20070136346A1 (en) 2004-02-03 2007-06-14 Morris John M Executing a join plan using data compression
US7225200B2 (en) 2004-04-14 2007-05-29 Microsoft Corporation Automatic data perspective generation for a target variable
JP2005302040A (ja) 2004-04-14 2005-10-27 Microsoft Corp 目標変数の自動データパースペクティブ生成
US20060218132A1 (en) 2005-03-25 2006-09-28 Oracle International Corporation Predictive data mining SQL functions (operators)
US20070011134A1 (en) 2005-07-05 2007-01-11 Justin Langseth System and method of making unstructured data available to structured data analysis tools
US7546312B1 (en) 2005-09-23 2009-06-09 Emc Corporation System and methods for modeling a report query database
US20070185868A1 (en) 2006-02-08 2007-08-09 Roth Mary A Method and apparatus for semantic search of schema repositories
US20070203893A1 (en) 2006-02-27 2007-08-30 Business Objects, S.A. Apparatus and method for federated querying of unstructured data
US7991583B2 (en) 2006-04-13 2011-08-02 Dspace Digital Signal Processing And Control Engineering Gmbh Diagnosis in automotive applications
JP2008102736A (ja) 2006-10-19 2008-05-01 Toshiba Corp 検索装置および検索方法
US20080263093A1 (en) 2007-04-20 2008-10-23 International Business Machines Corporation Generation of a set of pre-filters from a set of event subscriptions to more efficiently select events of interest
US20090164943A1 (en) 2007-12-19 2009-06-25 Teradata Corporation Open model ingestion for Master Data Management
US20090162824A1 (en) 2007-12-21 2009-06-25 Heck Larry P Automated learning from a question and answering network of humans
US20100082601A1 (en) 2008-09-29 2010-04-01 Bhashyam Ramesh Method, database system and computer program for joining temporal database tables
US20100106747A1 (en) 2008-10-23 2010-04-29 Benjamin Honzal Dynamically building and populating data marts with data stored in repositories
CN102714767A (zh) 2010-01-18 2012-10-03 日本电气株式会社 信息终端装置、信息终端装置的操作方法及其程序
US20120290981A1 (en) 2010-01-18 2012-11-15 Nec Corporation Information terminal apparatus, operation method by information terminal apparatus and program thereof
US20120330931A1 (en) 2010-04-09 2012-12-27 Sadaki Nakano Database management method, computer, sensor network system and database search program
US8620934B2 (en) 2010-05-05 2013-12-31 University Of Washington Through Its Center For Commercialization Systems and methods for selecting data elements, such as population members, from a data source
JP2011242851A (ja) 2010-05-14 2011-12-01 Nippon Telegr & Teleph Corp <Ntt> 和集合集約処理と等結合処理の組み合わせ方法及びデータベースシステム及びプログラム
US20110302187A1 (en) 2010-06-04 2011-12-08 Fujitsu Limited Schema definition generating device and schema definition generating method
JP2011257812A (ja) 2010-06-04 2011-12-22 Fujitsu Ltd スキーマ定義生成装置、スキーマ定義生成方法およびスキーマ定義生成プログラム
JP2013542478A (ja) 2010-08-25 2013-11-21 インターナショナル・ビジネス・マシーンズ・コーポレーション 地理空間データベース統合方法、およびデバイス
US20120054174A1 (en) 2010-08-25 2012-03-01 International Business Machines Corporation Geospatial database integration using business models
WO2012025493A1 (en) 2010-08-25 2012-03-01 International Business Machines Corporation Geospatial database integration
JP2012059173A (ja) 2010-09-13 2012-03-22 Fujitsu Marketing Ltd レポート作成装置およびそのプログラム
US20120173226A1 (en) 2010-12-30 2012-07-05 International Business Machines Corporation Table merging with row data reduction
WO2012128207A1 (ja) 2011-03-18 2012-09-27 日本電気株式会社 多変量データの混合モデル推定装置、混合モデル推定方法および混合モデル推定プログラム
US8731881B2 (en) 2011-03-18 2014-05-20 Nec Corporation Multivariate data mixture model estimation device, mixture model estimation method, and mixture model estimation program
JP2013152656A (ja) 2012-01-26 2013-08-08 Fujitsu Ltd 説明変数の決定のための情報処理装置、情報処理方法及びプログラム
JP2013164724A (ja) 2012-02-10 2013-08-22 Nippon Telegr & Teleph Corp <Ntt> 情報取引システム
JP2013182498A (ja) 2012-03-02 2013-09-12 Hitachi Solutions East Japan Ltd 需要予測装置、需要予測プログラムおよび需要予測方法
US20130246996A1 (en) 2012-03-19 2013-09-19 Enterpriseweb Llc Declarative Software Application Meta-Model and System for Self-Modification
US20140188918A1 (en) 2012-04-03 2014-07-03 Sas Institute Inc. Techniques to perform in-database computational programming
US20140136471A1 (en) 2012-11-13 2014-05-15 International Business Machines Corporation Rapid Provisioning of Information for Business Analytics
US20140201194A1 (en) 2013-01-17 2014-07-17 Vidyasagar REDDY Systems and methods for searching data structures of a database
US20140223284A1 (en) 2013-02-01 2014-08-07 Brokersavant, Inc. Machine learning data annotation apparatuses, methods and systems
US9146984B1 (en) 2013-03-15 2015-09-29 Google Inc. Enhancing queries for data tables with nested fields
US20140279074A1 (en) 2013-03-15 2014-09-18 Turn Inc. Data management platform for digital advertising
WO2014208205A1 (ja) 2013-06-26 2014-12-31 前田建設工業株式会社 表形式データ処理プログラム、方法及び装置
US10452632B1 (en) 2013-06-29 2019-10-22 Teradata Us, Inc. Multi-input SQL-MR
US20160173122A1 (en) 2013-08-21 2016-06-16 Hitachi, Ltd. System That Reconfigures Usage of a Storage Device and Method Thereof
US20160232213A1 (en) 2013-09-27 2016-08-11 Nec Corporation Information Processing System, Information Processing Method, and Recording Medium with Program Stored Thereon
WO2015045318A1 (ja) 2013-09-27 2015-04-02 日本電気株式会社 情報処理システム、情報処理方法およびプログラムを記憶する記録媒体
JP2015075970A (ja) 2013-10-09 2015-04-20 前田建設工業株式会社 表形式データ処理プログラム、方法、及び装置
WO2015085916A1 (zh) 2013-12-10 2015-06-18 中国银联股份有限公司 数据挖掘方法
US20150310082A1 (en) 2014-04-24 2015-10-29 Luke Qing Han Hadoop olap engine
US20150309990A1 (en) 2014-04-25 2015-10-29 International Business Machines Corporation Producing Insight Information from Tables Using Natural Language Processing
US20170109629A1 (en) 2014-06-03 2017-04-20 Nec Corporation Feature enumeration system, feature enumeration method and feature enumeration program
WO2015186278A1 (ja) 2014-06-03 2015-12-10 日本電気株式会社 属性列挙システム、属性列挙方法および属性列挙プログラム
US20150356123A1 (en) 2014-06-04 2015-12-10 Waterline Data Science, Inc. Systems and methods for management of data platforms
US20150379430A1 (en) 2014-06-30 2015-12-31 Amazon Technologies, Inc. Efficient duplicate detection for machine learning data sets
US20150379428A1 (en) 2014-06-30 2015-12-31 Amazon Technologies, Inc. Concurrent binning of machine learning data
US20170213158A1 (en) 2014-07-31 2017-07-27 Nec Corporation Behavioral characteristic prediction system, behavioral characteristic prediction device, method and program
WO2016017086A1 (ja) 2014-07-31 2016-02-04 日本電気株式会社 行動特性予測システム、行動特性予測器、方法およびプログラム
US20160042039A1 (en) 2014-08-06 2016-02-11 Martin Kaufmann Timeline index for partitioned temporal database tables
US11182691B1 (en) 2014-08-14 2021-11-23 Amazon Technologies, Inc. Category-based sampling of machine learning data
US9130832B1 (en) 2014-10-09 2015-09-08 Splunk, Inc. Creating entity definition from a file
US20160103897A1 (en) 2014-10-10 2016-04-14 Fmr, Llc Database, Data Structure and Framework Transformer Apparatuses, Methods and Systems
CN104408149A (zh) 2014-12-04 2015-03-11 威海北洋电气集团股份有限公司 基于社交网络分析的犯罪嫌疑人挖掘关联方法及系统
CN104881424A (zh) 2015-03-13 2015-09-02 国家电网公司 一种基于正则表达式的电力大数据采集、存储及分析方法
US9934266B2 (en) 2015-05-14 2018-04-03 Walleye Software, LLC Memory-efficient computer system for dynamic updating of join processing
US20160342606A1 (en) 2015-05-21 2016-11-24 Oracle International Corporation Declarative mapreduce using regular expressions
US20180373764A1 (en) 2015-11-25 2018-12-27 Nec Corporation Information processing system, descriptor creation method, and descriptor creation program
WO2017090475A1 (ja) 2015-11-25 2017-06-01 日本電気株式会社 情報処理システム、関数作成方法および関数作成プログラム
US10713589B1 (en) 2016-03-03 2020-07-14 Amazon Technologies, Inc. Consistent sort-based record-level shuffling of machine learning data
US20180095952A1 (en) 2016-09-15 2018-04-05 Gb Gas Holdings Limited System for data management in a large scale data repository
US20200057948A1 (en) 2016-10-31 2020-02-20 Nec Corporation Automatic prediction system, automatic prediction method and automatic prediction program
US20180121442A1 (en) 2016-11-02 2018-05-03 Sap Se Hierarchical map visualizations of geo-enriched data
US20200387664A1 (en) * 2017-03-30 2020-12-10 Dotdata, Inc. Information processing system, feature description method and feature description program
US20200387505A1 (en) 2017-03-30 2020-12-10 Dotdata, Inc. Information processing system, feature description method and feature description program
US20190043127A1 (en) * 2017-08-04 2019-02-07 Airbnb, Inc. Verification model using neural networks
US20200334246A1 (en) * 2017-10-05 2020-10-22 Dotdata, Inc. Information processing device, combination condition generation method, and combination condition generation program
US20210342341A1 (en) 2017-12-22 2021-11-04 Dotdata, Inc. Data analysis assistance device, data analysis assistance method, and data analysis assistance program
US20210357372A1 (en) 2017-12-22 2021-11-18 Dotdata, Inc. Data analysis assistance device, data analysis assistance method, and data analysis assistance program

Non-Patent Citations (8)

* Cited by examiner, † Cited by third party
Title
Generate Near Table, esri, Nov. 20, 2016, retrieved Oct. 11, 2022 from https://web.archive.org/web/20161113175527/http://desktop.arcgis.com/en/aremap/10.3/tools/analysis-toolbox/generate-near-table.htm (3 pages).
Nakamura, Shirou et al., "Practical Methods for Constructing Data Warehouse," Nikkei Computer, (non-official translation) Sep. 15, 1997 (Sep. 15, 1997), pp. 237-249.
Samorani et al., "A Randonmized Exhaustive Propositionalization Approach For Molecule Classification", Leeds School of Business, University of Colorado at Boulder, UCB 419, Boulder, Colorado 80309-0419, 27 pages.
Samorani et al., "Data Mining for Enhanced Operations Management Decision Making: Applications in Health Care", Department of Operations and Information Management, 2012, 159 pages.
Spatial Join, esri, Mar. 18, 2017, retrieved Oct. 11, 2022 from https://web.archive.org/web/20170318121018/http://desktop.arcgis.com/ja/arcmap/10.3/tools/analysis-toolbox/spatial-join.htm (4 pages).
StatSlice Systems Excel at Data Mining—Connecting to a SQL Data Source, Youtube, Jul. 18, 2014, URL: https://www.youtube.com/watch?v=7RYbDWa9or8, retrieved on Jun. 11, 2019, 1 page.
Swati Thacker et al., "Oracle Fusion Middleware", Oracle Reports User's Guide to Building Reports 11g Release 1 (11.1), Jul. 31, 2013 (Jul. 31, 2013), XP055659020, Retrieved from the Internet: URL:https:// docs.oracle.com/middleware/11119/classic/build-reports/B32122-03.pdf [retrieved on Jan. 17, 2020], 776 pages.
Zaman et al., "Modeling and querying multidimensional data sources in Siebel Analytics", SIGMOD 2005: Proceedings of the ACM SIGMOD International Conference On Management of Data; Baltimore, Maryland, Jun. 14-16, 2005, Association for Computing Machinery, New York, NY, Jun. 14, 2005 (Jun. 14, 2005), pp. 822-827 (6 pages).

Also Published As

Publication number Publication date
EP3696686A1 (en) 2020-08-19
JPWO2019069507A1 (ja) 2020-11-05
US20200301921A1 (en) 2020-09-24
EP3696686A4 (en) 2021-07-07
WO2019069507A1 (ja) 2019-04-11
SG11202003814TA (en) 2020-05-28

Similar Documents

Publication Publication Date Title
US20200334246A1 (en) Information processing device, combination condition generation method, and combination condition generation program
US12099907B2 (en) Automated geospatial data analysis
US20170337229A1 (en) Spatial indexing for distributed storage using local indexes
US11514062B2 (en) Feature value generation device, feature value generation method, and feature value generation program
Kosowska-Stamirowska et al. Evolving structure of the maritime trade network: evidence from the Lloyd’s Shipping Index (1890–2000)
US20180340787A1 (en) Vehicle routing guidance to an authoritative location for a point of interest
CN114723528A (zh) 基于知识图谱的商品个性化推荐方法及系统
CN113254630B (zh) 一种面向全球综合观测成果的领域知识图谱推荐方法
JP7476290B2 (ja) 道路情報更新方法、装置、電子機器、記録媒体及びコンピュータプログラム
CN103712628B (zh) 导航路径描绘方法和终端
WO2016113648A1 (en) Database statistical histogram forecasting
CN114595302A (zh) 空间要素的多层级空间关系构建方法、装置、介质及设备
CN114398315A (zh) 一种数据存储方法、系统、存储介质及电子设备
Bhattacharya et al. Smart cities intelligence system (smacisys) integrating sensor web with spatial data infrastructures (sensdi)
JP5721567B2 (ja) 画像分類装置、画像分類装置の制御方法及びプログラム
CN112883195A (zh) 个体出行的交通知识图谱构建方法及系统
CN112685618A (zh) 用户特征识别方法、装置、计算设备及计算机存储介质
CN115577147A (zh) 可视化情报图谱检索方法、装置、电子设备及存储介质
WO2019069506A1 (ja) 特徴量生成装置、特徴量生成方法および特徴量生成プログラム
CN113312364A (zh) 基于区块链的智慧云业务更新方法及区块链业务系统
CN107329730B (zh) 语音提示信息生成方法及装置
Gibas et al. Bayesian Modeling of Travel Times on the Example of Food Delivery: Part 1—Spatial Data Analysis and Processing
CN111460325A (zh) Poi搜索方法、装置与设备
Yann et al. Process for the Encapsulation and Visualization of Dominant Demand and Supply Corridors
JP2019053469A (ja) データベース生成装置、データベース生成方法、およびプログラム

Legal Events

Date Code Title Description
FEPP Fee payment procedure

Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY

FEPP Fee payment procedure

Free format text: ENTITY STATUS SET TO SMALL (ORIGINAL EVENT CODE: SMAL); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY

STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION

FEPP Fee payment procedure

Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY

AS Assignment

Owner name: DOTDATA, INC., CALIFORNIA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:CHEN, TING;KUSUMURA, YUKITAKA;FUJIMAKI, RYOHEI;AND OTHERS;SIGNING DATES FROM 20210429 TO 20210510;REEL/FRAME:056206/0505

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

FEPP Fee payment procedure

Free format text: ENTITY STATUS SET TO SMALL (ORIGINAL EVENT CODE: SMAL); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS

STPP Information on status: patent application and granting procedure in general

Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT RECEIVED

STPP Information on status: patent application and granting procedure in general

Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED

STCF Information on status: patent grant

Free format text: PATENTED CASE