JP2002024252A - Data aggregating method, and recording medium recorded with execution program for the method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a data aggregating method which can aggregate data while a user request is reflected without any limitation to a specific domain and releases a user from data selection and aggregation. SOLUTION: A propagation network is previously structured on the basis of observation data tables which contain observed values, domain layer knowledge, and inter-domain relation knowledge. According to a meaning-framed user request, a candidate data table which hold necessary data is specified among the observation data tables. For a node corresponding to the candidate key property of the candidate data table, a token is propagated to a node whose granularity matches a domain of each drawing element which is not a quantity scale along links of the propagation network. When it is reachable, aggregation possibility is judged and properties corresponding to the nodes that the token passes through are coupled in the candidate data table; and data tuples are taken out according to restriction conditions of the meaning frame and the same tuples are put together to aggregate data.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION The present invention relates to a method and an apparatus for retrieving necessary data in accordance with a user request described in a natural sentence, organizing the data into an appropriate level of detail (granularity), and visualizing the data. Complex hierarchies where there may be splitting methods,
Also, the present invention relates to a data aggregation method capable of capturing data shortage and collecting data even when an attribute in a database (DB) is incomplete in response to a user request.

[0002]

2. Description of the Related Art In recent years, with the improvement in processing capacity of computers (computers) and the spread of networks, a plurality of users have been able to share various and large amounts of data and use them with different intentions. Data visualization is one of the effective means for intuitively understanding such a huge amount of data. To visualize data according to user requirements,
This requires a series of tasks of (1) analyzing the type and range of required data, (2) aggregating the data with appropriate detail, and (3) drawing in an easy-to-understand format. Since the load increases according to the amount of data, a method of automatically visualizing according to the intention of each user is required.

To solve this problem, Japanese Patent Application No.
No. 154577, the method and apparatus, and the document "Hayato Yonezawa,
Mitsunori Matsushita, Toshiro Makino, Tsuneaki Kato, "Numerical Data Visualization Method Reflecting User's Request", 1999 All Japan Society for Artificial Intelligence Conference (13th), etc. have been proposed.

In the method and apparatus of Japanese Patent Application No. 11-154577 and the method described in the above-mentioned document, (1) a user request described in a natural language is morphologically analyzed and converted into a semantic frame format which can be understood by a computer. 2) Contents of the semantic frame and hierarchical knowledge (upper / lower relation knowledge with granularity tags)
The data type and range are determined based on the data, and the data is reconfigured and aggregated with an appropriate level of detail (granularity). (3) The type of graph matching the user request based on the characteristics of the reconfigured data ( Graph type), and (4) a highlighting operation method for clarifying a point of interest in the graph is selected, and drawing parameters such as a scale and a color scheme of the axis of the determined graph type are determined based on the selected method. 5) Visualization is realized by drawing a graph using the determined graph type and drawing parameters.

[0005]

However, in these methods, apparatuses, and systems, the hierarchical knowledge used for data aggregation is limited to a tree structure, and therefore corresponds to a complex hierarchy that actually exists. There is a problem that you can not. For example, when Japan is divided into detailed divisions, there are a method of dividing into regions such as the Kanto region and the Kinki region, and a method of dividing into Japan coastal region, Pacific coastal region, and inland region. For example, Kochi prefecture is classified into the Shikoku region by the former division method, and is classified into the Pacific coast by the latter division method. Since these two classification / division methods do not have a hierarchical relationship between the upper and lower layers, Japanese Patent Application No.
It cannot be expressed by the method and apparatus of No. 154577 or the method according to the above-mentioned document.

Further, in the method and apparatus of Japanese Patent Application No. 11-154577 and the method described in the above-mentioned document, the data table to be aggregated existing in the database (DB) matches the attribute of the user request in the meaning frame. Therefore, it is not possible to classify by attributes not specified in the data table. For example, in a data table as shown in Table 1, when it is desired to aggregate the sales volume of each vehicle type for each sales company, since the vehicle type and the sales company are not in the same domain, they cannot be described as hierarchical knowledge. Since the attribute relating to “company” does not exist in the data table, it cannot be aggregated. Here, the “domain” means a set of elements classified into the same category, for example, “Kinki region” and “Kyoto Prefecture” are elements of the “place” domain.

[0007]

[Table 1]

The present invention solves the above-mentioned problems, and provides a data aggregation method that enables aggregation of data tables reflecting user requests without limiting to a specific domain, and releases users from a data selection / aggregation process. The task is to

[0009]

In order to solve the above-mentioned problems, a data aggregation method according to the present invention employs a propagation network created by referring to a hierarchical relationship of the same domain and a correspondence relationship between different domains. A method of aggregating the observation data table holding the obtained data, wherein a candidate data table which is a candidate among the plurality of observation data tables is specified based on the meaning-framed user request; and Determining whether the candidate data table can be aggregated from the token reachability; and, for the candidate data table determined to be aggregated, an attribute corresponding to a node of the propagation network through which a token has passed. Combining and extracting a data tuple based on the constraints described in the semantic frame; Ri out was a step of creating a data aggregation table by collectively the same tuple, characterized in that it has.

Alternatively, the propagation network is constructed by referring to a plurality of observation data tables, domain hierarchy knowledge, and inter-domain relationship knowledge.

Alternatively, in the step of determining whether or not there is a possibility of aggregation of the candidate data table, a node having a candidate key attribute of the candidate data table may be drawn along a link of the propagation network from a node having a candidate key attribute of the semantic frame. When the token is reachable up to a node having the same granularity as the domain, the candidate data table may be aggregated.

[0012] Alternatively, a program for causing a computer to execute the steps in the above data aggregation method is
The program is recorded on a recording medium readable by the computer.

According to the present invention, a data table holding necessary data is selected in accordance with a user request, and the data tables are aggregated with an appropriate granularity. In the present invention, in order to select a necessary data table, two types of knowledge, domain hierarchy knowledge and inter-domain relationship knowledge, are prepared, and a propagation network is constructed by focusing on the functional dependency of these knowledges. To determine the possibility of aggregation from the observation data table and aggregate the data.

[0014]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below.

In the present invention, attributes A _1, ..., attribute A _n A _n-1 is a candidate key attribute is non-key attribute, a second normal form of a data table such that only A _n is an amount measure Target for aggregation. This data table is called “observation data table”
Call. For example, in the data table of Table 1, “sales model” and “date and time” are candidate key attributes, and “sales volume” is a non-key attribute. In this embodiment, d _ij is an element of the attribute A _i , and their relationship is

[0016]

(Equation 1)

[0017] However, each element of elem (A _i ) is described with the same granularity.

In the present invention, two types of knowledge, domain hierarchy knowledge and inter-domain relationship knowledge, are used to aggregate the data tables.

The domain hierarchy knowledge is obtained by layering elements described in various granularities in units of units. FIG. 1 shows an example. The domain element (for example, “Hyogo prefecture”) belongs to a granularity (for example, “Prefecture”) corresponding to the unit. An element belonging to the grain size G _i d
_ij and their relationship

[0020]

(Equation 2)

[0021] For any G _i , G _j (i） j), gr (G _i ) （gr (G _j ) = φ, and between the domain D and the granularity {G ₁ ,..., G _n } in that domain. D
= Gr (G ₁ ) ∪... ∪gr (G _n ).

In the present invention, the partial order between particle sizes is

[0023]

(Equation 3)

The partial order between the elements

[0025]

(Equation 4)

"G _j is coarser than G _i "

[0027]

(Equation 5)

[0028] At this time, G _j is referred to as the upper granularity of G _i . When a certain set is partially ordered, it means that the set can be uniquely ordered locally but cannot be uniquely ordered as a whole. Also, the relationship between elements belonging to each granularity is

[0029]

(Equation 6)

[0030]

[0031]

(Equation 7)

Then, for all d _ik ∈gr (G _i )

[0033]

(Equation 8)

[0034] to become d _jl ∈gr (G _j) is assumed to exist. This means functional dependence. At this time, an element set of lower granularity G _i corresponding to element d _jl of upper granularity G _j is defined as

[0035]

(Equation 9)

## EQU1 ##

[0037]

(Equation 10)

When there is no G _k , and only at that time, an arrow is given from G _i to G _j , thereby obtaining a hierarchical relationship of granularity as shown in FIG. The obtained hierarchical relationship is used for constructing a propagation network described later.

The inter-domain relationship knowledge indicates the relationship between two element sets belonging to different domains. Table 2 is an example of inter-domain relationship knowledge representing the relationship between a company and a product.

[0040]

[Table 2]

In this knowledge, each element set is described with a single granularity. Thus, this knowledge can be viewed as a relationship between two granularities belonging to different domains. This relationship assumes functional dependence. That is, a certain domain D
_For all the elements d _i ∈gr (G _i ) belonging to the particle size G _i in _i , the elements d _j ∈g belonging to the particle size G _j in another domain D _j
When r (G _j ) is in a function-dependent relation, this is treated as inter-domain relation knowledge, and is represented as G _i → G _j .

The proposed method according to the present embodiment will be described on the premise of the above-described observation data table, domain hierarchy knowledge, and inter-domain relationship knowledge.

First, the structure of the propagation network used in the proposed method will be described. FIG. 3 shows an example of the propagation network.

In the proposed method, in order to create a data table that satisfies user requirements, the observation data table is extended using this propagation network. This network is created in advance from the above two types of knowledge and the observation data table. FIG. 3 shows a part of the nodes of each domain. For example, in the time domain,
eneralization for Calenda
r Attributes Using Domain
Generalization Graphs' (R
and all et al. TIME-98, pp. 1
77-184 (1998)).

This network has three types of nodes: candidate key attribute nodes, non-key attribute nodes, and granularity nodes. The candidate key attribute node and the non-key attribute node correspond to the attributes of the observation data table, and the granularity node corresponds to the granularity of the domain hierarchy knowledge. These nodes are connected by three types of links: related links, granularity links, and inter-domain links.

A granularity link (solid line in FIG. 3) is a link representing a granularity relationship of domain hierarchy knowledge, and an inter-domain link (dashed line in FIG. 3) is a link of different domain hierarchy knowledge based on the inter-domain relationship knowledge. This is a link between the granularities. The partial network connected by these two links can be regarded as static knowledge of the system. That is, this partial network does not change depending on the task and the contents of the observation data.

The related link (dotted line in FIG. 3) is a link extending from each candidate key attribute node A _i to a granularity G satisfying elem (A _i ) ⊆gr (G). elem (A _i )
Is described in a single granularity, so it corresponds to only one granularity node, and the relationship can be regarded as A _i → G. If there is no granularity node that satisfies this condition, it is considered that the cause is that knowledge is insufficient or attribute elements are not described in a single granularity.

Since the links of this network are all based on functional dependency, if A → B and B → C hold between nodes A, B and C, A
→ C also holds. That is, C corresponding to an element belonging to A
Can be identified.

Hereinafter, a data table aggregation algorithm using this propagation network will be described.

In the proposed method, a user request described in a natural sentence is assumed, and the user request is converted into a semantic frame format in order to handle the request in a formal manner. An example of a user request targeted by the proposed method is, for example, the following natural sentence.

<Ex1> What is the monthly sales volume in 1997 of each car sales company? This user request is converted into the meaning frame format shown in FIG. The semantic frame has multiple lines depending on user requirements. Each line is a series of information required for drawing, and is called a “drawing element”. Each drawing element is represented by five slots of variables, domains, granularity, scale, and constraints. The meaning frame in the present invention has the above requirements. An example of a method of creating this semantic frame from a natural sentence is disclosed in Japanese Patent Application No. 11-154577, the outline of which is as follows, but the present invention is limited to this semantic frame creating method. What is necessary is just to have the above requirements.

First, a morphological analysis of a user request is performed using an existing technique. That is, the user request is decomposed into words, and its basic type and part of speech are analyzed. A meaning frame is created based on the analysis result.

In the creation of the meaning frame, the items of the meaning frame are determined by referring to the item dictionary from the noun of the user request that has been morphologically analyzed. The item dictionary classifies, for each item, a noun belonging to the item and a symbol representing a unit. "Items" in this dictionary are words that represent categories when nouns and symbols are classified, and "possibility" is a description of whether the categories can be item names.
“Nouns” are items that describe general nouns, symbols, and the like that are actually used during user requests and others in a typical format, and the item dictionary is composed of these three items.

Next, for each item determined, the attribute
The attribute of each item is determined with reference to the particle size dictionary. The attribute / granularity dictionary is used for items in the semantic frame for the items in the item dictionary described above, that is, for an item having a possibility of “○” in the item dictionary, possible attributes of each item and its granularity, This describes the priority of attributes that can be taken. This priority order is for selecting a candidate having high generality when it cannot be limited from a user request.

Next, the granularity of each item is determined with reference to the attribute / granularity dictionary and the upper / lower relation dictionary with a granularity tag.

Next, the constraints of each item are determined with reference to the constraint dictionary. The constraints employ a mathematical description format. The constraint dictionary describes the attributes in the semantic frame and the possible constraints, in a format using variables. Terms such as “minimum value” and “superordinate concept” in the constraint conditions are defined in advance for the system.

As described above, a user request for a natural sentence can be converted into a meaning frame format that can be understood by a computer.

Subsequently, in order to draw a graph suitable for the user's request, it is necessary to create an aggregation table having elements that satisfy each drawing element in the meaning frame. For this purpose, it is determined whether an aggregation table obtained from the observation data table can be created using the propagation network.
This is performed according to the following procedure.

First, a token propagation start node and a goal node are determined. FIG. 5 shows a flowchart of the operation. Attention is paid to the rendering element of the quantity scale in the semantic frame, and an observation data table having a non-key attribute matching the domain of the rendering element is selected. However, since there can be a plurality of such data tables, all of them are treated as a set of candidate data tables. Nodes corresponding to the candidate key attributes in these data tables are set as token propagation start nodes.
In addition, attention is paid to drawing elements that are not quantitative measures in the semantic frame, and a node whose grain size matches the domain is determined as a goal node.

Next, the token is propagated from the propagation start node through the network along the link. FIG. 6 shows a flowchart of the operation. If the token can reach the goal node from the propagation start node, it can be known that the attribute of the propagation start node and the attribute of the goal node are in a functional dependency, and the attribute in the data table corresponding to the propagation start node is changed. , It is possible to convert to the granularity indicated by the goal node. Therefore, among the candidate data tables, one having a propagation start node that can reach all goal nodes is adopted as a basic data table for constructing an aggregation table.

An aggregation table is created from the basic data table obtained by the above processing. The flowchart is shown in FIG.

First, a link (relation) on the shortest path from each candidate key attribute node to a goal node is connected to the basic data table by applying a natural connection operation. For example, if the token is a candidate key attribute node A _i
, And reaches the goal node G _j via G _k , the following calculation is performed: R [A _i = G _k ] S [G _k = G _k ] T (2) Here, R is an observation data table, S means a relation S (A _i , G _k ), and T means a relation T (G _k , G _j ). Strictly speaking, a natural join operation is applicable only when two relations have the same attribute name, otherwise a more general operation, the equal join operation, must be applied. However, it is clear from the definition that the attribute and the granularity connected by the related link indicate the same entity, and thus the natural join operation is applied instead of the equal join operation.

Next, a tuple that satisfies all the constraints of the drawing element is selected from the data table obtained by the above processing. For example, when drawing based on the semantic frame of FIG. 4, a tuple that satisfies “x _i ∈automotive industry” and “y _j ∈1997” is selected. In addition, tuple (tuple)
Means the element of the direct product. That is, D ₁ = {a, b},
Assuming that D ₂ = {c, d}, the direct product of D ₁ and D ₂ is (a,
c), (b, c), (a, d), and (b, d) are a set of four elements (= tuples).

Since the original data table does not guarantee the completeness of all elements of the domain, for example, if the data table has only the information of the Kyoto prefecture regarding the location domain, the aggregation into the Kinki region will not be performed by the information of other prefectures. Can not be done because of lack. However, if the user only wants to know about Kyoto, it can be aggregated. Therefore, the completeness of the constraint condition requested by the user is checked, and it is checked whether the direct product of the constraint condition is included in the data table in order to extract all tuples satisfying the condition. Here, the upper-level concept designating type constraint condition is expanded to the element having the finest granularity in the data table with reference to the domain hierarchy knowledge. For example, in FIG. 4, "x _i ∈ automotive industry" is "x _i = {Toyota, Nissan, ..., Honda}" expands to.

After confirming the existence of a necessary tuple in the data table by this check, an extra attribute is deleted. If the tuple is incomplete, the system informs the user that there is missing data. The extra attribute is a node on the path from the candidate key attribute node to the goal node, and thus can be easily specified. It should be noted that these attributes remain up to the point of this processing because there is a possibility that they will be required for a completeness check.

After the above-described processing, tuples having the same element in all the candidate key attributes are aggregated into one, and a two-dimensional table is created in which the attribute element having a plurality of elements is the front and the front. However, when there are three or more attributes having a plurality of elements, the user is prompted to input further constraint conditions. The result of this processing is an aggregation table most suitable for the user's request.

A specific example will be described below using the propagation network of FIG. 3 and the meaning frame expression of the user request <ex1> shown in FIG. In FIG. 4, since “Z” is a drawing element of the quantity scale and its domain is the sales volume, the “monthly automobile sales volume” data table is selected as the candidate data table.

The goal node is a rendering element in which the rendering elements X and Y are not the quantity scales, and is determined based on the domain and the value of the granularity of those rendering elements. That is, a company granularity node in the industrial domain and a year-month granularity node in the date domain.

The token starts to propagate from the sales key node and the date / time node, which are the candidate key attribute nodes in the data table. The token propagated from the sales model node reaches the company granularity node in the industrial domain via the product granularity node in the product domain. Also, the token propagated from the date / time node directly reaches the year / month granularity node in the date domain. Therefore, the "monthly vehicle sales volume" data table is adopted as the basic data table.

Next, the following natural join operation is applied to this data table in order to add each attribute of product, company, and date.

R [vehicle model = product] S [product = product] T [date = year / month] U (3) where R, S, T, and U are “monthly automobile sales volume” data table, S ( T (product, company) relationship in knowledge of inter-domain relationship, U (date and time, date)
Relationship. As a result, a data table shown in Table 3 is obtained.

[0072]

[Table 3]

Next, a tuple that satisfies all the constraints of the drawing element is selected from the data table obtained by the above processing. Constraints in the meaning frame of FIG. 4, "x _i ∈
The tuples satisfying these constraints are selected because “automobile industry” and “y _j ∈1997” are satisfied.

[0074]

[Equation 11]

After confirming the existence of a necessary tuple in the data table by this check, an extra attribute is deleted. Here, since the extra attribute is the “product” granularity node, the product attribute in the data table is deleted.

After the above processing, tuples having the same element in all the candidate key attributes are aggregated into one, and a two-dimensional table is created in which the attribute element having a plurality of elements is the front and the front. As a result, the data table of Table 4 is obtained as an aggregate table.

[0077]

[Table 4]

It is needless to say that the processing steps shown in the flowcharts of FIGS. 5, 6, and 7 can be executed by a computer, and the computer reads a program for causing the computer to execute the processing steps. It can be recorded on a possible storage medium, for example, FD (floppy (registered trademark) disk), MO, ROM, memory card, CD, DVD, removable disk, etc., stored, provided, and distributed. .

[0079]

According to the present invention described above,
Since data tables reflecting user requests can be aggregated without being limited to a specific domain, the user is released from a wasteful and time-consuming data selection / aggregation process.

[Brief description of the drawings]

FIG. 1 is a diagram illustrating an example of domain hierarchy knowledge according to an embodiment of the present invention.

FIG. 2 is a diagram illustrating an example of a granularity hierarchy according to an embodiment of the present invention.

FIG. 3 is a diagram illustrating an example of a propagation network according to an embodiment of the present invention.

FIG. 4 illustrates a user request <ex in an exemplary embodiment of the present invention.
FIG. 3 is a diagram showing an example of 1>.

FIG. 5 is a flowchart showing a procedure for determining a propagation start node and a goal node in one embodiment of the present invention.

FIG. 6 is a flowchart showing a procedure for expanding an observation data table in an embodiment of the present invention.

FIG. 7 is a flowchart showing a procedure for creating an aggregation table according to an embodiment of the present invention.

[Explanation of symbols]

G ₁ ~G ₄ ... in particle size d ₁₁ ... elements d ₂₁ to d ₂₃ ... elements d ₄₁ to d ₄₃ ... granularity G ₄ belonging to the element d _31, d ₃₂ ... granularity G ₃ belonging to the granularity G ₂ belonging to the granularity G ₁ Element to which it belongs

Claims (4)

[Claims] 1. A method for aggregating observation data tables holding observed data using a propagation network created with reference to a hierarchical relationship of the same domain and a correspondence relationship of different domains, comprising: Identifying a candidate data table that is a candidate among the plurality of observation data tables based on the user request, and determining whether the candidate data table can be aggregated from the token reachability of the propagation network; Combining an attribute corresponding to a node of the propagation network through which a token has passed with the candidate data table determined to be likely to be aggregated, and extracting a data tuple based on a constraint described in the semantic frame; Create a data aggregation table by grouping the same tuples Data aggregation method characterized by the steps, a. 2. The data aggregation method according to claim 1, wherein the propagation network is constructed with reference to a plurality of observation data tables, domain hierarchy knowledge, and inter-domain relationship knowledge. 3. The step of determining whether or not the candidate data table can be aggregated includes, from a node having a candidate key attribute of the candidate data table along a link of the propagation network, a rendering element having a non-quantity scale in a semantic frame. 3. The data aggregation method according to claim 1, wherein the candidate data table is considered to be aggregated when the token is reachable up to a node having the same granularity as the domain. 4. A program for causing a computer to execute the steps of the data aggregation method according to claim 1 on a recording medium readable by the computer. Recording medium on which an execution program of a data aggregation method to be executed is recorded.