JP2009265905A - Preprocessor using preliminary rule, preprocessing method, information extraction device using the preprocessor and information extraction method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To easily and appropriately determine a threshold for converting numerical data into category data. <P>SOLUTION: A preprocessor includes an event group database DB1; a threshold/constraint database DB2; a preliminary knowledge rule database DB3; an optimization parameter database DB4, a threshold optimization means 5; and a threshold parameter database DB6. The threshold optimization means 5 determines the value of threshold variables for categorizing the numerical data in order to satisfy a preliminary rule which a user has in advance. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an information extraction device that categorizes a vast amount of information and extracts useful correlation rules, and more particularly to a preprocessing device that calculates a threshold for categorizing information.

Technical background

  In recent years, with the development of sensors and storage devices, it has become possible to store various event data. Here, event data is data that is collected when an event occurs. For example, numerical data indicating customer positions obtained by observing operations in the store of all customers visiting the store, or customer data Purchase data obtained from the purchase log. However, since there is a large amount of event data collected and accumulated in this way, conventionally, useful data can be obtained by extracting and presenting highly correlated combinations as rules for these large amounts of event data. Is provided. Here, the correlation rule refers to a combination of items that appear simultaneously with respect to all event data, and a rule with a high correlation refers to a combination of items that appear simultaneously with a certain probability or more with respect to all event data.

  These highly correlated rules represent only the co-occurrence relationship between items appearing in the data, and the causal relationship between items does not necessarily exist, but there may be a rule having a causal relationship. Therefore, by selecting a rule that is likely to cause a causal relationship from among the plurality of extracted correlation rules, and checking that the causal relationship exists by another method, it is possible to help the user's decision making. For example, the operation data and purchase data in a store of one customer are taken as events, the operation in the store and the purchased product are considered as items, `` Desert section stay '', `` Desert section stay '', `` Bake section stay '', Four items of “Buy roll cake” are included in 10% of all events, and 90% of events including “Stay in dessert shop”, “Stay in sweet shop”, “Stay in bread shop” are “Roll cake” As a result of including `` purchase '' at the same time, the hypothesis is that roll cake buyers are looking for roll cakes more than necessary because the products in the dessert department, confectionery department, and bread department are not well distinguished. It is conceivable that the user can review the products placed on the sales floor.

  In this way, in order to extract useful association rules from a large amount of data, after performing preprocessing (categorization) such as categorizing numerical data obtained by observation into several categories, Generate a candidate for a certain correlation rule, search whether all the events include a candidate correlation rule, count the number of events that include it, and check whether it exists at a rate specified by the user or not. There is a need. (For example, see Patent Document 1).

  Here, for example, when the position coordinates indicating the position of the customer are included as numerical data, the numerical data is categorized as follows.

  That is, if the customer's position x and y satisfy (Expression 1) and (Expression 2), “dessert section”, if (1 expression) and (Expression 4) are satisfied, “Bread counter”, (Expression 3) ) And (4 formulas) are defined as “confectionery section”, and (2 formulas) and (3 formulas) are defined as “drinking water section”.

0 ≦ x <10 (1 set)
0 ≦ y <10 (2 formulas)
10 ≦ x <20 (3 formulas)
10 ≦ y <20 (4 formulas)
In this case, the position coordinates (1, 1) are converted to “dessert section”, and the position coordinates (1, 15) are converted to “bread section”.

  Conventionally, in order to convert numerical data into category data in this way, the user himself / herself sets a threshold value using a prior rule or the like. However, there are many cases where it is not possible to define a threshold value for clearly dividing into categories even if a user has prior rules. For example, even when determining threshold values for dividing the height into three categories of “high”, “normal”, and “low”, which is better “higher than 175 cm” or “higher than 174 cm”? It is difficult for the user to judge. For this reason, conventionally, it has been necessary for the user to obtain a desired result by repeating the operation of changing some of the respective threshold values and confirming the output result.

In addition, when categorization is performed with an incorrect threshold, the quality of the extracted association rule may be affected. For example, by setting a wide fish market area and a narrow meat market area, “fish market” and “pork purchase” may be extracted as correlation rules. An incorrect rule extracted at such a time may confuse the user.
JP-A-11-250084

  As described above, it is difficult for the user to have a clear criterion for the value of the threshold variable set when categorizing numerical data. Therefore, it is difficult for the user to obtain a useful correlation rule efficiently because it is necessary to repeat the operation of changing some of the plurality of threshold variables and confirming the correlation rule generated thereby.

  Furthermore, when categorization is performed with the wrong threshold variable value, the quality of the extracted association rule may be affected. For example, by setting a wide fish market area and a narrow meat market area, “fish market” and “pork purchase” may be extracted as correlation rules. An incorrect rule extracted at such a time may confuse the user.

  The pre-processing device of the information extraction device of the present invention has been made in view of the above-described problem. Trajectory data having the same ID and categorized category data each having a first label are provided. The recorded event set database, the threshold variable conditions and constraints necessary for categorizing the trajectory data recorded in the event set database, and the trajectory data categorized based on these conditions and constraints A threshold / constraint database in which a second label to be attached is recorded; a prior knowledge rule database in which a combination of the first and second labels co-occurring with a predetermined probability is recorded as a prior rule together with the probability; All the prior rules previously included in the prior knowledge rule database are converted into the event set under the constraints. A threshold value optimization means for calculating the value of the threshold variable so as to be extracted from the database; a threshold value database in which the value of the threshold variable calculated by the threshold value optimization means is recorded; and the threshold value database recorded in the threshold value database. And a display device that displays the value of the threshold variable.

  Further, the preprocessing method of the information extraction apparatus of the present invention records trajectory data having the same ID and categorized category data each labeled with a first label in an event set database. Threshold variable conditions and constraints necessary for categorizing the trajectory data recorded in the database, and a second label attached to the trajectory data categorized based on these conditions and constraints are set as threshold values. A combination of the first and second labels that are recorded in a constraint database and co-occur with a predetermined probability is recorded in the prior knowledge rule database as a prior rule together with the probability, and the prior rule is recorded from the prior knowledge rule database. The trajectory data which the ID including the first label of the extracted prior rule is extracted and has Are extracted from the event set database, the conditions and constraints are extracted from the threshold / constraint database, and the threshold variable is set so that the prior rule appears with the highest probability under the extracted constraints. It is a preprocessing method of the information extraction apparatus characterized by calculating the value of.

  That is, the present invention uses a prior rule that a user has in advance to determine in advance a specific item set, an item that has a high probability of being included in an event that includes the set, and a probability (co-occurrence probability) that these are at least included. It is recorded in the database as a rule, and the value of the threshold variable necessary for categorizing numerical data is automatically determined so that this prior rule always appears.

  According to the present invention as described above, even when the user cannot directly determine the value of the threshold variable, the value of the threshold variable necessary for categorizing numerical data can be determined easily and appropriately.

Embodiments of the present invention will be described below with reference to FIGS.

(First embodiment)
FIG. 1 is a block diagram schematically showing a configuration of a preprocessing apparatus using a pre-rule in an embodiment of the present invention.

  As shown in FIG. 1, the preprocessing device using the pre-rule according to this embodiment includes an event set database DB1, a threshold / constraint database DB2, a prior knowledge rule database DB3, an optimization parameter database DB4, and a threshold value. The optimization unit 5, the threshold parameter database DB 6, and the threshold display device 7 are configured.

  Next, each element which comprises the pre-processing apparatus using such a prior rule is demonstrated with reference to FIGS.

  First, data recorded in the event set database DB1 will be described with reference to FIG.

  The event set database DB1 stores the ID of the observed object and data related to the observed object. Among these, the data regarding the observed object is classified into numerical data and category data. The numerical data is, for example, a measured value of the motion of the observed person obtained by observing the body, head, foot, hand position, velocity, acceleration, body orientation, etc. of the observed person with a sensor, or age. , A part of the attribute value of the observed person obtained by using a questionnaire such as income, a purchase amount obtained from a POS terminal or the like, the number of points purchased, and the like. The category data is, for example, a part or request of the attribute value of the observed object obtained from questionnaire data such as sex or occupation, or a purchased product obtained from a POS terminal or the like. Here, as an example of data relating to the observed object, a case will be described in which the numerical data is the movement trajectory data of the observed person's body and the category data is the purchase data of the observed person.

  FIG. 2 shows data recorded in the event set database DB1, FIG. 2A shows numerical data, and FIG. 2B shows category data.

  As shown in FIG. 2A, the table in which the movement trajectory data of the observation object is recorded includes a plurality of records, and one record includes an observation object ID, a data name, a data type, an observation time, and an observation object. The x-coordinate, y-coordinate, and z-coordinate indicating the position of the body are recorded. In FIG. 2A, the observed object 001 is at the position of the x-axis 30 cm, the y-axis 20 cm, and the z-axis 170 cm at 15:32:00 on August 1, 2007, and at 15:32:01 This indicates that the robot was at a position of 15 cm, a y-axis of 20 cm, and a z-axis of 170 cm, and at 15:32:02 it was at a position of x-axis of 15 cm, y-axis of 10 cm, and z-axis of 170 cm. Further, the type of these data is a numerical value, and the data name indicates the body position. For example, there may be a plurality of tables for each person to be observed, or these may be collectively recorded in one table. Further, the number of records recorded in the table may be singular or plural as shown in the present embodiment.

  In addition, as shown in FIG. 2B, the table in which the purchase data of the observed object is recorded includes a plurality of records, and one record includes the observed object ID, data name, data type, observation time, purchased product. Is recorded. In FIG. 2B, the observed object 001 purchased Item 10 and Item 21 at 15:30 on August 1, 2007, and the observed object 002 received Item 35 at 15:10:00 on August 1, 2007. The purchased object 003 indicates that Item 42 was purchased on August 1, 2007 at 15:00:00. The type of these data is category data, and the data name indicates purchase. For example, there may be a plurality of tables for each person to be observed, or these may be collectively recorded in one table. Further, the number of records recorded in the table may be singular or plural as shown in the present embodiment.

  Next, data recorded in the threshold / constraint database DB2 will be described with reference to FIG. Here, a case will be described in which categorization is performed in which the position coordinates of the observed object, which is numerical data recorded in the event set database DB1, are divided into product areas.

  FIG. 3 shows data recorded in the threshold / constraint database DB2, FIG. 3A shows numerical data names and threshold variables to be calculated, and FIG. 3B shows categorized labels and numerical data to be categorized. FIG. 3C shows threshold variable constraints.

  As shown in FIG. 3A, the table in which the numerical data name and the threshold variable to be calculated are composed of a plurality of records, and the data name and the threshold variable name are stored in one record. These threshold variable names are specified by the user. In FIG. 3A, the data name is the position of the body, and the threshold variables for classifying the numerical data indicating the position of the body into the category that is the product area are X1, X2, Y1, Y2, and T1. ing. Note that, for example, a plurality of tables may be provided for each product area to be classified, or as shown in the present embodiment, these tables may be collectively recorded in one table. Further, the number of records recorded in the table may be singular or plural as shown in the present embodiment.

  Also, as shown in FIG. 3B, a table in which labels given after categorizing numerical data and conditions for categorizing the numerical data are composed of a plurality of records, and one record includes a data name, a numerical value The label name given after categorizing the data and the conditions for giving the label are recorded. Here, the conditions for categorizing the label name and numerical data are specified by the user. FIG. 3B shows a condition for classifying the observed object into a category of “A product area stay”, for example. In order to be given the label “A product area stay” and be classified into the category “A product area stay”, the position coordinates x and y of the numerical data must satisfy (Equation 5) and (Equation 6), respectively. .

X1 <= x <X2 (5 formulas)
Y1 <= y <Y2 (6 formulas)
Furthermore, t indicating the time included in the category “A product area stay” classified under the conditions of (Expression 5) and (Expression 6) must satisfy (Expression 7).

T1 <t (7 formulas)
That is, in order for the object to be observed to be classified into the category “A product area stay” and be given the label “A product area stay”, (Equation 5), (Equation 6), (Equation 7) must be satisfied. Indicates that it must be done. Note that, for example, a plurality of tables may be provided for each category to be classified, or these may be collectively recorded in one table as shown in the present embodiment. Further, the number of records recorded in the table may be singular or plural as shown in the present embodiment.

  Further, as shown in FIG. 3C, the table in which the constraints of threshold variables for categorizing numerical data are composed of a plurality of records, and one record includes a data name, a threshold variable name, and a threshold variable. The constraint equation is recorded. This constraint equation is specified by the user. In FIG. 3C, threshold variables X1, X2, Y1, and Y2 for categorizing numerical data must be in ranges that satisfy (Expression 8), (Expression 9), (Expression 10), and (Expression 11), respectively. It is shown that.

0 <= X1 <= 2000 (8 formulas)
0 <= X2 <= 2000 (9 formulas)
0 <= Y1 <= 2000 (10 formulas)
0 <= Y2 <= 2000 (11 formulas)
Here, these constraint formulas define, for example, the size of a store as a maximum value, and each threshold value cannot be set to be larger than the size of the store. Note that, for example, a plurality of tables may be provided for each category to be classified, or these may be collectively recorded in one table as shown in the present embodiment. Further, the number of records recorded in the table may be singular or plural as shown in the present embodiment.

  Next, data recorded in the prior knowledge rule database DB3 will be described with reference to FIG.

  As shown in FIG. 4, the table recorded in the prior knowledge rule database DB3 is composed of a plurality of records, which are given after categorizing items of category data and numerical data included in many events. An item set including items having labels and the probability of including them (co-occurrence probability) are recorded as prior rules. This pre-rule is specified by the user. In FIG. 4, for example, in the first record, the item “Purchase Item 10” of the purchase data has an item with the label “A product area stay” of the numerical data indicating the position of the body with a probability of 90% or more. Indicates that it is included in the included event. The other is the same, the second record is the item “Purchase Item 20” of the purchase data, “stays in the B product area” with a probability of 80% or more, the third record is the item “Purchase Item 30” of the purchase data , “C product area stay” with a probability of 80% or more, and the fourth record is an item of purchase data “Item 40 purchase” with an item with each label of “D product area stay” with a probability of 80% or more. Indicates that it is included in the included event. Note that a single record may be recorded in this table.

  That is, these data shown in FIG. 4 are so-called rules that are created using the prior knowledge of the user, for example, “90% or more of those who purchased Item 10 stay in the A product area”.

  The prior knowledge rule database DB3 uses a product master in which product categories are hierarchically recorded in accordance with the abstraction level of the classification, so that the user can specify a hierarchy and share it individually and individually. By inputting different co-occurrence probabilities, a prior rule may be automatically created and recorded in the prior knowledge rule database DB3. For example, as shown in FIG. 5 as an example of the product master, the first record includes that the small category Item 10-01 is included in Item 10 in the middle category, and Item 10 is included in Item A in the large category. Represents. Similarly, the second record indicates that the minor category Item 10-02 is included in Item 10 in the middle category, and Item 10 is included in Item A in the larger category. In these classifications, the major classification labels are labels based on broad product categories such as candy, meat and fish, while the middle classification labels are classified according to minor product classifications such as chocolate and rice crackers. The subcategory label is a label given by a detailed product classification including, for example, product names and tastes. By using such a product master, for example, when the user designates a major classification, Item A purchase, stay in the Item A area, and the co-occurrence probability input by the user are recorded in the prior knowledge rule database DB 3 as prior rules. Is done. Similarly, when the middle class is designated by the user, Item 10 purchase, Item 10 area stay, and the co-occurrence probability input by the user are recorded as prior rules in the prior knowledge rule database DB3, and the user designates a smaller class. In this case, the Item 10-01 purchase, the Item 10-01 area stay, and the co-occurrence probability input by the user are recorded as prior rules in the prior knowledge rule database DB3.

  Next, data recorded in the optimization parameter database DB4 will be described with reference to FIG.

  As shown in FIG. 6, the table recorded in the optimization parameter database DB4 is composed of a plurality of records, and one record uses one of the parameters used in the threshold optimization means 5 described later. The parameters used by the threshold optimization means 5 are specified by the user. For example, the initial value maximum search number (Lth), which is a parameter indicating the maximum number of times to search for the initial value, and the maximum number of times to search for the optimum value are set. The maximum search count (Nth), which is a parameter to be expressed, is a threshold increase / decrease value D1, D2 indicating increase / decrease in two types of threshold variables, and a parameter (R) necessary for generating initial values. FIG. 6 shows that Lth = 100, Nth = 10000, D1 = 10, D2 = 1, and R = 10. The threshold increase / decrease values D1 and D2 may be functional expressions that decrease with respect to the number of searches, for example, as shown in (Expression 12).

D1 = α / Nth × N (12 formulas)
N: Number of searches α: Constant value Note that there may be a plurality of records included in this table as described above. For example, when there is one parameter, there may be a single record.

  Next, the threshold optimization unit 5 uses the numerical data and category data recorded in the event set database DB1 under the restriction on the threshold variable recorded in the threshold / constraint database DB2, although detailed description will be described later. This is a means for automatically obtaining the value of the threshold variable recorded in the threshold value / constraint database DB2 so that the prior rule recorded in the prior knowledge rule database DB3 appears appropriately, and the obtained threshold value is a threshold parameter described later. Recorded in the database DB6. As for this threshold variable, the value of the threshold variable may be obtained for each prior rule included in the prior knowledge rule database DB3, or John H. Holland “Adaptation in Natural and Artificial Systems” University of the well-known technology. Optimal threshold that satisfies all the rules of the prior rules contained in the prior knowledge rule database DB3 using a multi-objective optimization method such as GA (Genetic Algorithm) as described in Michigan Press, 1975 You may obtain the value of the variable at once.

  Next, data recorded in the threshold parameter database DB6 will be described with reference to FIG.

  As shown in FIG. 7, the table recorded in the threshold parameter database DB 6 includes a plurality of records. The record includes a data name, a threshold variable name, and an optimum threshold variable value calculated by the threshold optimization unit 5. Is recorded. In FIG. 7, the first record indicates that the value of the threshold variable X1 for categorizing numerical data indicating the position of the body is 100. Similarly, the value of the threshold variable X2 is 200, the value of the threshold variable Y1 is 100, the value of the threshold variable Y2 is 200, and the value of the threshold variable T1 is 15. Note that there may be a plurality of this table for each category to be classified, for example, or these may be recorded together in one table as shown in the present embodiment. For example, when the threshold variable is singular, the record recorded in the table may be singular.

  Finally, the threshold display device 7 is a device for displaying the optimum threshold variable recorded in the threshold parameter database DB6 to the user, and corresponds to, for example, a normal display device.

  Next, a method for obtaining an optimum solution of threshold variables for one pre-rule as the threshold optimization means 5 will be described with reference to FIGS.

  The threshold optimization means 5 is a processing procedure 1 for initializing the value of a threshold variable for categorizing numerical data, and a processing procedure for optimizing the value of the threshold variable set in the processing procedure 1 It is roughly divided into two. FIG. 8 shows a flowchart showing the processing procedure 1 and FIG. 9 shows a processing procedure 2.

  First, the processing procedure 1 for initializing the value of the threshold variable for categorizing numerical data will be described with reference to FIG.

  As shown in FIG. 8, in the processing procedure 1, first, the prior knowledge rule database DB3 is referred to, and the jth prior rule is extracted from the prior rules recorded there (S101). j indicates the order in which the extracted prior rules are stored. For example, when j = 1, the stored first prior rule is extracted. When the prior knowledge rule database DB3 is FIG. 4, the extracted prior rules are “Item 10 purchase” and “A product area stay”, and the co-occurrence probability is 90%.

  Next, with reference to the event set database DB1, all IDs of all observed objects including the corresponding category labels are extracted. In the above-described example, all the IDs of the observed objects having the category data “Purchase Item 10” are extracted. Then, based on the data name of the numerical data to be categorized included in the pre-rule and the extracted ID of the object to be observed, the target object data is extracted from the event set database DB1, and the pre-rule corresponding event set Creation is performed (S102). Here, the number of objects to be observed included in the event set is N1.

  Next, with reference to the threshold / constraint database DB2, a threshold variable, its conditional expression and constraint expression regarding the label of the numerical data to be categorized included in the pre-rule extracted in S101 are extracted (S103). For example, when the label of the numerical data to be categorized included in the prior rule is “A product area stay” and the threshold variable, the conditional expression, and the constraint expression are FIG. 3A, FIG. 3B, and FIG. Five types of X1, X2, Y1, Y2, and T1 are extracted. Further, the above-described (Expression 5), (Expression 6), and (Expression 7) are extracted as conditional expressions. Further, the above-described (Equation 8), (Equation 9), (Equation 10), and (Equation 11) are extracted as constraint equations.

  Next, the number (N2) of data points included in a circle with a radius R is calculated for all data points included in the pre-rule applicable event set data set created in S102. This circle is a circle centered at the i-th largest point (Xs, Ys). Using (Xs, Ys) at this time, the initial values of X1, X2, Y1, and Y2 are expressed by Equation (13). ), (14 formula), (15 formula), (16 formula), and (17 formula) (S104).

X1S = Xs-R (13 formulas)
X2S = Xs + R (14 formulas)
Y1S = Ys-R (15 formulas)
Y2S = Ys + R (16 formulas)
T1S = N2 / N1 (17 formulas)
Here, i represents the number of times the initial value is generated, and is a variable that increases each time the generation of the initial value is repeated. R may be set freely by the user.

  Next, it is determined whether or not the initial value generated in S104 satisfies the constraint expression extracted in S103 (S105). As a result of the determination, if the initial value satisfies the constraint equation, the process proceeds to process procedure 2 described later.

  If the initial value does not satisfy the constraint expression in S105, i = i + 1 is set, and the optimization parameter database DB5 is referenced to determine whether the initial search count i is smaller than the maximum initial search count Lth (S106). If it is determined that the initial search count is smaller than the maximum initial search count, the process proceeds to S104.

  On the other hand, if the result of determination in S106 is that the initial search count i is greater than or equal to the maximum initial search count Lth, the user is informed that the threshold value is not found (S107).

  By the processing procedure 1 as described above, initial setting of a threshold variable value for categorizing numerical data is performed. In the above example, when i = 1, for example, among all the data points having the ID of the observed object including the category data “Purchase Item 10”, the data point included in the circle with the radius R is the largest. The threshold is initially set according to (Expression 13) to (17) with the center position as a reference.

  Next, process procedure 2 for optimizing the threshold set in process procedure 1 will be described with reference to FIG.

  As shown in FIG. 9, in the process procedure 2, first, the numerical data is categorized using the threshold values X1S, X2S, Y1S, Y2S, and T1S obtained in the process procedure 1 to obtain the co-occurrence probability (R ′) ( S201). At this time, categorization is determined by whether or not the numerical data having the ID of the observed object including the category data of “Item 10 purchase” satisfies (Expression 5) to (Expression 7) in the above example, For example, a label “stay in the product A area” is given to the ID to be satisfied. Further, the co-occurrence probability indicates, for example, a ratio in which numerical data included in this ID among all IDs including category data “Item 10 purchase” is classified into the category “product A area stay”.

  Next, the co-occurrence probability (R) recorded in the prior knowledge rule database DB3 is compared with the co-occurrence probability (R ') obtained in S201 (S202). If R ′ <R is not satisfied, X1 = X1S, X2 = X2S, Y1 = Y1S, Y2 = Y2S, and T1 = T1S, and these values are used as the output of the threshold optimization unit 5.

On the other hand, if R ′ <R is satisfied in S202, X1S ′ = X1S−D1 is set (S203). Next, with reference to the threshold / constraint database DB2, it is determined whether or not X1S ′ obtained in S203 satisfies the constraint (S204). If X1S ′ does not satisfy the constraint, R′1 = 0 is set, and the process proceeds to S206 described later.

  On the other hand, if X1S ′ satisfies the constraints, the numerical data is categorized using X1S ′, X2S, Y1S, Y2S, and T1S, and the co-occurrence probability (R′1) is obtained (S205).

  Next, X2S '= X2S + D1 is set (S206).

Next, referring to the threshold / constraint database DB2, it is determined whether or not X2S ′ obtained in S206 satisfies the constraint (S207). If X2S ′ does not satisfy the constraint, R′2 = 0 is set, and the process proceeds to S209 described later.

  On the other hand, if X2S ′ satisfies the constraints, the numerical data is categorized using X1S, X2S ′, Y1S, Y2S, and T1S, and the co-occurrence probability (R′2) is obtained (S208).

  Next, Y1S '= Y1S-D1 is set (S209).

Next, with reference to the threshold / constraint database DB2, it is determined whether or not Y1S ′ obtained in S209 satisfies the constraint (S210). If Y1S ′ does not satisfy the constraint, R′3 = 0 is set, and the process proceeds to S212 described later.

  On the other hand, if Y1S ′ satisfies the constraints, the numerical data is categorized using X1S, X2S, Y1S ′, Y2S, and T1S, and the co-occurrence probability (R′3) is obtained (S211).

  Next, Y2S '= Y2S + D1 is set (S212).

Next, referring to the threshold / constraint database DB2, it is determined whether or not Y2S ′ obtained in S212 satisfies the constraint (S213). If Y2S ′ does not satisfy the constraint, R′4 = 0 is set, and the process proceeds to S215 described later.

  On the other hand, if Y2S ′ satisfies the constraint, the numerical data is categorized using X1S, X2S, Y1S, Y2S ′, and T1S to obtain the co-occurrence probability (R′4) (S214).

  Next, T1S '= T1S-D2 is set (S215).

Next, with reference to the threshold / constraint database DB2, it is determined whether or not T1S ′ obtained in S215 satisfies the constraint (S216). If T1S ′ does not satisfy the constraint, R′5 = 0 is set, and the process proceeds to S218 described later.

  On the other hand, if T1S 'satisfies the constraint, the numerical data is categorized using X1S, X2S, Y1S, Y2S, T1S' to determine the co-occurrence probability (R'5) (S217).

  Next, the maximum value of R′1, R′2, R′3, R′4, and R′5 is set as R ′, and the value of the threshold variable when the co-occurrence probability is maximized is updated (S218). . For example, when R′1 is the maximum value, only the value of X1S is set to X1S = X1S ′. Similarly, when R′2 is the maximum value, only the value of X2s is X2S = X2S ′, and when R′3 is the maximum value, only the value of Y1S is Y1S = Y1S ′, and R′4 is the maximum value. If there is, only the value of Y2S is T2S = T2S ′, and if R′5 is the maximum value, only the value of T1S is T1S = T1S ′.

  Next, the co-occurrence probability (R) recorded in the prior knowledge rule database DB3 is compared with the co-occurrence probability (R ') obtained in S218 (S219). If R ′ <R is not satisfied, X1 = X1S, X2 = X2S, Y1 = Y1S, Y2 = Y2S, and T1 = T1S, and these values are used as the output of the threshold optimization unit 5.

  On the other hand, when R ′ <R is satisfied in S219, the optimization parameter database DB4 is referred to, and the maximum number of searches (Nth) is compared with the current number of searches N (S220). If N <Nth and R′1 = R′2 = R′3 = R′4 = R′5 = 0 are satisfied, the number of searches N is updated to N = N + 1, and the process proceeds to S203. On the other hand, when N <Nth and R′1 = R′2 = R′3 = R′4 = R′5 = 0 are not satisfied, the number of generations i of the initial value of the threshold shown in the processing procedure 1 is set to i. = I + 1, and the process proceeds to S106 of process procedure 1.

  By performing the processing as described above, it is possible to automatically obtain the optimum solution of the value of the threshold variable for categorizing numerical data.

  As described above, according to the preprocessing device using the pre-rule according to the present embodiment, the threshold variable recorded in the threshold / constraint database DB2 so that the pre-rule appears appropriately by the threshold optimizing means 5. Can be automatically determined. That is, even if the user cannot determine the threshold value, the value of the threshold variable can be determined easily and appropriately.

  In addition, the value of the threshold variable obtained in this way can be displayed to the user in various forms using the threshold display device 7. For example, the figure may be drawn using the value of the variable designated by the user among the threshold variables included in the condition of the same label having the same data name. An example of this is shown in FIG. As shown in FIG. 10, all the product areas 20 can be illustrated by using the value of the threshold variable derived from the prior rule. In this way, it is possible to visually grasp the place where a purchaser of a certain product stays, the place where a plurality of product areas are excessively congested, and the like. Can gain knowledge when taking measures such as reviewing

(Second Embodiment)
Next, an information extraction apparatus using a preprocessing apparatus using a pre-rule according to the first embodiment will be described with reference to FIGS.

  FIG. 11 is a block diagram schematically showing the configuration of an information extraction device using a preprocessing device using a pre-rule according to the first embodiment.

  As shown in FIG. 11, the information extraction apparatus according to this embodiment includes a threshold optimization processing unit and an information extraction unit. Among these, the threshold optimization processing unit has the same configuration as the preprocessing apparatus using the pre-rule shown in the first embodiment. However, in the present embodiment, the event set database DB1 is referred to as a first event set database DB1. Further, the threshold value display device 7 is not necessary, and if it is desired to display an optimized threshold value, it may be presented to the user using the correlation rule display device 13 described later with reference to the threshold parameter database DB6.

  On the other hand, the information extraction unit includes a second event set database DB7, a continuous data categorization means 8, a post-conversion event set database DB9, a correlation rule extraction parameter database DB10, a correlation rule extraction means 11, and a correlation rule database DB12. And the correlation rule display device 13.

  Then, each element which comprises such an information extraction apparatus is demonstrated with reference to FIGS. In this information extraction apparatus, the threshold optimization processing unit is the same as that in the first embodiment, and thus description thereof is omitted. Here, each element constituting the information extraction unit will be described.

  First, data recorded in the second event set database DB7 will be described.

  The second event set database DB7 is basically the same as the first event set database DB7. That is, the second event set database DB7 has a table composed of a plurality of records, respectively, and the ID of the observed object and the numerical data or category data of the observed object are recorded in one record. The data relating to the observed object recorded in the second event set database DB7 may be data acquired by the same type of sensor or device as the data recorded in the first event set database DB1, or a part thereof. But you can. Further, it may be data acquired by different types of sensors or devices. However, if the data related to the object recorded in the second event set database DB7 includes data acquired by a sensor or device different from the data recorded in the first event set database DB1, these data Must be categorical data, not numeric data. The data recorded in the first event set database DB1 and the data recorded in the second event set database DB7 may be exactly the same.

  Next, the numerical data categorizing means 8 will be described.

  The numerical data categorizing means 8 uses the numerical data of the observed object recorded in the second event set database DB7, the conditional expression recorded in the threshold / constraint database DB2, and the optimum recorded in the threshold parameter database DB6. It is a means for converting into category data using the value of the normalized threshold variable. A label corresponding to the category is given to the ID of the observed object having the numerical data categorized by the numerical data categorizing means 8 and recorded in the post-conversion event set database DB9.

  Next, data recorded in the post-conversion event set database DB9 will be described with reference to FIG.

  The post-conversion event set database DB9 is composed of a table in which labels to be given by categorizing numerical data by the numerical data categorizing means 8 and a table in which labels of category data are recorded. It is recorded with. Among these, the table in which the labels assigned to the categorized numerical data are recorded is composed of a plurality of records as shown in FIG. 12, and one record includes an object ID, a data name, and a data type. The observation time and label name are recorded. FIG. 12 shows that, for example, the observed object 001 “stays in the product area A” at 15:30:30 on 2007/08/01. Similarly, the observed object 001 “passed through the B product area” at 15:40:00 on 2007/08/01 and “stayed in the C product area” at 15:45:00 on 2007/08/08 Is shown. On the other hand, the table in which the label of category data is recorded is the same as that shown in FIG. 2B, for example. Note that there may be a plurality of these tables, for example, for each person to be observed, or these may be collectively recorded in one table. Further, the number of records recorded in the table may be singular or plural as shown in the present embodiment.

  Next, data recorded in the correlation rule extraction parameter database DB10 will be described with reference to FIG.

  As shown in FIG. 13, the table recorded in the correlation rule extraction parameter database DB10 is composed of a plurality of records, and parameters necessary for extracting correlation rules by the correlation rule extraction means 11 described later are included in one record. One of these is recorded. In FIG. 13, as an example of parameters used in the correlation rule extraction unit 11, the minimum support level (Sup) representing a condition that must be satisfied to be extracted as a correlation rule is 0.2, and the minimum certainty factor (Sup) Conf) is 0.6. The minimum support level and the minimum confidence level are specified by the user. Here, the support level is calculated using (Equation 18) and the certainty factor is calculated using (Equation 19).

Degree of support S (X∧Y) = M (X∧Y) / M (18 formulas)
M (X∧Y): Number of events (observed objects) including item set “X and Y” M: Number of all events (observed objects) Confidence C (X∧Y) = M (X∧Y) / M (X) (Equation 19)
M (X): Number of events (observed objects) including item set “X” In the above-described correlation rule extraction parameter database DB10, the minimum support level and the minimum certainty factor are recorded. If the item set Y is “Item 01 purchase” and the item set Y is “Item 01 stay”, “Item A stay”, “Item B stay”, and “Item 01 purchase” are included. When the observed objects are 20% or more of all the observed objects, and 60% or more of the observed objects including ““ A product stay ”and“ B product stay ”include“ Item01 purchase ”. “If“ A product stay ”and“ B product stay ”,“ Item 01 purchase ”” is extracted as a correlation rule. In addition, the record contained in this table may be single, for example, when there is one parameter.

  Next, the correlation rule extracting means 11 extracts a correlation rule for the observed event recorded in the post-conversion event set database DB 9 using the parameters recorded in the correlation rule extraction parameter database DB 10. , Means for recording in the correlation rule database DB12. Hereinafter, the correlation rule extracting unit 11 will be described with reference to FIG.

  FIG. 14 is a flowchart showing the processing procedure of the correlation rule extraction means 11.

  As shown in FIG. 14, the correlation rule extraction unit 11 first refers to the post-conversion event set database DB9 and generates a candidate set of correlation rules having a sequence length k (S301). Here, the sequence length refers to the number of items included in the association rule. The initial value of k is 1, and the method for generating association rule candidates is different between k = 1 and k> 1.

  When k = 1, items included in all events are candidates. On the other hand, in the case of k> 1, among the association rules extracted as association rules having a sequence length of k−1, candidates are generated by combining association rules that share k−2 items. For example, as an association rule with a sequence length of 3, “Product A Area Stay”, “Product B Area Stay”, “Item 01 Area Purchase”, “Product B Area Stay”, “Product C Area Stay”, “Item 11 Purchase” ”,“ Product B Area Stay ”,“ Product C Area Stay ”,“ Item01 Purchase ””, the candidate for the correlation rule of sequence length 4 is ““ Product A Area Stay ”, “Product B area stay”, “Product C area stay”, “Item 01 purchase” and “Product B area stay”, “Product C area stay”, “Item 01 purchase”, “Item 11 purchase”.

  Next, the number of correlation rule candidates included in the correlation rule candidate set generated in S301 is counted, and it is determined whether or not the number is greater than 0 (S302). In the case of 0, the correlation rule extracting means 11 is terminated.

  On the other hand, when the number of correlation rule candidates included in the correlation rule candidate set generated in S301 is greater than 0, the conversion event group database DB9 is referred to, and each correlation rule candidate generated in S301 is converted into the conversion event group database DB9. To check whether it is included. Then, the number of events (observed objects) including the generated association rules is counted, and the support level and the certainty level are calculated (S303).

  Next, referring to the correlation rule extraction parameter database DB10, if the support level of the correlation rule candidate calculated in S303 is greater than or equal to the minimum support level and the confidence level of the correlation rule candidate is greater than or equal to the minimum confidence level, this correlation rule candidate Is recorded in the correlation rule database DB12 described later as a correlation rule (S304).

  Next, the number of association rules of sequence length k recorded in S304 is counted, and it is determined whether or not the number is greater than 0 (S305). In the case of 0, the correlation rule extracting means 11 is terminated.

  On the other hand, if the correlation rule of sequence length k recorded in S304 is greater than 0, k = k + 1 is set and the process returns to S301.

  An association rule can be generated by the above procedure. In the above example, the order between items is not considered, but may be considered.

  Finally, data recorded in the correlation rule database DB12 will be described with reference to FIG.

  As shown in FIG. 15, the correlation rule database DB12 has a table in which the extracted correlation rules are recorded. This table is composed of a plurality of records, and a correlation rule extracted by the correlation rule extraction means 11 is recorded in one record. In FIG. 15, for example, the first record records a correlation rule “If“ A product area stay ”(condition part) is“ Item 01 purchase ”(conclusion part)”, and supports this correlation rule. The degree is 0.5 and the certainty is 0.7. Similarly, in the second record, the correlation rule “If“ A product area stay ”and“ B product area stay ”(condition part) is“ Item 10 purchase ”(conclusion part)” is recorded. If the association rule support level is 0.3 and the confidence level is 0.8, the third record is “A product area stay” and “C product area stay” (condition part) The correlation rule “It is an Item 20 purchase” (conclusion part) is recorded, which indicates that the support degree of this correlation rule is 0.4 and the certainty factor is 0.7. Note that the number of records included in this table may be singular if, for example, there is one extracted correlation rule.

  Finally, the correlation rules generated as described above and recorded in the correlation rule database DB 12 are displayed on the correlation rule display device 13. At this time, the order of the association rule sequence length, the degree of support, the degree of certainty may be displayed in a different order, or only the association rules including a specific item set may be extracted and displayed. , You may display freely according to the purpose. The correlation rule display device 13 is, for example, a normal display device, and is the same as the threshold value display device 7 in the first embodiment. These correlation rule display device 13 and threshold value display device 7 may be anything as long as they can provide the user with a correlation rule or an optimized threshold value visually.

  As described above, according to the information extraction apparatus according to the present embodiment, the value of the threshold variable can be automatically determined so that the prior rule appears appropriately. That is, even if the user cannot determine the threshold value, the value of the threshold variable can be determined easily and appropriately. Since the numerical data is categorized using the value of the threshold variable thus obtained, it is possible to easily and efficiently extract useful correlation rules that the user could not know.

  It should be noted that by extracting correlation rules that the user could not know, it is possible to apply mainly to the field related to marketing, such as where to display an advertisement for a certain product.

  Although the embodiment of the present invention has been described above, the embodiment is not limited to this and can be applied in various ways.

  For example, by using the pointer movement trajectory on the web page as numerical data and the clicked location on the web page as category data, the value of the threshold variable is automatically set so that these prior rules appear appropriately. Can be determined. That is, even if the user cannot determine the threshold value, the value of the threshold variable can be determined easily and appropriately. Since the numerical data is categorized using the value of the threshold variable thus obtained, it is possible to easily and efficiently extract useful correlation rules that the user could not know.

It is a block diagram which shows roughly the structure of the pre-processing apparatus using the prior rule in embodiment of this invention. It is a figure which shows the table which recorded the data which show the position of the body contained in an event set database. It is a figure which shows the table which recorded the purchase data contained in an event set database. It is a figure which shows the table which recorded the threshold variable contained in a threshold value and restrictions database. It is a figure which shows the table which recorded the numerical data name contained in a threshold value and restrictions database, the label after categorization, and the conditions of numerical data applicable to the label. It is a figure which shows the table by which the restriction | limiting of the threshold variable contained in a threshold value and restrictions database was recorded. It is a figure which shows the table contained in the prior knowledge rule database. It is a figure which shows a goods master It is a figure which shows the table contained in the optimization parameter database. It is a figure which shows the table contained in a threshold value parameter database. It is a flowchart which shows the procedure which performs the initial setting of a threshold value in a threshold value optimization means. It is a flowchart which shows the procedure which optimizes a threshold value in a threshold value optimization means. It is a figure which shows the example of a display of a threshold value display apparatus. It is a block diagram which shows the correlation rule extraction apparatus in embodiment of this invention. It is a figure which shows the table which recorded the result which categorized numerical data which shows the position of the body contained in the event set database after conversion. It is a figure which shows the table contained in a correlation rule extraction parameter database. It is a flowchart which shows the procedure which extracts an association rule in an association rule extraction means. It is a figure which shows the table contained in a correlation rule database.

Explanation of symbols

  DB1 ... (first) event set database, DB2 ... threshold / constraint database, DB3 ... prior knowledge rule database, DB4 ... optimization parameter database, DB6 ... threshold parameter database, DB7 ... -Second event set database, DB9 ... converted event set database, DB10 ... correlation rule extraction parameter database, DB12 ... correlation rule database, 5 ... threshold optimization means, 7 ... threshold display Device: 8 ... Numerical data categorizing means, 11 ... Correlation rule extracting means, 13 ... Correlation rule display device, 20 ... Product area, 21 ... Product shelf.

Claims (9)

An event set database that records trajectory data having the same ID and category data that is categorized and each labeled with a first label;
Conditions and constraints of threshold variables necessary for categorizing the trajectory data recorded in the event set database, and a second label attached to the trajectory data categorized based on these conditions and constraints A threshold / constraint database with
A prior knowledge rule database in which a combination of the first and second labels co-occurring with a predetermined probability is recorded as a prior rule together with the probability;
Threshold optimization means for calculating the value of the threshold variable so as to extract all the prior rules previously included in the prior knowledge rule database from the event set database under the constraints;
A threshold database in which values of the threshold variables calculated by the threshold optimization means are recorded;
A display device for displaying the value of the threshold variable recorded in the threshold database;
A preprocessing apparatus for an information extraction apparatus, comprising:   The prior rule recorded in the prior knowledge rule database is an association rule automatically generated using a product master in which categories of the category data are hierarchically recorded according to the abstraction level of classification. The preprocessing device for an information extraction device according to claim 1. A preprocessing device for an information extraction device according to claim 1 or 2,
Numeric data categorizing means for converting the trajectory data included in the event set database into category data using the calculated threshold variable value;
A post-conversion event set database that records candidate association rules composed of combinations of the second labels and the first labels attached to the trajectory data categorized by the numerical data categorizing means;
Correlation rule extraction means for extracting as a correlation rule from the correlation rule candidates recorded in the post-conversion event set database using a correlation rule extraction parameter;
A correlation rule database that records the correlation rules extracted by the correlation rule extraction means;
And the display device is a display device for displaying the correlation rule. The association rule extraction parameters are support level and certainty level,
4. The information according to claim 3, wherein the correlation rule extraction unit is a unit that extracts the correlation rule candidate having a certain degree of support and certainty among the correlation rule candidates as a correlation rule. Extraction device. Record the trajectory data having the same ID and the categorized category data each labeled with the first label in the event set database,
Conditions and constraints of threshold variables necessary for categorizing the trajectory data recorded in the event set database, and a second label attached to the trajectory data categorized based on these conditions and constraints In the threshold / constraint database,
A combination of the first and second labels co-occurring with a predetermined probability is recorded in the prior knowledge rule database as a prior rule together with the probability;
Extracting the prior rules from this prior knowledge rule database,
Extracting all the trajectory data of the ID including the first label of the extracted prior rule from the event set database,
Extract the conditions and constraints from the threshold / constraint database,
A pre-processing method for an information extracting apparatus, wherein the value of the threshold variable is calculated so that the pre-rule appears with the highest probability under the extracted restriction. The means for calculating the value of the threshold variable is
Under the constraint, the initial value of the threshold variable value is calculated from the extracted trajectory data,
The trajectory data is categorized using the calculated initial value,
Calculating a co-occurrence probability between the second label attached to the categorized trajectory data and the first label;
Among the calculated co-occurrence probabilities and the co-occurrence probabilities newly obtained by changing at least one of the initial values, calculating the value of the threshold variable when the highest co-occurrence probability is obtained. The preprocessing method of the information extraction apparatus according to claim 5, wherein the information extraction apparatus is preprocessed.   The prior rule recorded in the prior knowledge rule database is a rule automatically generated using a product master in which categories are hierarchically recorded in accordance with an abstraction level of classification. The preprocessing method of the information extraction apparatus of 5 or 6. Using the value of the threshold variable optimized by the preprocessing method of the information extraction device according to any one of claims 5 to 7, categorizing the trajectory data recorded in the event set database,
Record the combination of the second label attached to the categorized trajectory data and the first label as a correlation rule candidate in the post-conversion event set database,
An information extraction method characterized in that, from the correlation rule candidates recorded in the post-conversion event set database, the correlation rule candidates extracted using correlation rule parameters are used as correlation rules. The association rule extraction parameters are support level and certainty level,
The information extraction method according to claim 8, wherein among the correlation rule candidates, the correlation rule candidate having a certain degree of support and certainty is extracted as a correlation rule.

Images