JP2012208754A - Information processing device, update method for updating database, and program for updating database - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce a load on a user by enhancing the efficiency of work relating to optimization of a database.SOLUTION: An information processing device connected to a database in which analysis data including an analysis key for analyzing document data is accumulated, comprises: a unit analysis key extraction unit 25 that extracts one or plural analysis keys to be a reference for recognizing the structure of an analysis key from the database as unit analysis keys; a structure recognition unit 26 that recognizes the structure of the analysis key included in the database using the unit analysis keys; and a database update unit 29 that updates information related to the analysis key included in the database using information related to the unit analysis keys according to the structure recognized by the structure recognition unit 26.

Description

  The present invention relates to a technique for updating an analysis key used when analyzing document data.

  Conventionally, update proposals for received hierarchical classification dictionaries are stored in the proposal history storage unit, and approximate proposals that approximate the received update proposals are searched and extracted from past update proposals stored in the proposal history storage unit. There is a classification dictionary update method that presents extracted approximate proposals (see Patent Document 1).

  In addition, a keyword extraction unit that extracts keywords from the input text information, a keyword statistics unit that obtains statistics on keyword appearance, and a keyword evaluation value that calculates an evaluation value of a keyword extracted based on the statistics on keyword appearance Calculation unit, determination unit for determining whether to register or delete this keyword based on the calculated evaluation value, registration or deletion of a keyword in the dictionary database according to a result of determination performed by the determination unit There is a dictionary creation / deletion unit that performs and a dictionary creation device that includes a dictionary database (see Patent Document 2).

  Furthermore, with regard to natural language analysis technology, in particular, for each word of a sentence composed of natural language, word meaning assignment that assigns a semantic tag indicating a meaning that matches the context, a semantic class indicating a concept corresponding to the corresponding meaning, and the like. There is a device (see Patent Document 3). In addition, various techniques for constructing or updating a dictionary have been proposed (see Patent Documents 4 and 5).

JP 2006-309446 A Reissue 2005/0666837 JP 2009-181408 A JP 2008-234429 A JP-A-2005-174116

  Conventionally, there is a technique for analyzing target data such as document data using dictionary data. In order to obtain a highly accurate analysis result using such a technique, it is necessary to maintain a database (dictionary) including dictionary data. However, when organizing and optimizing the database, the user needs to be aware of the structure of the database and each data to be organized, and the dictionary data optimization work is cumbersome and burdensome to the user It was a thing.

  In view of the above-described problems, an object of the present invention is to improve the efficiency of work related to database optimization and reduce the burden on the user.

The present invention has the following configuration to solve the above-described problems. That is, the present invention is an information processing apparatus connected to a database in which analysis data including an analysis key for analyzing document data is stored, and a reference for grasping the configuration of the analysis key from the database A unit analysis key extracting means for extracting one or a plurality of analysis keys as unit analysis keys, a structure grasping means for grasping a structure of an analysis key included in the database using the unit analysis key, and the structure An information processing apparatus comprising: database updating means for updating information associated with an analysis key included in the database using information associated with the unit analysis key according to a configuration grasped by the grasping means.

  Here, the analysis key is information used as a key for analyzing document data, and is information used as a search key when searching for a character string of document data, for example. The analysis data includes an analysis key and may further include information related to the analysis key (such as attribute information and semantic information of the analysis key).

  The analysis key can include other analysis keys. Therefore, the information processing apparatus according to the present invention extracts an analysis key serving as a reference for grasping the configuration of the analysis key as a unit analysis key, and uses this unit analysis key to analyze including the unit analysis key. Know the key structure. The present invention updates the database according to the configuration grasped in this way, thereby making it possible to improve the efficiency of the work related to the optimization of the database and reduce the burden on the user.

  Further, the unit analysis key extracting means may extract, as the unit analysis key, an analysis key that is searched only when it is used as a search key from the database.

  In other words, the unit analysis key extraction means can extract, as the unit analysis key, an analysis key that is not searched when an analysis key other than itself is used as a search key from the database. Since such a unit analysis key does not include other analysis keys, it can be used as a minimum unit for grasping the configuration of the analysis key.

  The information processing apparatus further includes database search means for searching the database using the analysis key stored in the database, and the unit analysis key extraction means is a result of the search by the database search means, An analysis key searched only when it is used as a search key may be extracted as the unit analysis key.

  The information processing apparatus further includes an additional unit analysis key extracting unit that further extracts, as an additional unit analysis key, a character string that does not correspond to any of the unit analysis keys in the configuration grasped by the configuration grasping unit. May be.

  By further providing such additional unit analysis key extraction means, it becomes possible to extract unit analysis keys that have not been extracted by a method such as search. At this time, a character string having no meaning as the character string itself (for example, the conjunctions “ga”, “ha”, etc.) may be excluded from the objects to be extracted as the additional unit analysis key.

  The information processing apparatus further includes additional unit analysis key setting means for setting semantic information generated based on a character string that does not correspond to any of the unit analysis keys as semantic information of the additional unit analysis key. You may prepare.

  Further, the database updating means may update the semantic information of the analysis key associated with the analysis key included in the database using the semantic information of the unit analysis key associated with the unit analysis key. Good.

The analysis key may be defined using a regular expression. By using the analysis key defined by regular expressions, when analyzing the target data such as document data using the analysis key, the necessary feature part is searched without being affected by the fluctuation of the target data. I can do it. In order to more accurately search for such a feature portion, it is preferable to use a regular expression of an analysis key that is not affected by fluctuations in notation such as colloquial style or subject omission.

  Furthermore, the present invention can be understood as a method executed by a computer or a program executed by a computer. Further, the present invention may be a program in which such a program is recorded on a recording medium readable by a computer, other devices, machines, or the like. Here, a computer-readable recording medium is a recording medium that stores information such as data and programs by electrical, magnetic, optical, mechanical, or chemical action and can be read from a computer or the like. Say.

  According to the present invention, work related to database optimization can be made efficient, and the burden on the user can be reduced.

It is a figure which shows the hardware constitutions of the document data analysis apparatus which concerns on embodiment. 1 is a diagram illustrating a conceptual configuration of a document data analysis apparatus according to an embodiment. It is a figure which shows the outline of a function structure of the document data analysis apparatus which concerns on embodiment. It is a figure which shows the structure of the dictionary data table which concerns on embodiment. It is a figure which shows the structure of the unit analysis key table which concerns on embodiment. It is a flowchart which shows the flow of the document data analysis process which concerns on embodiment. It is a figure which shows the result of the search process using the dictionary data table in embodiment. It is flowchart A which shows the flow of the database update process which concerns on embodiment. It is a flowchart B which shows the flow of the database update process which concerns on embodiment. It is a flowchart C which shows the flow of the database update process which concerns on embodiment. It is a table | surface which shows the relationship between the determination result by the conformity degree determination part which concerns on embodiment, and the content of the process corresponding to it. It is a figure which shows a mode that each table in a dictionary database is updated using the data for an update contained in the data table for an update when the database update process which concerns on embodiment is performed. It is a flowchart which shows the flow of the database optimization process which concerns on embodiment.

  Hereinafter, an embodiment in which an information processing apparatus according to the present invention is implemented as a document data analysis apparatus 1 will be described with reference to the drawings. The document data analysis apparatus 1 according to the present embodiment can be used, for example, for analyzing reception logs input and accumulated by an operator at a call center. In the call center, telephone calls such as inquiries from customers are accepted, and the operator inputs customer correspondence records into the computer. A part of the reception log input and accumulated by the operator is input in text. However, the document data analysis apparatus 1 according to the present embodiment can be used for the purpose of analyzing various sentences (for example, questionnaire results and the like) in addition to the call center reception log.

However, the information processing apparatus according to the present invention is not limited to the document data analysis apparatus 1. The information processing apparatus according to the present invention may be an information processing apparatus connected to a database in which analysis data including an analysis key for analyzing document data is stored.

<System configuration>
FIG. 1 is a diagram illustrating a hardware configuration of a document data analysis apparatus 1 according to the present embodiment. The document data analysis apparatus 1 includes a CPU (Central Processing Unit) 11, a RAM (Random Access Memory) 13 as a main storage device, a ROM (Read Only Memory) 12, an HDD (Hard Disk Drive), an SSD (Solid State Drive), and the like. The auxiliary storage device 14, the display device 15 as a display, and the input device 16 as a computer (information processing device) including a keyboard and a mouse. The document data analysis apparatus 1 is connected to a dictionary database.

  FIG. 2 is a diagram showing a conceptual configuration of the document data analysis apparatus 1 according to the present embodiment. According to the document data analysis apparatus 1 according to the present embodiment, a form that takes into account the context of a sentence from a document composed of a plurality of sentences, which cannot be extracted by a conventional knowledge acquisition technique based on text analysis, or a human intuitive The feature expression of a simple language can be converted into a regular expression and registered in the language analysis dictionary, the range of analysis can be expanded, and analysis, classification, and knowledge acquisition of documents with many abbreviations such as questionnaires can be handled. Further, since the document data analysis apparatus 1 according to the present embodiment has a document analysis function based on the feature pattern of the regular expression, the pattern analysis of the dependency relation extracted after being finely divided into conventional morphemes, It is possible to more accurately extract the relationship between actual notation patterns and to easily extract intuitive expression patterns by people. In the description of the present embodiment, details of processing when the database update processing and the database optimization processing are applied to the document data analysis apparatus 1 will be described.

  FIG. 3 is a diagram showing an outline of a functional configuration of the document data analysis apparatus 1 according to the present embodiment. In the computer having the configuration shown in FIG. 1, the program recorded in the auxiliary storage device 14 is read out to the RAM 13 and executed by the CPU 11, whereby the update analysis key acquisition unit 21 and the database search unit 22 are executed. , Conformity determination unit 23, update processing content determination unit 24, unit analysis key extraction unit 25, configuration grasping unit 26, additional unit analysis key extraction unit 27, additional unit analysis key setting unit 28, database update unit 29, and document data analysis It functions as the document data analysis apparatus 1 including the unit 30.

  The dictionary database includes various tables (analysis including a plurality of analysis keys defined in advance using normal character strings or regular expressions, attribute information corresponding to the analysis key, and semantic information indicating the meaning of the analysis key itself). A table in which keys are grouped for each attribute, a unit analysis key table, and the like). It should be noted that it is preferable that different dictionaries are prepared for each field, and a dictionary in an appropriate field is preferentially used according to the field to which the document data belongs.

FIG. 4 is a diagram showing a configuration of the dictionary data table according to the present embodiment. The dictionary data table includes an analysis key ID for identifying an analysis key, one analysis key defined by a normal character string or a character string using a regular expression, attribute information corresponding to the analysis key, and the analysis key. It is preferable that different dictionary data is prepared for each field of the analysis target document data. In principle, the analysis key is defined by a regular expression, and attribute information of a sentence that matches the regular expression is associated with each analysis key. For example, a sentence that matches the regular expression “. * I want to know” is classified into a response and an opinion, and the meaning of the sentence is “I want to know”. For this reason, the analysis key “. * Know (Tai | Takata)” has attribute information 1 “response” and attribute information 2 “
Opinion "and semantic information" I want to know "are associated.

  The update analysis key acquisition unit 21 acquires update data including an update analysis key that is an analysis key used for updating the dictionary database. In the present embodiment, the analysis key is defined using a regular expression. See FIG. 10 for a specific example of the update data.

  The database search unit 22 searches the dictionary database using a predetermined search key. Examples of the search key used here include an analysis key stored in the same dictionary database in addition to an analysis key for update.

  The degree of conformity determination unit 23 determines the degree of conformity between the information included in the analysis data and the information associated with the update analysis key. More specifically, the suitability determination unit 23 determines the suitability when the database search unit 22 does not search for analysis data including the update analysis key from the dictionary database. The conformity degree determination unit 23 includes at least one of an analysis key included in the analysis data, attribute information and semantic information of the analysis key, an analysis key associated with the update analysis key, attribute information of the analysis key, and The degree of conformity with at least one of the semantic information is determined. Note that the result of the determination is held using information indicating “complete match”, “partial match”, “mismatch”, “determination unnecessary”, and the like. In the table shown in FIG. 9, the complete match is displayed with a symbol “◯”, the partial match with “Δ”, the non-match with “x”, and the determination unnecessary with “?”.

  The update process content determination unit 24 determines the content of the update process when updating the dictionary database using the update analysis key according to the determination result by the conformity degree determination unit 23. The content of the update process determined here includes, for example, an addition position when an update analysis key is added to the dictionary database. Refer to FIG. 9 for the determination result by the conformity degree determination unit 23 and the processing content corresponding to the determination result. Note that FIG. 9 illustrates the analysis key update process having two pieces of attribute information, but the number of pieces of attribute information is not limited to two.

  The unit analysis key extraction unit 25 extracts one or more analysis keys serving as a reference for grasping the configuration of the analysis key from the dictionary database as unit analysis keys. In the present embodiment, the unit analysis key extraction unit 25 extracts, as a unit analysis key, an analysis key that is searched only when it is used as a search key as a result of the search by the database search unit 22. Here, the analysis key that is searched only when it is used as a search key is, in other words, an analysis key that is not searched when an analysis key other than itself is used as a search key.

  FIG. 5 is a diagram showing the configuration of the unit analysis key table according to the present embodiment. The unit analysis key extraction unit 25 extracts an analysis key serving as a reference for grasping the configuration of the analysis key from the dictionary shown in FIG. The analysis key extracted by the unit analysis key extraction unit 25 is registered as a unit analysis key in the unit analysis key table shown in FIG.

  The configuration grasping unit 26 grasps the configuration of the analysis key included in the dictionary database using the unit analysis key.

  The additional unit analysis key extraction unit 27 further extracts a character string that does not correspond to any of the unit analysis keys in the configuration grasped by the configuration grasping unit 26 as an additional unit analysis key.

  The additional unit analysis key setting unit 28 generates and sets semantic information of the additional unit analysis key based on a character string that does not correspond to any of the unit analysis keys in the configuration grasped by the configuration grasping unit 26.

The database update unit 29 has an update analysis key and information associated with the update analysis key according to the content of the update process determined by the update process content determination unit 24.
Update the dictionary database. For example, the database update unit 29 adds the analysis data including the update analysis key and the information associated with the update analysis key to the additional position determined by the update processing content determination unit 24, so that the dictionary database Update.

  Further, the database update unit 29 updates information associated with the analysis key included in the dictionary database using information associated with the unit analysis key according to the configuration grasped by the configuration grasping unit 26. For example, the database updating unit 29 uses the semantic information of the unit analysis key associated with the unit analysis key to update the semantic information of the analysis key associated with the analysis key included in the dictionary database.

  The document data analysis unit 30 analyzes the document data using the analysis key stored in the dictionary database, and acquires attribute information and semantic information related to the document data.

<Process flow>
Next, the flow of processing executed by the document data analysis apparatus 1 according to this embodiment will be described. Note that the order of processing and specific processing contents shown in the present embodiment are examples that can be adopted in carrying out the present invention, and the present invention is implemented in the actual processing order and specific processing contents. Therefore, various processing orders and specific processing contents that can be employed by those skilled in the art may be employed.

  FIG. 6 is a flowchart showing the flow of document data analysis processing according to this embodiment. The process shown in this flowchart is started when a user receives an instruction to execute a document data analysis process. However, the processing shown in this flowchart may be executed according to a preset schedule or periodically.

  In step S101, input of document data is accepted. The document data analysis unit 30 accepts input of document data via a network such as a LAN or a portable recording medium such as a USB memory or a CD-ROM. The document data to be processed in the present embodiment is, for example, a reception log in a call center, and the reception log includes text input by an operator using character codes in units of calls or a series of cases. included. Hereinafter, the data in units of calls or cases included in the reception log is referred to as “case”.

  Each case includes a case ID for identifying the case and case attribute information input by the operator. The attribute information is information for indicating the attribute of the case. For example, in order to be able to determine the meaning content of the call or case related to the case, such as “claim”, “request for opinion”, “praise”, etc. This is information set by the operator who made the call. However, the attribute information is not necessarily set for all cases, and there may be cases where the attribute information is not set in the reception log due to forgetting the setting by the operator. When the input document data is recorded in the RAM and the input acceptance is completed, the process proceeds to step S102.

  In step S102, the notation in the document is regulated. The document data analysis unit 30 controls the notation in the document by executing a search / replacement process using a notation control dictionary (not shown). The dictionary for notation control contains analysis keys that define notation fluctuations and frequent misprints as normal character strings or regular expression character strings, and the replacement character strings corresponding to these analysis keys, that is, notations. The post-control character string is included. By using such a dictionary for notation control, search / replacement processing is applied to document data, so that notation in the document is controlled and there are no notation or typographical errors (or reduced typographical or typographical errors). Document data) is generated.

  Here, the term “control” refers to standardizing expressions used in sentences under a certain standard. Specifically, correction of typographical errors, unification of words and phrases having a plurality of notation methods (for example, replacing all character strings such as “fax” and “facsimile” with “FAX”), synonym control (for example, “ (Easy, quick, and quickly) replaces the string with “immediately”), deletes prefix, corrects phrasing, unifies to key, simplifies, has meaning No end of sentence is deleted and corrected. When the notation control ends, the process proceeds to step S103.

  In step S103, search processing using a dictionary data table is performed. The document data analysis unit 30 searches the document data after the notation control generated in step S102 with the analysis key included in the dictionary data table, and searches for the searched character string (hereinafter referred to as “searched character string”). ), The case ID of the target case including the search character string, the position of the search character string in the document, the analysis key related to the search, the attribute information of the search character string, the semantic information, and the like, and the analysis result Held in the RAM. Thereafter, the processing shown in this flowchart ends.

  FIG. 7 is a diagram illustrating a result of the search process using the dictionary data table in the present embodiment. In the case of a search using a regular expression, the search is performed by finding a character string that matches the regular expression using a delimiter specified in the system as a data delimiter. The delimiter can be freely specified. For example, “.” Or a line feed can be specified as the delimiter. Further, the position of the searched character string can be specified by using the number of characters from the top of the document, the number of bytes, the number of logical lines, and the like.

  Hereinafter, database update processing and database optimization processing executed for dictionary data table maintenance used in the document data analysis processing described with reference to FIGS. 6 and 7 will be described with reference to FIGS. 8A to 11. To do.

  8A to 8C are flowcharts showing a flow of database update processing according to the present embodiment. In the processing shown in this flowchart, an update data table to be integrated into a dictionary database created by a user is input to the document data analysis apparatus 1, and an instruction to integrate the update data table into the dictionary database is issued by the user. It is started when it is accepted. However, the processing shown in this flowchart may be executed according to a preset schedule or periodically. Note that the processing shown in this flowchart is executed for each update data included in the update data table.

  Prior to the execution of the processing shown in step S201 and subsequent steps, the update analysis key acquisition unit 21 performs an update analysis key via a network such as a LAN, a portable recording medium such as a USB memory or a CD-ROM, or the like. Get the update data table containing. The update data table is a collection of update analysis data created in advance by the user. However, according to the present embodiment, in the database update process described below, since the content of the appropriate update process is determined, the user can update the database without worrying about the configuration of the dictionary database or the current contents. Data can be created.

  In step S201 and step S202, the contents of the update data are organized. Here, the update data is data used for adding data to the dictionary database or overwriting information in the dictionary database. The update analysis key acquisition unit 21 deletes data in which the analysis key included in the update data is NULL (step S201), leaving data with the analysis key completely duplicated, leaving the last added data. By deleting (step S202), the contents of the update data are organized.

At this time, in order to improve the accuracy of the search described later, a control process for organizing and standardizing the format and expression related to the update data, the data holding order, and the like according to a predetermined rule may be performed. For example, in the present embodiment, the analysis key is defined using a regular expression, but a more accurate search result is obtained by regulating the expression of the analysis key described in the regular expression according to a predetermined rule. Can be used as an analysis key. More specifically, by specifying the specified order of the strings to be combined with the OR condition according to a predetermined rule, “. * Knew (want | wanted)” and “. * Knew (wish) It is possible to clarify that “I want” is an analysis key to be matched, and to search for a correct search result by a subsequent search.

In step S203 and step S204, determination of the degree of conformity and determination of update processing contents are performed using an analysis key (hereinafter referred to as “update analysis key”) included in the update data. The conformity degree determination unit 23 uses the update analysis key to cause the database search unit 22 to search for an analysis key included in all data stored in the dictionary database, and uses the information included in the analysis data and the update data. The degree of matching with the included information is determined (step S203). Here, the update analysis key string (for example, ". * Know (I wanted to)")
The database search unit 22 searches only for an analysis key that completely matches. Here, “completely matched” means that the information to be compared is matched without excess or deficiency.

When the degree of conformity is determined by the degree of conformity determination unit 23, the update process content determination unit 24 determines the content of the dictionary database update process according to the determination result of the degree of conformity (step S204). If an analysis key that matches completely is found, the process proceeds to step S205. The process related to the analysis key that does not completely match (not match) the update analysis key proceeds to step S209. For example, if the update analysis key is “. * Know”, only the analysis key “. * Know” is searched as a completely matching analysis key. Other analysis keys (for example, ". * Description (ga | is)? ((Not | none)" or ". * Know (would have wanted) *. )? ((None | None) ") is not considered an exact match.

  In step S205 and step S206, the attribute information and the semantic information are compared. The degree-of-fit determination unit 23 determines whether or not the attribute information and semantic information included in the data having the analysis key that completely matches the update analysis key matches the attribute information and semantic information of the update data (step) S205). When the degree of conformity is determined by the degree of conformity determination unit 23, the update process content determination unit 24 determines the contents of the dictionary database update process according to the determination result of the degree of conformity (step S206). If it is determined that the attribute information and semantic information match the attribute information and semantic information of the update data, the process proceeds to step S207. On the other hand, if the attribute information and the semantic information do not match the attribute information and the semantic information of the update data (that is, at least one of the attribute information and the semantic information is different from that included in the update data), the process proceeds to step S208. Proceed (step S206).

  In step S207, the addition of update data to the dictionary database is cancelled. In step S205, the database update unit 29 discards or leaves the update data determined to have the attribute information and semantic information matching the attribute information and semantic information of the update data without adding them to the dictionary database. That is, an analysis key that is completely matched is searched from the dictionary database in step S204, and the update data determined that the attribute information and semantic information match the attribute information and semantic information of the update data in step S205 are the same. Since the data is already registered in the dictionary database, it is not added to the dictionary database. Thereafter, the processing shown in this flowchart ends.

FIG. 9 is a table showing the relationship between the determination result by the suitability degree determination unit 23 according to the present embodiment and the content of the processing corresponding to the determination result. In FIG. 9, the process executed in step S207 in response to the determination result by the suitability degree determination unit 23 ("not registering update data, that is, adding update data to the dictionary database is canceled"). In the table). It is shown in row 0. In FIG. 9, only two pieces of attribute information “attribute 1” and “attribute 2” are shown, but the number of pieces of attribute information is not limited to two. The same applies to FIG.

  FIG. 10 is a diagram illustrating a state in which each table in the dictionary database is updated using the update data included in the update data table when the database update process according to the present embodiment is executed. Referring to FIG. 10, No. in the table of FIG. It can be seen that the data of ID1 which is the update data corresponding to 0 is not registered because it completely matches the data already existing in the dictionary database.

  In step S208, the data already registered in the dictionary database is updated using the contents of the update data. The database updating unit 29 uses the attribute information and semantic information of the data determined in step S205 that the attribute information and semantic information do not match the attribute information and semantic information of the update data, and the registration retrieved in step S204. Overwrite and update the attribute information and semantic information of the completed data. However, when the attribute information and semantic information of the update data are NULL, the attribute information and semantic information of the update data are not overwritten with the attribute information and semantic information of the registered data. Thereafter, the processing shown in this flowchart ends.

  In FIG. 9, the process executed in step S208 corresponding to the determination result by the conformity degree determination unit 23 (overwriting update of registered data) is shown in No. of the table. It is shown in line 1. Referring to FIG. 10, No. in the table of FIG. It can be seen that the data of IDs 2 and 3 which are update data corresponding to 1 are overwritten on the data already registered in the dictionary database.

  In step S209 and step S210, a dictionary database search is performed using attribute information included in the update data (hereinafter referred to as “update attribute information”). Using the update attribute information included in the update data, the suitability degree determination unit 23 uses the update attribute information included in the data in the dictionary database to the database search unit 22 (however, the data included in the data in the processing in step S204 is excluded). Information is searched, and the degree of matching between the information included in the analysis data and the information included in the update data is determined (step S209). Here, a search using a character string (for example, “opinion”) of the attribute information for update is performed, and the database search unit 22 searches for data having one or more matching attribute information.

  When the degree of conformity is determined by the degree of conformity determination unit 23, the update process content determination unit 24 determines the content of the dictionary database update process according to the determination result of the degree of conformity (step S210). If data having one or more matching attribute information is found, the process proceeds to step S211. If data having attribute information that matches the update attribute information is not found, the process proceeds to step S219. For example, when the update attribute information 1 is “response” and the update attribute information 2 is “opinion”, data in which at least one of the attribute information 1 or the attribute information 2 is “response” or “opinion” Sought out. Thereafter, the process proceeds to step S211.

In step S211 and step S212, determination of the degree of conformity and determination of update processing contents are performed using the semantic information of the update data. The degree-of-fit determination unit 23 searches the database search unit 22 for the longest-matching semantic information as a character string from the data group searched for in step S209 and having one or more completely matching attribute information. The degree of matching between the information included in the analysis data and the information included in the update data is determined by searching for the data having (step S211).

  When the degree of conformity is determined by the degree of conformity determination unit 23, the update process content determination unit 24 determines the content of the dictionary database update process according to the determination result of the degree of conformity (step S212). If the semantic information of the retrieved data completely matches the semantic information of the update data, the process proceeds to step S213. If the semantic information of the retrieved data partially matches the semantic information of the update data, the process proceeds to step S214. On the other hand, if data including semantic information that matches the semantic information of the update data has not been found (does not match), the process proceeds to step S215.

  Here, partial match is when the semantic information of the retrieved data and the semantic information of the update data do not match all of the multiple semantic information, but one or more match. In addition, there is a case where the number of characters matches the predetermined lower limit length (threshold value) but is not completely matched. Inconsistency is a character string that is less than a predetermined lower limit length (threshold) in addition to the case where there is no matching semantic information between the semantic information of the retrieved data and the semantic information of the update data. Including the case where only matches.

  In step S213, the update data is added to the dictionary database. If the semantic information of the retrieved data completely matches the semantic information of the update data, the database update unit 29 searches the update data for the data whose semantic information is completely matched in step S211. Append to the end of the group. Thereafter, the processing shown in this flowchart ends.

  In FIG. 9, the process executed in step S213 corresponding to the determination result by the suitability degree determination unit 23 (the update data is added to the end of the data group whose semantic information completely matches) No. in the table. 2, 3, 8, 9, 14 and 15 are shown. Referring to FIG. 10, No. in the table of FIG. Data of IDs 4, 5, 10, 11, 16, and 17 that are update data corresponding to 2, 3, 8, 9, 14, and 15 may be added to the end of the data group whose semantic information completely matches. I understand.

  In step S214, the update data is added to the dictionary database. When the semantic information of the retrieved data partially matches the semantic information of the update data, the database update unit 29 retrieves the update data from the data whose semantic information partially matches in step S211. Append to the end of the group. Thereafter, the processing shown in this flowchart ends.

  In FIG. 9, the process executed in step S214 corresponding to the determination result by the suitability degree determination unit 23 (the update data is added to the end of the data group in which the semantic information partially matches) No. in the table. 4, 5, 10, 11, 16 and 17 are shown. Referring to FIG. 10, No. in the table of FIG. Data of IDs 6, 7, 12, 13, 18, and 19 that are update data corresponding to 4, 5, 10, 11, 16, and 17 may be added to the end of the data group that partially matches the semantic information. I understand.

  In step S215 and step S216, the “analysis key” is compared in the same attribute information. The conformity degree determination unit 23 compares the analysis key included in the data having one or more completely matched attribute information found in step S209 with the update analysis key, and is included in the analysis data. The degree of matching between the information and the information included in the update data is determined (step S215).

When the degree of conformity is determined by the degree of conformity determination unit 23, the update process content determination unit 24 determines the contents of the dictionary database update process according to the determination result of the degree of conformity (step S216). As a result of the comparison, if there is an analysis key that partially matches the update analysis key in the data retrieved in step S209, the process proceeds to step S217. On the other hand, as a result of the comparison, if there is no analysis key that matches the update analysis key in the data retrieved in step S209, the process proceeds to step S218.

  In step S217, the update data is added to the dictionary database. If there is an analysis key that partially matches the update analysis key in the data retrieved in step S209, the database update unit 29 adds the update data to the end of the data group related to the partial match (step S217). . Thereafter, the processing shown in this flowchart ends.

  In FIG. 9, the processing executed in step S217 in response to the determination result by the suitability degree determination unit 23 (the update data is added to the end of the data group related to partial matching) is shown in the table. No. 6, 12 and 18 are shown. Referring to FIG. 10, No. in the table of FIG. It can be seen that the data of IDs 8, 14 and 20 which are update data corresponding to 6, 12 and 18 are added to the end of the data group in which the update analysis keys in the dictionary database partially match.

  In step S218, the update data is added to the dictionary database. If the data retrieved in step S209 does not have an analysis key that matches the update analysis key, the database update unit 29 adds the update data to the end of the data group having the same attribute information (step S218). ). Thereafter, the processing shown in this flowchart ends.

  In FIG. 9, the process executed in step S218 corresponding to the determination result by the suitability degree determination unit 23 (the update data is added to the end of the data group having the same attribute information) No. in the table. 7, 13 and 19 are shown. Referring to FIG. 10, No. in the table of FIG. It can be seen that the data of IDs 9, 15 and 21 which are update data corresponding to 7, 13, and 19 are added to the end of the data group having the same attribute information in the dictionary database.

  In steps S219 and S220, determination of the degree of conformity and determination of update processing contents are performed using semantic information included in the update data (hereinafter referred to as “update semantic information”). The conformity degree determination unit 23 uses the update semantic information included in the update data, and the database search unit 22 includes the data in the dictionary database (however, excluding the data that completely matches in the process of step S204). The semantic information is searched, and the degree of matching between the information included in the analysis data and the information included in the update data is determined (step S219). Here, a search using a character string (for example, “I want to know”) of the semantic information for update is performed.

  When the degree of conformity is determined by the degree of conformity determination unit 23, the update process content determination unit 24 determines the content of the dictionary database update process according to the determination result of the degree of conformity (step S220). If the semantic information of the retrieved data completely matches the semantic information of the update data, the process proceeds to step S221. If the semantic information of the retrieved data partially matches the semantic information of the update data, the process proceeds to step S222. On the other hand, when data including semantic information that matches the semantic information of the update data is not found (non-coincidence), the process proceeds to step S223.

  In steps S221 to S223, the update data is added to the dictionary database. Note that the contents of the processes relating to step S221 and step S222 are substantially the same as those of step S213 and step S214, and thus the description thereof is omitted.

In FIG. 9, the process executed in step S <b> 221 corresponding to the determination result by the suitability degree determination unit 23 (update data is added to the end of the data group whose semantic information completely matches) No. in the table. 20 and 21, and the processing executed in step S 222 corresponding to the determination result by the conformity degree determination unit 23 (update data is added to the end of the data group whose semantic information partially matches. In the table). It is shown in lines 22 and 23. Referring to FIG. 10, No. in the table of FIG. The data of IDs 22 and 23 which are update data corresponding to 20 and 21 are added to the end of the data group in which the semantic information completely matches, and No. in the table of FIG. It can be seen that the data of IDs 24 and 25, which are update data corresponding to 22 and 23, are added to the end of the data group in which the semantic information partially matches.

  In step S223, the update data is added to the dictionary database. When the semantic information of the retrieved data does not match the semantic information of the update data, the database update unit 29 adds the update data to the end of the last table in the dictionary database (step S223). . That is, in the present embodiment, the last table in the dictionary database is used as a table for storing data including analysis keys that cannot be classified.

  In FIG. 9, the process executed in step S223 corresponding to the determination result by the suitability degree determination unit 23 (adding update data to the end of the last table) It is shown in lines 24 and 25. Referring to FIG. 10, No. in the table of FIG. It can be seen that the data of IDs 26 and 27, which are update data corresponding to 24 and 25, are added to the end of the last table in the dictionary database.

  According to the document data analysis apparatus 1 according to the present embodiment, the position related to addition / update of dictionary data is determined based on the pattern match between the information included in the update data and the content included in the dictionary database to be updated. You can identify and update the dictionary database. In this embodiment, the contents of the update process are determined by executing the processes shown in the flowcharts of FIGS. 8A to 8C. Instead of such a method, attribute information, semantic information, and analysis are performed. A method of determining the content of the update process by determining the degree of conformity for the key and referring to a table as shown in FIG. 9 may be adopted.

  FIG. 11 is a flowchart showing the flow of database optimization processing according to this embodiment. The process shown in this flowchart is started when a user receives an instruction to execute a database optimization process. However, the processing shown in this flowchart may be executed according to a preset schedule or periodically.

  In step S301, the dictionary database is searched using the analysis key included in the dictionary database. The database search unit 22 searches the dictionary database using the analysis keys stored in the dictionary database, and thereby includes an analysis key including an analysis key used for the search (hereinafter also referred to as “search analysis key”). Extract. The database search unit 22 repeats selection and search of search analysis keys for all analysis keys included in the dictionary database. For this reason, the database search unit 22 obtains a dictionary database search result for each analysis key for all analysis keys included in the dictionary database as a result of the processing in step S301. Thereafter, the process proceeds to step S302.

In step S302, a unit analysis key is extracted. The unit analysis key extraction unit 25 extracts one or a plurality of analysis keys searched out only when using the search key as a result of the search in step S301 as a unit analysis key. Here, the analysis key that is searched only when it is used as a search key is, in other words, an analysis key that is not searched when an analysis key other than itself is used as a search key. The unit analysis key is an analysis key serving as a reference for grasping the configuration of the analysis key, and does not include other analysis keys, and therefore can be used as a minimum unit for grasping the configuration of the analysis key. The extracted unit analysis keys are collected in a table, and a unit analysis key table (unit analysis key dictionary, minimum unit dictionary) is generated.

In step S303, the configuration of the analysis key is grasped based on the unit analysis key. The configuration grasping unit 26 searches the dictionary database using the unit analysis key, and grasps the configuration of the analysis key included in the dictionary database. In other words, the configuration grasping unit 26 grasps the usage pattern of the unit analysis key for each analysis key included in the dictionary database. Here, the usage pattern is information indicating the unit analysis key match status (complete match, partial match, and partially matched location) in the analysis key. For example, the analysis key is ". * Know (I wanted to). * (
In the case where the description is “(||)” ((not | none)], the configuration grasping unit 26 performs the analysis using the unit analysis key, and the analysis key “. . * (But | but). * Explanation (but | is)? ((Not | none)] is the unit analysis key ". * Know (I wanted to)", unit analysis key ". * (Is | ) ”And the unit analysis key“. * Description (but | is)? ((Not | none) ”]. Then, the process proceeds to step S304.

  In step S304, an additional unit analysis key is extracted and set. The additional unit analysis key extraction unit 27 extracts a character string that does not correspond to any of the unit analysis keys included in the unit analysis key table generated in step S302 as an additional unit analysis key. Here, the additional unit analysis key extraction unit 27 refers to the analysis key configuration grasped in step S303 in order to extract such an analysis key. For example, the analysis key is ``. * Know (I wanted to). * (Called). * Explanation (but | is)? ) ”And the unit analysis key“. * Description (but | is)? (Not | none) ”exist in the unit analysis key table, but the unit analysis key“. * (Called) ”does not exist, The additional unit analysis key extraction unit 27 newly extracts “. * (Called)” as an additional unit analysis key.

  Then, the additional unit analysis key setting unit 28 generates semantic information from a character string that does not correspond to any of the unit analysis keys in the analysis key, and sets this as semantic information of the newly extracted additional unit analysis key. To do. For example, if the character string that does not correspond to any of the unit analysis keys in the analysis key is ``. * (Called) '', the character excluding the expression for regular expression is used as the semantic information of the additional unit analysis key The column “Called” is generated and set.

  However, the character string that does not correspond to any of the unit analysis keys in the analysis key is a character string that has no meaning as the character string itself (no direct meaning) such as the conjunctions “ga” and “ha”. There is a case. In this embodiment, in such a case, the additional unit analysis key extraction unit 27 does not extract such a character string as an additional unit analysis key in order to use the unit analysis key as a unit of information description (minimum unit). .

In addition, when the use frequency of the unit analysis key included in the analysis key is lower than a predetermined threshold, such an entire analysis key may be extracted as the additional unit analysis key. In this case, the additional unit analysis key extraction unit 27 extracts an analysis key including a character string that does not correspond to any of the unit analysis keys included in the unit analysis key table generated in step S302 as an additional unit analysis key. For example, the analysis key is ``. * Know (I wanted to). ) "And the unit analysis key". * Description (but | is)? ((Not | none) "exist in the unit analysis key table, but". * (
When there is no unit analysis key such as “.”, “. *” Is a character string that has no meaning as the character string itself. However, if the usage frequency of the unit analysis key ". * Know (I want to)" and the unit analysis key ". * Description (is | is)? ((None | None)" is lower than the predetermined threshold, it is added The unit analysis key extraction unit 27 reads “. * Know (I want to know).
"None | None)" may be newly extracted as an additional unit analysis key.

When an analysis key including a character string that does not correspond to any of the unit analysis keys is extracted as an additional unit analysis key, the additional unit analysis key setting unit 28 uses the newly extracted additional unit analysis key as semantic information. The semantic information of other unit analysis keys included in the additional unit analysis key is set. For example, the additional unit analysis key is ``. * Know (I wanted to). If you include `` Know (Tai | Takata) '' and the unit analysis key ``. * Description (but | is)? ((Not | None) '', the semantic information of these unit analysis keys as additional unit analysis key semantic information “I want to know” and “No explanation” are set.

In step S305, the dictionary database is updated. The database update unit 29 updates the semantic information of the analysis key included in the dictionary database using the semantic information associated with the unit analysis key included in the configuration grasped in step S303. For example, the analysis key is “. * Know (would have wanted). When the analysis key ". * Description (ga | ha)? ((None | None)" is included, the semantic information of these unit analysis keys "I want to know" and "No description" are added as the semantic information of the analysis key After that, the process shown in this flowchart ends.

  According to the document data analysis apparatus 1 according to the present embodiment, a unit analysis key that is a unit of information description (minimum unit) and its semantic information are acquired, and the analysis key is expressed by the semantic information of the unit analysis key and structured. By optimizing, dictionary optimization based on semantic information is performed. For this reason, according to the document data analysis apparatus 1 according to the present embodiment, it is possible to construct a regular expression dictionary optimized from the viewpoint of semantic structure, and it is possible to improve the work related to dictionary maintenance. Further, according to the document data analysis apparatus 1 according to the present embodiment, the dictionary administrator can easily understand the meaning of the dictionary contents in the work related to the addition, correction, and update of the registered contents of the regular expression dictionary.

DESCRIPTION OF SYMBOLS 1 Document data analysis apparatus 23 Conformity degree determination part 24 Update process content determination part 25 Unit analysis key extraction part 26 Structure grasp part 29 Database update part 30 Document data analysis part

Claims (9)

An information processing apparatus connected to a database in which analysis data including an analysis key for analyzing document data is stored,
Unit analysis key extraction means for extracting one or more analysis keys serving as a reference for grasping the configuration of the analysis key from the database as unit analysis keys;
Using the unit analysis key, configuration grasping means for grasping the configuration of the analysis key included in the database;
Database updating means for updating information associated with the analysis key included in the database using information associated with the unit analysis key according to the structure grasped by the structure grasping means;
An information processing apparatus comprising: The unit analysis key extracting means extracts, from the database, an analysis key that is searched only when using itself as a search key, as the unit analysis key.
The information processing apparatus according to claim 1. Further comprising database search means for searching the database using the analysis key stored in the database;
The unit analysis key extracting means extracts, as a unit analysis key, an analysis key that is searched only when using the search key by itself as a result of the search by the database search means.
The information processing apparatus according to claim 2. In the configuration ascertained by the configuration grasping means, further comprising an additional unit analysis key extracting means for further extracting a character string that does not correspond to any of the unit analysis keys as an additional unit analysis key.
The information processing apparatus according to any one of claims 1 to 3. The system further comprises additional unit analysis key setting means for setting semantic information generated based on a character string that does not correspond to any of the unit analysis keys as the semantic information of the additional unit analysis key.
The information processing apparatus according to claim 4. The database update means updates the semantic information of the analysis key associated with the analysis key included in the database using the semantic information of the unit analysis key associated with the unit analysis key.
The information processing apparatus according to any one of claims 1 to 5.   The information processing apparatus according to claim 1, wherein the analysis key is defined using a regular expression. A computer connected to a database in which analysis data including an analysis key for analyzing document data is stored,
A unit analysis key extraction step of extracting one or a plurality of analysis keys serving as a reference for grasping the configuration of the analysis key from the database as a unit analysis key;
Using the unit analysis key, a configuration grasping step for grasping the configuration of the analysis key included in the database,
A database update step for updating information associated with an analysis key included in the database using information associated with the unit analysis key according to the configuration grasped in the configuration grasping step;
To update the database. To a computer connected to a database that stores analysis data including analysis keys for analyzing document data,
A unit analysis key extraction step of extracting one or a plurality of analysis keys serving as a reference for grasping the configuration of the analysis key from the database as a unit analysis key;
Using the unit analysis key, a configuration grasping step for grasping the configuration of the analysis key included in the database,
A database update step for updating information associated with an analysis key included in the database using information associated with the unit analysis key according to the configuration grasped in the configuration grasping step;
Database update program to execute

Images