JP2002230015A - Data analyzing method and its device and computer program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a data analyzing method capable of automatically analyzing data to be analyzed by reusing a certain analytic result including in an event data base. SOLUTION: Data 100 to be analyzed constituted of the retrieval request and retrieval result of a data base are read (a step S3001). Then, event data similar to the data 100 to be analyzed are retrieved in an event data base constituted of the retrieval request, retrieval result, and analytic result of the data base (step S3002). When any similar data are present (a step S3003: YES route), analytic result data 500 are prepared by reusing the analytic result in the similar event data (a step S3004).

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a data analysis method and apparatus, and a computer program. The present invention relates to a data analysis method and apparatus for analyzing data, and a computer program.

[0002]

2. Description of the Related Art In an information search system that searches a database in response to a data search request and outputs the search result, it is necessary to analyze the validity, validity, and the like of the data of the search result corresponding to the search request. It is extremely important for the improvement of information retrieval system processing performance and retrieval quality. In order to reduce the work labor and time burden associated with the data analysis processing by humans, a program that operates on a computer and supports using a GUI has been developed.

[0003] Hereinafter, conventional data analysis and support for data analysis will be described by taking as an example a case where a question is received from a client and a search result of an information search system that answers the question is analyzed. Such an information search system is provided with a set (database) of data in which a plurality of questions received from a client or a plurality of questions to be received and an answer to the question are stored in association with each other. . Then, when a question (search request) is received from the client, an answer corresponding to the question is searched from the database, and the answer (search result) is transmitted to the client. Information about the question and answer processing is accumulated and recorded in log data (case data) in the information search system.

[0004] An operator of the information search system analyzes the search results in the information search system by referring to the contents described in the log data. In a conventional data analysis support program, analysis processing using log data by an operator is supported by devising a GUI.

FIG. 1 shows an example of a main screen displayed on a display when a data analysis support program is started on a conventional information retrieval system. The main screen shown in FIG. 1 has a reception number 10 for identifying a question.
1. Information including an acceptance time 102, a question 103 received from a client, an area 104 to be pressed to display detailed contents of a question and an answer, and an analysis result 105 including an area 106 to be pressed to analyze a search result. Is displayed in a list format.

[0006] When the operator uses a mouse or the like, area 1
Clicking on 06 displays the search result analysis screen shown in FIG. The search result analysis screen displays a question 201 and an answer 20 extracted from the database for the question.
2 is displayed. The operator determines whether the question displayed on this screen is a valid question, an invalid question, or whether an error has occurred when receiving the question, and selects the corresponding option button 203. Further, it is determined whether or not the content of the answer is a correct answer to the question, and a corresponding option button is selected.

[0007] The operator proceeds with the above-described analysis on the search result including the question and the answer. As described above, in the analysis of the search result in the conventional information search system, a human analyzes each one of all data to be analyzed.

[0008]

However, in the data analysis using the conventional data analysis support program,
Humans must analyze all search results, which is time and labor intensive. For this reason, there has been a problem that the probability of erroneous analysis due to human error is high.

In addition, since human judgment is ambiguous, the conventional data analysis using a data analysis support program has a problem that the results of data analysis are not consistent and the quality of the analysis results is not always high. there were.

The present invention has been made in view of such a problem, and an object of the present invention is to reuse data of analysis in a case database and automatically analyze data to be analyzed. An object of the present invention is to provide an analysis method, a device therefor, and a computer program.

[0011]

According to one aspect of the present invention, there is provided a data analysis method for a data analysis apparatus for analyzing a result of a database search, wherein the data analysis method includes the steps of: For the new case, a past case of the database search stored in the storage unit is searched, and as a result of the search, an analysis result of the past case determined to be similar is read out from the storage unit and the new case is read. It is characterized in that it is output as an analysis result of a case.

The invention described in claim 2 is the first invention.
The data analysis method according to the above, wherein an analysis result responding to the new case is input from the outside in accordance with a result of the search of the past case.

[0013] The invention described in claim 3 is the first invention.
Or the data analysis method according to item 2, wherein the past case and the new case include a search request for the database and a search result of the database performed in response to the search request.

The invention described in claim 4 is the third invention.
Wherein the similar past case is a case in which a search request matches or is similar to the search request of the new case, and a search result matches or is similar to the search result of the new case. It is characterized by the following.

The invention described in claim 5 is the invention according to claim 4.
Wherein the search request is made using a sentence.

The invention described in claim 6 is the same as the invention in claim 5
, At least a part of the past case is in the form of a vector expression created based on a plurality of index words and the search request.

The invention described in claim 7 is the same as the invention in claim 6
In the data analysis method according to the above, when searching the storage unit, a vector expression of the new case is created based on the plurality of index words, and the created vector expression and the created vector expression are stored in the storage unit. The similarity between the past case and the new case is determined based on the inner product of the past case and the vector expression.

[0018] The invention according to claim 8 is a data analysis device for analyzing a result of a database search, wherein the storage unit stores a past case of the database search and an analysis result of the past case. For a new case of the database search, a search unit for searching past cases stored in the storage unit and an analysis result of a past case determined to be similar as a result of the search are read out from the storage unit. Output means for outputting the analysis result of the new case.

The present invention according to a ninth aspect is a data analysis program for analyzing a result of a database search, wherein a new case of the database search is stored in a storage means in a past of the database search. A case is searched, and as a result of the search, an analysis result of a past case determined to be similar is read out from the storage unit and output as an analysis result of the new case.

[0020]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below in detail by taking as an example a case where a question is received from a client and a search result of an information search system that answers the question is analyzed.

FIG. 3 shows a data format used in the data analysis method according to the present embodiment.

The case data 401 is data relating to past cases, and is used when automatically analyzing data to be analyzed, which constitutes a case database in a data analysis device described later. This data includes a search request 501 and a search result 5
02 and an analysis result 503.

The search request 501 is composed of a question document and a vector expression. In the vector expression, the value is updated each time new case data is added to the case database. The search result 502 includes a data number for identifying an answer in the database and a document of the answer extracted as the search result. Analysis result 503
Is a document indicating a valid question, an invalid question, a correct answer, an incorrect answer, or the like.

The example of the above case data is shown in FIG.
(A) to (c).

The analysis target data 100 shown in FIG. 3 is data relating to a new case input into the data analysis device.
It is composed of a search request 601 and a search result 602. As a method of inputting data into the data analyzer, a method of reading data recorded in a memory, a method of receiving data from outside via a communication line (not shown), or the like can be used.

The analysis result data 500 in FIG. 3 is data obtained by adding an analysis result to the analysis target data 100, and includes a search request 901, a search result 902, and an analysis result 903.
Consists of

Next, an example of a hardware configuration of a data analysis apparatus to which the present invention is applied will be described with reference to FIG.
The data analysis device 600 includes a central processing unit (CPU) 10 that controls the entire device, a memory 20 that provides a storage area for a computer program and temporary data necessary for execution of the computer program, A keyboard 30 and a mouse 40 for inputting data, and a display 5 for generating a display screen
0. A computer program for executing the processing according to the present invention is stored in the memory 20.

By adopting such a configuration, it is possible to give an instruction from the keyboard 30 or the mouse 40 for designating a cursor to a predetermined position on the display 50 to the CPU 10. Is performed. In the example shown in FIG. 5, the data analyzer 600 has a stand-alone configuration. However, the data analyzer 600 may have a configuration incorporated in another system.

Next, the function of the data analyzer according to the present embodiment will be described with reference to the block diagram of FIG.

The data analysis apparatus 600 is an analysis support unit 200 that supports an operator to analyze data by using a GUI by a browser corresponding to Java, which is an object-oriented programming language, and calculates case data similar to the data 100 to be analyzed. Similar case calculation unit 300 and case database 40 in which a plurality of case data are registered
0.

The analysis support unit 200 includes a data display unit 210 for displaying a screen necessary for supporting data analysis by an operator on the display 50, and an analysis input unit 220 for inputting an analysis result from the keyboard 30 or the mouse 40. And In the case database 400, a plurality of the above-described case data are recorded.

The similar case calculation unit 300 includes a data acquisition unit 310 for acquiring the analysis target data 100, an automatic analysis determination unit 320 for automatically performing data analysis, and an analysis support unit 20.
It has a linking unit 330 that performs processing linked to 0, a reuse unit 340 for reusing the analysis result of the case data, and a data output unit 350 that outputs the analysis result data 500.

The analysis automatic determination unit 320
21, a database reading unit 322, and a similarity determination unit 323.

The vectorization unit 321 performs a method of creating an index word based on a search request included in the case data in the case database 400 and vectorizes the search request.
More specifically, the search request sentence included in all the case data is sent to “Software Science Vol.15 Natural Language Processing” (Makoto Nagao, April 26, 1996, Iwanami Lecture) pp117-13
An index word is created by performing morphological analysis as described in 0. Then, for a search request to be vectorized, a vector having the same dimension as the number of index words is created.

FIG. 7 is a diagram showing an example of a vector representation of a search request. For example, in the question "What are the characteristics of XX University of Technology?"
“0” is assigned to an index word that is not included. By such an operation, a vector "1111000000010" is created for the search request based on this question.

The database reading unit 322 reads the case data in the case database 400. The similarity determination unit 323 calculates the read case data and the analysis target data 1
The similarity with 00 is determined. The details of this determination method will be described later.

The interlocking section 330 includes a data transfer section 331, a result fetching section 332, a database registration section 333, and a vectorization section 334. Data transfer unit 331
Receives the analysis target data 100 from the data acquisition unit 310 and passes it to the analysis support unit 200. The result capturing unit 332
The analysis result input from the analysis input unit 220 is captured. The database registration unit 333 stores the analysis result captured by the result capturing unit 332 in the case database 400.
Register to The vectorization unit 334 vectorizes the search request included in the case data in the case database 400.

The reuse unit 340 is provided in the case database 40.
The analysis result of the analysis target data 100 is determined using the case data in 0.

These functions are realized by the CPU 10 executing instructions of a data analysis program implemented in Java.

Next, a data analysis method by the data analysis device configured as described above will be described with reference to the flowchart of FIG. The similar case calculation unit 300 reads the analysis target data 100 (step S3001),
The analysis target data 100 in the case database 400
Search for data that matches or is similar to (step S3
002). If there is matching or similar data (step S3003: YES route), analysis results in the matching or similar case data are reused to create analysis result data 500 (step S3004), and the created analysis is performed. The result data 500 is output to the memory 20 or the like (step S3005).

On the other hand, when there is no matching or similar case data in the case database 400 (step S300)
3: No route), as in the conventional method, the analysis processing of the analysis target data 500 by the operator is supported, and the input of the analysis result is received (step S3006). And
The analysis result input by the operator is added to the analysis target data 500 to create the analysis result data 500 (step S3007), and the analysis result data 500 is stored in the memory 20.
And the like (step S3005).

Next, a detailed processing procedure of the similar case calculation unit 300 will be described with reference to the flowchart of FIG. The analysis target data 100 is acquired (step S600).
1), a search request included in the analysis target data 100 is vectorized (step S6002). Next, it is determined whether or not there is case data that has not been read in the case database 400 (step S6003).
If there is, case data is read from the case database 400 (step S6004). Then, the inner product value of the vectorized search request of the analysis target data 100 and the case data is calculated.

The calculation of the inner product value is specifically performed as follows. The number of index words (vector dimension) included in the search request is m, and the vector x of the search request in the case data and the vector y of the search request in the analysis target data are x = (x ₁ , x _{2,... .,} x _m ) Let y = (y ₁ , y _{2, ...,} y _m ). Similarity sim (x, y) between vector x and vector y
Is

[0044]

(Equation 1)

Is represented by

The inner product value is obtained from this equation, and it is determined whether or not the following two equations are satisfied (step S600)
5).

Inner product value of vectorized search request of analysis target data and case data> threshold (2) Search result of analysis target data = search result of case data (3)

In this embodiment, an example in which the search result of a new case matches the search result of a past case is described as Expression (3). It may be a condition that the search result of the case is similar.

Of the case data satisfying these two equations, the analysis result having the highest similarity is determined to be the analysis result of the analysis target data 100 (step S6011), and the analysis target data 100 and the analysis result are analyzed by the analysis result data 500. (Step S6010). On the other hand, when the above equations (2) and (3) are not satisfied, the processing is repeated from the determination processing in step S6003.

If it is determined in step S6003 that there is no unread data in the case database 400, the data to be analyzed 100 is transferred to the analysis support unit 20.
0 (step S6006), and the analysis support input unit 2
20 is obtained (step S).
6007). The analysis target data 100 and the obtained analysis result are registered in the case database 400 (step S6).
008) After the search requests for all the case data in the case database 400 are vectorized (step S600)
9) Output the analysis target data 100 and the analysis result to the analysis result data 500 (step S6010).

FIG. 10 shows an example of the analysis target data 100 input to the data analysis device 600. FIG.
Creates an index term based on the case data shown in FIG.
11 shows a vector representation when the analysis target data shown in FIG. 10 is vectorized. The similarity between the search request shown in FIG. 4 and the search request shown in FIG. 10 is calculated based on the vectors shown in FIG. 7 and FIG. When the similarity is calculated, the search request in FIG. 10A is most similar to the search request in FIG. FIG.
The search request shown in FIG. 4B is most similar to the search request shown in FIG.

Therefore, if the similarity threshold is set to 0.7, the data to be analyzed in FIG.
The analysis result of FIG. 4A can be reused. On the other hand, FIG.
For the analysis target data of (b), the analysis result of the case data cannot be reused, so that automatic data analysis is not performed. In this case, the operator analyzes the analysis target data in FIG. The data analysis device 600 includes:
When the analysis is performed by the operator, the above-described conventional data analysis support is performed.

FIG. 12 shows an example of the analysis result data 500 output by the data analysis device.
The analysis result data for the data to be analyzed in (a) is
FIG. 11B is a diagram illustrating analysis result data for the analysis target data in FIG. Analysis result 90 of FIG.
3 is a value in which the analysis result of FIG. 4A is automatically reused. On the other hand, FIG.
The result of the actual analysis performed by the operator based on the data analysis support performed by the operator is entered.

Next, a main screen displayed on the display 50 during execution of the above-described data analysis method will be described with reference to FIGS.

FIG. 13 shows an example of a main screen displayed on the display of the data analyzer when the data analysis program is started. On the screen shown in FIG. 13, the user operates the area 1201 using the mouse 40 when browsing the details of the automatic analysis in each information search. FIG. 14 shows an example of a screen displayed by this operation.

FIG. 14 shows an example of a screen for browsing the result of the automatic analysis executed by the data analysis device. In the example shown in FIG. 14, the current log and the case data reused for data analysis are displayed side by side.

Although the preferred embodiment of the present invention has been described above, it is needless to say that the present invention is not limited to this embodiment and can be implemented in other various modes. For example, the search request and search results included in the analysis target data are not limited to the question and the answer to the question, but what kind of contents are related to the database search request and the search result corresponding to the request. It may be.

The contents of the data are not limited to sentences,
It may be image data, audio data, or data composed of a combination of these.

[0059]

As described above, according to the present invention,
It is possible to reduce the burden of data analysis of the information retrieval system that has conventionally been performed by humans. As a result, the probability of erroneous analysis due to human error is reduced,
Consistent results can be obtained.

[Brief description of the drawings]

FIG. 1 is a diagram showing an example of a main screen displayed on a display when a data analysis support program is started on a conventional information search system.

FIG. 2 is a diagram showing an example of a search result analysis screen displayed by a conventional data analysis support program.

FIG. 3 is a diagram showing a format of data used in the data analysis support method according to the embodiment.

FIG. 4 is a diagram showing an example of case data.

FIG. 5 is a block diagram illustrating an example of a hardware configuration of the data analysis device according to the embodiment.

FIG. 6 is a block diagram illustrating functions of the data analysis support system according to the present embodiment.

FIG. 7 is a diagram showing an example of a vector expression of a search request included in case data.

FIG. 8 is a flowchart showing a flow of a data analysis support method according to the embodiment.

FIG. 9 is a flowchart showing a detailed processing flow of a similar case calculation unit.

FIG. 10 is a diagram showing an example of analysis target data.

FIG. 11 is a diagram showing a vector expression when vectorization is performed on the analysis target data of FIG. 10;

FIG. 12 is a diagram illustrating an example of analysis result data corresponding to the analysis target data of FIG. 10;

FIG. 13 is a diagram illustrating an example of a main screen displayed on a display of the data analysis device when a data analysis program is activated.

FIG. 14 is a diagram showing an example of a screen for browsing the results of automatic analysis.

[Explanation of symbols]

 Reference Signs List 10 CPU 20 memory 30 keyboard 40 mouse 50 display 100 analysis target data 101 reception number 102 reception time 103 question 104 area 105 analysis result 106 area 200 analysis support unit 201 question 202 answer 203 option button 210 data display unit 220 analysis input unit 300 similar Case calculation unit 310 Data acquisition unit 320 Automatic analysis determination unit 321 Vectorization unit 322 Database reading unit 323 Similarity determination unit 330 Interlocking unit 331 Data transfer unit 332 Result capture unit 333 Database registration unit 334 Vectorization unit 340 Reuse unit 350 Data output unit 400 Case database 401 Case data 500 Analysis result data 501 Search request 502 Search result 503 Analysis result 601 Search request 602 Search result 01 search request 902 Results 903 analysis result 1201 Area

 ──────────────────────────────────────────────────続 き Continued on the front page F term (reference) 5B075 ND20 NK02 PP02 PP03 PQ02 PR03 PR06 QM08 QP01

Claims (9)

[Claims] 1. A data analysis method for a data analysis device for analyzing a result of a database search, wherein a past case of a database search stored in a storage unit is searched for a new case of the database search, As a result of the search, an analysis result of a past case determined to be similar is read from the storage unit and output as an analysis result of the new case. 2. The data analysis method according to claim 1, wherein an analysis result responding to the new case is input from the outside in accordance with a search result of the past case. 3. The method according to claim 1, wherein the past case and the new case include a search request for the database and a search result of the database performed in response to the search request. Data analysis method described. 4. The similar past case, wherein a search request is a case that matches or is similar to the search request of the new case, and a search result is a case that matches or is similar to the search result of the new case. The data analysis method according to claim 3, wherein 5. The data analysis method according to claim 4, wherein the search request is made using a sentence. 6. The data analysis method according to claim 5, wherein at least a part of the past case is in the form of a vector expression created based on a plurality of index words and the search request. . 7. When searching the storage unit, a vector expression of the new case is created based on the plurality of index terms, and the created vector expression and a past case stored in the storage unit are created. 7. The data analysis method according to claim 6, wherein a similarity between a past case and a new case is determined based on an inner product of the vector expression and the new case. 8. A data analyzing apparatus for analyzing a result of a database search, comprising: storage means for storing past cases of database search and analysis results of the past cases; A search unit that searches for past cases stored in the storage unit, and a result of the search, an analysis result of a past case determined to be similar is read out from the storage unit, and is read as an analysis result of the new case. A data analysis device comprising: an output unit that outputs the data. 9. A data analysis program for analyzing a result of a database search, wherein for a new case of the database search, a past case of a database search stored in a storage unit is searched. A computer program for reading an analysis result of a past case determined to be similar from the storage means and outputting the result as an analysis result of the new case.