JP6885211B2 - Information analyzer, information analysis method and information analysis program - Google Patents

Description

This case relates to information analyzers, information analysis methods and information analysis programs.

A technique for performing clustering in text analysis is disclosed (see, for example, Patent Documents 1 and 2). In text analysis, for example, the text information to be analyzed is divided into elements such as names and causes, morphological analysis and feature word extraction are performed for each element, and then the feature words included have similar contents. The elements in the text are grouped into the same cluster.

Japanese Unexamined Patent Publication No. 2013-065097 International Publication No. 2016/147219

When clustering is performed, a cluster number is assigned to each cluster for convenience. However, the order and size of the numbers are meaningless, and there is no semantic correlation between clusters with adjacent numbers. Therefore, even if a diagram is created and visualized by path analysis based on this cluster information, the path becomes complicated and the diagram becomes difficult to understand intuitively. Therefore, additional measures such as limiting the analysis target by filtering or the like are required.

As one aspect, an object of the present invention is to provide an information analyzer, an information analysis method, and an information analysis program that can impart a semantic correlation between adjacent clusters.

In one embodiment, the information analyzer extracts a plurality of factors indicating characteristic words for a plurality of elements from the same text sample, and consists of an order scale for one or more of the plurality of elements. the target variable extraction unit to impart target variable, using the target variable, by performing multidimensional correspondence analysis for said plurality of elements, and cluster classification unit to perform the cluster classification, by the cluster classification unit, the multidimensional A tentative identifier is assigned to each cluster formed after the correspondence analysis, and the average value of the order scale of all text samples is calculated for each tentative identifier for each element other than the element to which the target variable is assigned. A sorting unit that sorts the identifiers of the clusters in ascending order of the average value and assigns a new identifier number is provided.

It is possible to give a semantic correlation between adjacent clusters.

(A) and (b) are diagrams illustrating path analysis. (A) is a block diagram for explaining the hardware configuration of the information analyzer according to the first embodiment, and (b) is a functional block diagram of each part. It is a flowchart which shows an example of the information analysis processing by an information analyzer. It is a figure which exemplifies the cluster classification result when "stop time" is set as a target variable for "cause". (A) to (c) are diagrams illustrating the process of calculating the main cluster number from the temporary cluster number. (A) shows a list of "temporary cluster number", "target variable average value", and "main cluster number" in "name", and (b) shows "temporary cluster number", "target variable average value" in "cause". , "This cluster number" is shown, and (c) shows a list of "temporary cluster number", "target variable average value", and "this cluster number" in "correspondence method". It is a figure which illustrates the created diagram. It is a diagram obtained as a result of performing cluster classification without using "stop time" as a target variable (comparative example). (A) is a diagram illustrating the first index, and (b) is a diagram illustrating the second index. (A) shows a list of "temporary cluster number", "target variable average value", and "main cluster number" in "name", and (b) shows "temporary cluster number", "target variable average value" in "cause". , "This cluster number" is shown, and (c) shows a list of "temporary cluster number", "target variable average value", and "this cluster number" in "correspondence method". It is a figure which illustrates the created diagram.

Prior to the description of the embodiment, the outline of the path analysis will be described. 1 (a) and 1 (b) are diagrams illustrating path analysis. In the path analysis, first, a plurality of factors meaning feature words are extracted for each element from the same text sample including text information. After that, the same processing is performed for all text samples, factors are extracted for all elements of all text samples, and then a plurality of factors extracted from each text sample are extracted for each element by text clustering. Classify into clusters. Specifically, for element A, each factor of text information is classified into cluster A1, cluster A2, cluster A3, .... Here, these cluster numbers are numbers assigned for convenience, but each factor assigned within each cluster consists of characteristic word information similar to each other. In each cluster of FIGS. 1 (a) and 1 (b), the number of the sample in which the corresponding factor is described is described. For example, the text sample No. 1 is classified into cluster A1 of element A.

First, sort by element in ascending order of cluster number. For example, the element A is arranged in the order of cluster A1, cluster A2, cluster A3, and so on. Next, for each sample, a diagram is created by connecting the factor of element A, the factor of element B, the factor of element C, ..., With a line (path). Thereby, the connection between the elements included in each text sample can be visualized by the cluster number meaning the coordinates.

However, in the order of the classified clusters (cluster A1, cluster A2, cluster A3, ...), Adjacent clusters do not have a correlation. In this case, as illustrated in FIG. 1A, the created diagram has a complicated path. For example, sample No. 1 passes from cluster A1 of element A to cluster B2 of element B to cluster C4 of element C. Sample No. At x, the cluster A2 of the element A passes through the cluster B4 of the element B and reaches the cluster C1 of the element C. In this way, if the classified clusters do not have a correlation, the path connecting the elements (horizontal direction) will move significantly within the element (vertical direction). As a result, a diagram that is difficult to understand intuitively is created. Therefore, in order to obtain effective knowledge from the path analysis results, additional data processing such as data filtering is required.

On the other hand, FIG. 1B is an example in which an order scale is given to each cluster and rearranged. In this example, adjacent clusters will have a correlation. In this example, if the factors are connected by a path for each sample, the amount of movement within the elements (vertical direction) becomes small when connecting the elements (horizontal direction). This will create an intuitively easy-to-understand diagram. In the following examples, an information analyzer, an information analysis method, and an information analysis program that can create intuitive path analysis results by making it possible to give a semantic correlation between adjacent clusters. explain.

FIG. 2A is a block diagram for explaining the hardware configuration of the information analyzer 100 according to the first embodiment. As illustrated in FIG. 2A, the information analysis device 100 includes a CPU 101, a RAM 102, a storage device 103, an input device 104, a display device 105, and the like. Each of these devices is connected by a bus or the like. The CPU (Central Processing Unit) 101 is a central processing unit. The CPU 101 includes one or more cores. The RAM (Random Access Memory) 102 is a volatile memory that temporarily stores a program executed by the CPU 101, data processed by the CPU 101, and the like. The storage device 103 is a non-volatile storage device. As the storage device 103, for example, a ROM (Read Only Memory), a solid state drive (SSD) such as a flash memory, a hard disk driven by a hard disk drive, or the like can be used. The input device 104 is a keyboard, a mouse, or the like. The display device 105 is a liquid crystal display, an electroluminescence panel, or the like, and displays a processing result of the information analyzer 100 or the like.

FIG. 2B is a functional block diagram of each part realized by the CPU 101 executing a program stored in the storage device 103. As illustrated in FIG. 2B, the information analyzer 100 functions as an analysis processing unit 10, a database unit 20, and the like. The analysis processing unit 10 includes a target variable extraction unit 11, a cluster classification unit 12, a sorting unit 13, a diagram creation unit 14, and the like. The database unit 20 includes a text information storage unit 21 and the like.

FIG. 3 is a flowchart showing an example of information analysis processing by the information analysis device 100. Hereinafter, the information analysis process by the information analysis apparatus 100 will be described with reference to FIGS. 2B and 3.

First, the target variable extraction unit 11 reads a plurality of text samples including text information from the text information storage unit 21 (step S1). In this embodiment, as an example, a production line report will be described as a sample including text information. For example, in the report, the factors are described as text for each element as follows. In the following example, "report number", "occurrence date and time", "stop time", "name", "action method", "cause" are elements, "report number""1" and "occurrence date and time""January 1, 2017" is a factor. Since the report contents regarding different events are described for each sample, different factors will be included for each element for each sample. For example, the description is as follows.
[Report number] 1
[Date and time of occurrence] January 1, 2017
[Stop time] 15 minutes
[Name] Part A
[Correspondence method] Replacement
[Cause] Disconnection

Next, the target variable extraction unit 11 extracts a plurality of elements from each sample, replaces any of the elements with an order scale, and then sets the target variable (step S2). In this embodiment, "stop time", "name", "cause", and "countermeasure" are extracted as a plurality of elements, and "stop time" is set as a target variable among them. As an example, here, the stop time is selected as the target variable, the sample with the stop time of 30 min or less is classified into cluster 1, the sample with stop time of 31 min or more and 60 min or less is classified into cluster 2, and the sample with stop time of 61 min or more and 90 min or less is classified into cluster 3. , 91 min or more and 120 min or less were classified into cluster 4, and 121 min or more sample was classified into cluster 5 (step S3). Here, for example, the number of samples allocated to the clusters is made to be substantially uniform among the clusters. These cluster numbers are referred to as the cluster numbers of the target variables.

Next, in the cluster classification unit 12, a multidimensional correspondence analysis is performed with each element of "name", "cause", and "correspondence method" as a text analysis target and "stop time" having an order scale as a target variable. Cluster classification is performed according to, and each factor is assigned a temporary cluster number (step S4). Here, the temporary cluster number means an identifier for identifying the formed cluster. In the cluster classification in step S4, the feature words included in the factors of each element are used for the classification. As a multidimensional correspondence analysis in the text analysis, in this embodiment, for example, the expected maximum algorithm is used.

FIG. 4 is a diagram illustrating a cluster classification result when "stop time" is used as a target variable for "cause". As illustrated in FIG. 4, in the temporary cluster number 1, characteristic words including "head failure", "head impossible", etc. are classified as characteristic words, and in the temporary cluster number 2, "fall" and "bolt" are classified. Characteristic words including, etc. are classified. As a result, a temporary cluster number is assigned to each factor belonging to each element. As for the "name" and "correspondence method", the cluster classification result can be obtained by following the same procedure.

Next, in the sorting unit 13, the "cluster number" of the target variable of the text sample is associated with the "temporary cluster number", and the average value of the cluster numbers of the target variables in all the samples is calculated for each temporary cluster number. (Step S5). In the example of FIG. 5A, in the cluster classification result when "stop time" is set as the target variable for "name", in sample number 1, cluster 1 of the target variable is associated with temporary cluster number 2. In sample 3, cluster 3 is associated with temporary cluster number 9. After that, in the example of FIG. 5B, the calculation of the average value for the provisional cluster number 1 is illustrated in the cluster classification result when the “stop time” is set as the target variable for the “name”. Similarly, in the cluster classification result when "stop time" is set as the target variable for "cause", cluster 1 is associated with temporary cluster number 2 in sample 1, and cluster 3 is temporary cluster number in sample 3. In the cluster classification result when "stop time" is set as the target variable linked to 6, the average value for each temporary cluster number is calculated.

Next, the cluster classification unit 12 assigns a larger “main cluster number” in descending order of the average value of the cluster numbers of the target variables (step S6). FIG. 5C is a diagram illustrating the present cluster number in the cluster classification result when "stop time" is set as the target variable for "name". FIG. 6A shows a list of “temporary cluster number”, “average value of cluster number of target variable”, and “main cluster number” in “name”. FIG. 6B shows a list of “temporary cluster number”, “average value of cluster number of target variable”, and “main cluster number” in “cause”. FIG. 6C shows a list of the “temporary cluster number”, the “average value of the cluster numbers of the target variables”, and the “main cluster number” in the “correspondence method”.

Next, the diagram creation unit 14 creates a path analysis result based on the obtained cluster numbers of the “name”, “cause”, and “correspondence method” (step S7). FIG. 7 is a diagram illustrating the created diagram. In the diagram, the "name" factor, the "cause" factor, the "response method" factor, and the "stop time" factor are connected by lines for each sample. The "stop time" is arranged in the order of the cluster number of the target variable. In FIG. 7, as the number of paths passing through the same route increases, the line looks thicker. The created diagram is displayed on the display device 105. After that, the execution of the flowchart ends. As illustrated in FIG. 7, in the created diagram, the amount of vertical movement of the path between the elements is small. This makes the diagram intuitive and easy to understand.

FIG. 8 is a diagram obtained as a result of performing cluster classification without using "stop time" as a target variable (comparative example). In the example of FIG. 8, the amount of movement in the vertical direction is large between the elements. As a result, the shape becomes intricate and the diagram is difficult to understand intuitively.

Next, the diagram of FIG. 7 (example) and the diagram of FIG. 8 (comparative example) are verified. In the verification, for example, the dispersion degree of the cluster is used as the first index, and the mobility of the link is used as the second index.

The dispersion of a cluster is the sum of the links connecting the elements. A link is between the same elements connected by one or more paths. As illustrated in FIG. 9A, one or more paths are connected from the name "1" to the three places of the cause "1", the cause "2", and the cause "6". In this case, the number of extended links is three. Similarly, the number of links extending from the name "2" is six. These are calculated as (1,0) = 3, (2,0) = 6, and the sum of the links connected to the elements is calculated from all the elements. The smaller the total number of links, the smaller the number of paths seems to be, so the total number of links is used as the dispersion degree of the cluster. As a result of the calculation, as shown in Table 1, the number was 168 in the example of FIG. 7, and 167 in the example of FIG. 8, and the dispersity was almost the same in both cases.

Next, the mobility of the link is the difference between the cluster numbers at the start and end points of the link. If this mobility is high, the clusters are separated (distributed). As illustrated in FIG. 9B, since the link A connects the name “1” and the cause “1”, the difference between the cluster numbers, that is, the mobility is 0. Since the link B connects the name "1" and the cause "2", the difference between the cluster numbers, that is, the mobility is 1. In this way, the sum of the mobility of all the links is calculated and used as the mobility of the link. As shown in Table 1, it was 521 in the example of FIG. 7, and 596 in the example of FIG. As a result, in the example of FIG. 7, the dispersity was reduced by about 15% as compared with the example of FIG. Therefore, it was found that the diagram in FIG. 7 was intuitively easy to understand as compared with the example in FIG.

According to this embodiment, factor information consisting of feature words can be extracted for each element of each text sample, and a target variable group with an introductory scale can be extracted for at least one or more elements. After that, among the extracted plurality of elements, each element other than the target variable is subjected to multidimensional correspondence analysis and cluster classification is performed. By doing so, the order scale of the target variable is reflected in the classification result, so that the classified clusters can be correlated with each other. By creating a diagram by path analysis based on this cluster information and performing visualization, the complexity of the path is eliminated and the diagram becomes intuitive and easy to understand.

In this embodiment, a numerical range that is continuous with each other, such as a stop time, is used as an order scale, but the order is not limited to this. In addition to the numerical range, an interval scale, a proportional scale, and the like can be applied.
(Modification example)

When calculating the average value of the cluster numbers of the target variables, the influence of the ordinal scale of the target variables may be large. For example, a cluster number of 30 min or less is "1", a cluster number of 31 min or more and 60 min or less is "2", a cluster number of 61 min or more and 90 min or less is "3", and a cluster number of 91 min or more and 120 min or less is "8". , The cluster number of 121 min or more may be set to "10", and the long stop time may be weighted. The clustering results in this case are shown in FIGS. 10 (a) to 10 (c). FIG. 10A shows a list of “temporary cluster number”, “average value of cluster number of target variable”, and “main cluster number” in “name”. FIG. 10B shows a list of “temporary cluster number”, “average value of cluster number of target variable”, and “main cluster number” in “cause”. FIG. 10C shows a list of the “temporary cluster number”, the “average value of the cluster numbers of the target variables”, and the “main cluster number” in the “correspondence method”. Further, FIG. 11 shows a diagram diagram of each cluster number of the obtained "name", "cause", and "countermeasure method:".

Comparing FIGS. 6 (a) to 6 (c) with FIGS. 10 (a) to 10 (c), there is a difference in the cluster numbers. The above-mentioned first index and second index were calculated with respect to the result of FIG. The results are shown in Table 2. As shown in Table 2, the first index has a small value, and the second index has a smaller value. This is because the weighting makes it possible to better reflect the influence of the order scale.

In each of the above examples, the target variable extraction unit 11 extracts a plurality of factors indicating characteristic words for a plurality of elements from the same text sample, and for one or more of the plurality of elements. It functions as an example of a target variable extraction unit that assigns a target variable consisting of an ordinal scale. The cluster classification unit 12 functions as an example of a cluster classification unit that performs cluster classification by performing a multidimensional correspondence analysis on the plurality of elements using the target variables. The sorting unit 13 assigns a tentative identifier to each cluster formed after the multidimensional correspondence analysis by the cluster classification unit, and for each element other than the element to which the target variable is assigned, for each of the tentative identifiers. It functions as an example of a sorting unit that calculates the average value of the ordinal scale of all text samples, sorts the identifiers of the clusters in ascending order of the average values, and assigns new identifier numbers. The diagram creation unit 14 arranges the clusters rearranged by the rearrangement unit for each element, and creates a diagram by connecting factors belonging to the same text sample with adjacent elements with a line. Works as an example.

Although the examples of the present invention have been described in detail above, the present invention is not limited to the specific examples, and various modifications and modifications are made within the scope of the gist of the present invention described in the claims. It can be changed.

10 Analysis processing unit 11 Target variable extraction unit 12 Cluster classification unit 13 Sorting unit 14 Diagram creation unit 20 Database unit 21 Text information storage unit 100 Information analyzer

Claims (5)

A target variable extraction unit that extracts a plurality of factors indicating characteristic words for a plurality of elements from the same text sample and assigns a target variable consisting of an order scale to one or more of the plurality of elements. ,
A cluster classification unit that performs cluster classification by performing multidimensional correspondence analysis on the plurality of elements using the target variables.
The cluster classification unit assigns a tentative identifier to each cluster formed after the multidimensional correspondence analysis, and for each element other than the element to which the target variable is assigned, the above-mentioned of all text samples for each tentative identifier. An information analyzer comprising a sorting unit that calculates an average value of an ordinal scale, sorts the identifiers of the clusters in ascending order of the average values, and assigns a new identifier number. The cluster is rearranged by the rearrangement unit, and the cluster is arranged for each element, and a diagram creation unit for creating a diagram by connecting factors belonging to the same text sample with adjacent elements with a line is provided. The information analyzer according to claim 1. The information analyzer according to claim 1 or 2, wherein the target variable extraction unit gives a larger weight to the order scale as the order scale is larger. A process of extracting a plurality of factors indicating characteristic words for a plurality of elements from the same text sample and assigning a target variable consisting of an order scale to one or more of the plurality of elements.
A process of performing cluster classification by performing a multidimensional correspondence analysis for each of the elements using the target variable, and
A tentative identifier is assigned to each cluster formed after the multidimensional correspondence analysis, and for each element other than the element to which the target variable is assigned, the average value of the ordinal scale of all text samples is calculated for each tentative identifier. An information analysis method characterized in that a computer executes a process of calculating, rearranging the identifiers of the clusters in ascending order of the average value, and assigning a new identifier number. On the computer
A process of extracting a plurality of factors indicating characteristic words for a plurality of elements from the same text sample and assigning a target variable consisting of an order scale to one or more of the plurality of elements.
A process of performing cluster classification by performing a multidimensional correspondence analysis on the plurality of elements using the target variables, and
A tentative identifier is assigned to each cluster formed after the multidimensional correspondence analysis, and for each element other than the element to which the target variable is assigned, the average value of the ordinal scale of all text samples is calculated for each tentative identifier. An information analysis program characterized in that a process of calculating, rearranging the identifiers of the cluster in ascending order of the average value, and assigning a new identifier number is executed.

Images