JP2011103059A - Technical term extraction device and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To extract a keyword that features an individual by excluding a general term, on the basis of the speech of a member in a community. <P>SOLUTION: A technical term extraction device 1 is provided with: a morphological analytic part 20 for morphologically analyzing a document input in response to the operation of a contributor; a deviation score calculation part 30 for calculating deviation scores between words included in the document, between the word and the contributor, and between the word and a contribution destination group to which the contributor belongs; a general term extraction part 40 for extracting a general term included in the document according to the value of the deviation score; and an index extraction part 50 for extracting a keyword showing individual features by excluding the general term extracted by the general term extraction part 40 from the document. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a technical term extraction device program.

  In recent years, various topics have been discussed in social network services and communities in newsgroups, and active communication is carried out among speakers. Among them, experts who are familiar with a specific field may speak, and information on what field an individual is familiar with is important information for smooth communication. In social network services, the community to which you belong is often announced. However, information on the community to which they belong is not enough, and it is desirable to know what topics each person is actually interested in.

  As a function for knowing in which field an individual is familiar, there is a Know-how function (expert search function). There are two approaches to realizing the Know-how function. One is a method of determining a person who has the highest weight as a document book among a group of documents searched with a specific keyword as an expert. The other is a method of explicitly extracting keywords that characterize individuals.

  For example, Patent Document 1 discloses a technical term extraction device that extracts technical terms from a category-attached document set. Further, Patent Document 1 discloses a method of extracting a term having a close relationship with a category by making a document with a department name, a person's name, an e-mail address, etc. added to the document content as a category-added document. In particular, a method for extracting technical terms from a document set to which a plurality of categories are assigned is disclosed.

  In Patent Document 1, since a plurality of categories are assigned to one document, the degree of bias of words appearing in each category is reduced, and leakage from technical terms occurs when judging a technical term above a certain threshold. Is prevented.

JP 2007-79948 A JP 2007-79948 A

However, Patent Document 1 does not consider the case where words that are not technical terms are ranked higher. Here, a word that does not look like a technical term is a word that has a score that indicates that it is like a technical term for a plurality of categories. Moreover, in patent document 1, there existed a problem that the difference specialized in the property of the category, such as a category being a person name or an organization, was not considered.
Therefore, the present invention has been made in view of the above problems, and an object of the present invention is to extract keywords that characterize individuals by excluding general terms based on the comments of members in the community. It is an object of the present invention to provide a new and improved terminology extraction device and program that can be used.

  In order to solve the above problems, according to an aspect of the present invention, a morphological analysis unit that performs morphological analysis on a document input in accordance with a contributor's operation, a word and a contributor between words included in the document A bias score calculation unit for calculating a bias score between a word and a posting destination group to which the poster belongs, and a general term included in the document in accordance with the value of the bias score A technical term extraction device comprising: a term extraction unit; and an index extraction unit that extracts a keyword indicating an individual characteristic by removing the general term extracted by the general term extraction unit from the document. Is provided.

  In addition, the technical term extraction device may include a storage unit that stores a document input in accordance with a contributor's operation, a contributor, and a posting destination group to which the contributor belongs.

  Further, the bias score calculation unit may calculate the bias score using a chi-square value.

  In addition, the general term extraction unit has a bias score value that is equal to or less than a predetermined value among a combination of a poster and a word or a posting destination group to which the poster belongs and a word. Words related to the posting destination group may be extracted as general terms.

  In addition, the bias score calculation unit calculates the bias score again except for the word extracted as the general term, and the general term extraction unit calculates the poster and the word or the posting destination group to which the poster belongs. Of the combinations with words, the bias score value may be equal to or less than a predetermined value, and a word related to a plurality of contributors or a plurality of posting destination groups may be extracted again as a general term.

  The index extraction unit may extract a word having a bias score value between a poster and a word that is equal to or greater than a predetermined value as a keyword indicating personal characteristics.

  In addition, the index extraction unit extracts a word indicating the characteristics of the posting destination group, and the bias score between the posting person and the posting destination group to which the posting person belongs is a predetermined value or more. A word indicating a feature may be extracted as a keyword indicating an individual feature.

  The index extraction unit may include a period in which a document including the word is posted and a period in which the poster has posted in the posting group among words indicating the characteristics of the extracted posting group. In addition, the words may be extracted as keywords indicating individual characteristics.

  In order to solve the above-described problem, according to another aspect of the present invention, a computer uses a morphological analysis unit that performs a morphological analysis on a document input in accordance with a contributor's operation, and a word between words included in the document. A bias score calculation unit for calculating a bias score between a word and a poster, a word and a posting destination group to which the poster belongs, and a general score included in the document according to the value of the bias score A general term extraction unit that extracts terms; and an index extraction unit that extracts keywords indicating individual characteristics by removing the general terms extracted by the general term extraction unit from the document. A program for functioning as a technical term extraction device is provided.

  As described above, according to the present invention, keywords that characterize individuals can be extracted based on the remarks of members in a community, excluding general terms.

It is a block diagram which shows the function structure of the technical vocabulary extraction apparatus concerning the 1st Embodiment of this invention. It is explanatory drawing explaining the memory content of the document storage part with a category concerning the embodiment. It is explanatory drawing explaining the memory content of the morpheme temporary storage part concerning the embodiment. It is explanatory drawing explaining the memory content of the general term temporary storage part concerning the embodiment. It is explanatory drawing explaining the memory content of the index storage part concerning the embodiment. It is explanatory drawing explaining an example of the confirmation screen of the index concerning the embodiment. It is a flowchart which shows the detail of the technical vocabulary extraction process concerning the embodiment. It is a flowchart which shows the detail of the process which calculates the appearance number of each combination concerning the embodiment. It is explanatory drawing explaining the memory content of the co-occurrence list temporary storage part concerning the embodiment. It is explanatory drawing explaining the memory content of the appearance number temporary storage part concerning the embodiment. It is explanatory drawing explaining the memory content of the bias score temporary storage part concerning the embodiment. It is a flowchart which shows the detail of the extraction process of the technical term of the community concerning the 2nd Embodiment of this invention.

  Exemplary embodiments of the present invention will be described below in detail with reference to the accompanying drawings. In addition, in this specification and drawing, about the component which has the substantially same function structure, duplication description is abbreviate | omitted by attaching | subjecting the same code | symbol.

Further, the “detailed description of the embodiments” will be described in the order shown below.
[1] Purpose of this embodiment [2] First embodiment [2-1] Functional configuration of technical term extraction device [2-2] Details of technical term extraction processing [3] Second embodiment [3-1] Functional configuration of technical term extraction device [3-2] Details of technical term extraction processing

[1] Object of this embodiment First, the object of the embodiment of the present invention will be described. In recent years, various topics have been discussed in social network services and communities in newsgroups, and active communication is carried out among speakers. Among them, experts who are familiar with a specific field may speak, and information on what field an individual is familiar with is important information for smooth communication. In social network services, the community to which you belong is often announced. However, information on the community to which they belong is not enough, and it is desirable to know what topics each person is actually interested in.

  As a function for knowing in which field an individual is familiar, there is a Know-how function (expert search function). There are two approaches to realizing the Know-how function. One is a method of determining a person who has the highest weight as a document book among a group of documents searched with a specific keyword as an expert. The other is a method of explicitly extracting keywords that characterize individuals.

  For example, a technical term extraction device that extracts technical terms from a document set with categories is disclosed. In addition, this apparatus discloses a method of extracting a term having a close relationship with a category by setting a document with a department name, a person's name, an e-mail address, etc. to the document content as a document with a category. In particular, a method for extracting technical terms from a document set to which a plurality of categories are assigned is disclosed.

  In the above device, by assigning a plurality of categories to one document, the degree of bias of words appearing in each category is reduced, and when a certain threshold or more is judged as a technical term, it is omitted from the technical term. Is prevented. However, the above apparatus does not consider the case where words that are not technical terms are ranked higher. Here, a word that does not look like a technical term is a word that has a score that indicates that it is like a technical term for a plurality of categories. In addition, the above apparatus has a problem that it does not take into account differences specific to the nature of the category, such as whether the category is a person name or an organization.

  Therefore, the technical term extraction device 1 according to the embodiment of the present invention has been created with the above circumstances as a focus. According to the technical term extraction device 1 according to the present embodiment, it is possible to extract keywords that characterize individuals by excluding general terms based on the comments of members in the community. In the present embodiment, the community means a posting destination group to which a contributor belongs. For example, in social networks and bulletin boards, members belong to a community, so it can be said that the terminology of the community indicates the specialty of the member belonging to the community. Therefore, in consideration of the words included in the remarks of the community and the members, the relationship between the members and the words, and the relationship between the words, keywords that characterize individuals are expanded by excluding general terms.

[2] First Embodiment The object of the embodiment of the present invention has been described above. Next, a functional configuration of the technical term extraction device 1 according to the present embodiment will be described with reference to FIG. In describing the functional configuration of the technical term extraction device 1, FIGS. 2 to 9 will be referred to as appropriate. As the technical term extraction device 1, for example, a computer device such as a personal computer (whether a notebook type or a desktop type) can be exemplified. However, the technical term extraction device 1 is not limited to such an example, and may be a mobile phone or a PDA (Personal Digital Assistant). It may be configured.

[2-1] Functional Configuration of Technical Term Extraction Device Before describing the functional configuration of FIG. 1, an example of a hardware configuration of the technical term extraction device 1 will be described. The technical term extraction device 1 includes, for example, a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), an input device, an output device, a storage device (HDD), and the like.

  The CPU functions as an arithmetic processing device and a control device, and controls the overall operation of the technical term extraction device 1 according to various programs. The CPU may be a microprocessor. The ROM stores programs used by the CPU, calculation parameters, and the like. The RAM primarily stores programs used in the execution of the CPU, parameters that change as appropriate during the execution, and the like. These are connected to each other by a host bus including a CPU bus.

  The input device includes, for example, an input means for a user to input information, such as a mouse, keyboard, touch panel, button, microphone, switch, and lever, and input control that generates an input signal based on the input by the user and outputs the input signal to the CPU. It consists of a circuit.

  The output device includes, for example, a display device such as a CRT (Cathode Ray Tube) display device, a liquid crystal display (LCD) device, an OLED (Organic Light Emitting Display) device and a lamp, and an audio output device such as a speaker and headphones. .

  The storage device can include a storage medium, a recording device that records data on the storage medium, a reading device that reads data from the storage medium, a deletion device that deletes data recorded on the storage medium, and the like. The storage device is composed of, for example, an HDD (Hard Disk Drive). This storage device drives a hard disk and stores programs executed by the CPU and various data.

  Heretofore, the hardware configuration of the technical term extraction device 1 has been described. Next, the functional configuration of the technical term extraction device 1 will be described with reference to FIG. As shown in FIG. 1, the technical term extraction device 1 includes an input unit 10, a morpheme analysis unit 20, a bias score calculation unit 30, a general term extraction unit 40, an index extraction unit 50, a confirmation display unit 60, and a document storage with categories. Unit 70, morpheme temporary storage unit 80, co-occurrence list temporary storage unit 90, appearance number temporary storage unit 100, bias score temporary storage unit 110, general term temporary storage unit 120, index storage unit 130, and the like.

  The input unit 10 is configured by the input device described above. The user of the technical term extraction device 1 can input various data and instruct a processing operation to the technical term extraction device 1 by operating the input unit 10. Specifically, it has a function of accepting an extraction request for a keyword characterizing an individual in response to a user operation.

  The morpheme analysis unit 20 has a function of performing morphological analysis on the text information stored in the category-added document storage unit 70 when a request for technical term extraction is received from the user via the input unit 10. The category includes the creator who posted the document, the community or topic where the document was posted, and the like. Here, the contents stored in the category-added document storage unit 70 will be described with reference to FIG. FIG. 2 is an explanatory diagram for explaining the stored contents of the category-added document storage unit 70.

  As shown in FIG. 2, the category-added document storage unit 70 holds items of document ID, community ID, topic ID, poster ID, posting time, and posting content. Posts to newsgroups and social network systems are made to topics within the community. A topic corresponds to an individual topic posted to each community (or news group), and a plurality of comments can be posted on one topic. Individual topics are identified by topic IDs. Each post is identified by a document ID.

  Each post is identified by a document ID. The contributor ID stores the contributor ID for identifying the contributor, and the post time stores the time when the post was made. The posted content item stores posted text information. For example, in R71 (document ID = d1) in FIG. 3, the poster m1 posted “(Special Feature) Programming Language...” On the topic t1 on “December 11, 2008 12:00”. It shows that.

  Returning to FIG. 1, the morphological analysis unit 20 has a function of performing morphological analysis on text information of a posted document. The morpheme analyzer 20 stores the result of the morpheme analysis of the text information in the morpheme temporary storage unit 80. The morpheme analysis unit 20 stores the general noun and the saun noun among the morpheme analysis results in the morpheme temporary storage unit 80.

  Here, the stored contents of the morpheme temporary storage unit 80 will be described with reference to FIG. FIG. 3 is an explanatory diagram for explaining the stored contents of the morpheme temporary storage unit 80. As shown in FIG. 3, the result 805 of the morphological analysis is stored in association with the document ID 801, community ID 802, topic ID 803, and contributor ID 804. The result of the morphological analysis of the record R71 stored in the category-added document storage unit 70 shown in FIG. 2 is the record R81 stored in the morpheme temporary storage unit 80 shown in FIG. The morpheme analysis unit 20 provides the result of the morpheme analysis to the bias score calculation unit 30.

  Returning to FIG. 1, the bias score calculation unit 30 uses the morpheme analysis result processed by the morpheme analysis unit 20 to determine what words have appeared in each document, and the co-occurrence list temporary storage unit 90 and the number of appearances. Store in the temporary storage unit 100. Further, chi-square values and Simpson values between words or between words and categories are calculated as bias scores. The contents stored in the co-occurrence list temporary storage unit 90 and the appearance number temporary storage unit 100 will be described later in detail.

  The bias score calculation unit 30 calculates the number of appearances for each combination in order to calculate chi-square values for word combinations, poster ID / word combinations, and community ID / word combinations. The calculation of the number of appearances will be described in detail later. The bias score calculation unit 30 uses the stored contents of the co-occurrence list temporary storage unit 90 and the appearance number temporary storage unit 100 to calculate chi-square values for word combinations, poster / word combinations, and community ID / word combinations. calculate.

  A method for calculating the chi-square value will be described later in detail. The bias score calculation unit 30 stores the calculated bias score in the bias score temporary storage unit 110. The contents stored in the bias score temporary storage unit 110 will be described in detail later. In addition, the bias score calculation unit 30 provides the calculated result to the general term extraction unit 40.

  The general term extraction unit 40 extracts words with less bias as general terms using the result calculated by the bias score calculation unit 30. The general term extraction unit 40 extracts records satisfying a predetermined condition from the records stored in the bias score temporary storage unit 110. The record extraction process by the general term extraction unit 40 will be described in detail later. The general term extraction unit 40 stores the extracted general terms in the general term temporary storage unit 120. As shown in FIG. 4, for example, the word “special feature” is stored as a general term in the general term temporary storage unit 120. The general term extraction unit 40 provides the extracted general terms to the index extraction unit 50.

  Returning to FIG. 1, the index extraction unit 50 calculates the index of each category from the bias score between categories and words and the bias score between words based on the bias score value calculated by the bias score calculation unit 30. Extract. The index extracted by the index extraction unit 50 becomes a keyword that characterizes an individual characteristic. Further, the index storage unit 130 stores the extracted index in the index storage unit 130.

  As shown in FIG. 5, the index storage unit 130 stores a contributor ID 1301, a technical term 1302, and a score 1304 in association with each other. Thereby, a word (technical term 1302) characterizing an individual (each contributor 1301) is extracted. Furthermore, the score 1304 shows how deeply the relationship between an individual and a word is. In addition, an approval presence / absence 1305 indicating whether or not the index (technical term 1302) stored in the index storage unit 130 is approved by the person to whom the index is assigned is also stored.

  Returning to FIG. 1, the confirmation display unit 60 has a function of confirming whether the index stored in the index storage unit 130 is correct through the display screen to the person to whom the index is assigned. For example, as shown in FIG. 6, whether or not to approve an index (technical term) of “XX language” or “C ++” as an index characterizing the person on the display screen is displayed on the confirmation screen. Ask for.

  When “approval” is selected in accordance with user input, the approval / non-approval item in the index storage unit 130 is updated from “unapproved” to “approved”. On the other hand, when “reject” is selected according to the user's input, the approval / non-approval item in the index storage unit 130 is updated from “unapproved” to “reject”.

  The morpheme analysis unit 20, the bias score calculation unit 30, the general term extraction unit 40, and the index extraction unit 50 are configured by a computer, and the operation thereof is based on a program stored in the ROM. Is executed. The index storage unit 130 includes the storage device (HDD) described above, and includes a category-added document storage unit 70, a morpheme temporary storage unit 80, a co-occurrence list temporary storage unit 90, an appearance number temporary storage unit 100, a bias score. The temporary storage unit 110 and the general term temporary storage unit 120 are configured by the storage device (HDD) or RAM described above.

[2-2] Details of Technical Term Extraction Processing The functional configuration of the technical term extraction device 1 according to the present embodiment has been described above. Next, with reference to FIG. 7 and FIG. 8, the details of the technical term extraction process in the technical term extraction device 1 will be described. In describing the details of the technical term extraction processing, FIGS. 9 to 11 are referred to as appropriate. FIG. 7 is a flowchart showing details of the technical term extraction process.

  As shown in FIG. 7, first, the input unit 10 receives an index creation request (S100). In step S100, when an index creation request is received by the input unit 10, the morpheme analysis unit 20 performs morpheme analysis on the text data in the category-added document storage unit 70 of FIG. 2 (S110). .

  The morpheme analysis unit 20 performs morpheme analysis on the posted content item in the category-added document storage unit 70. Of the morpheme analysis results, general nouns and saun nouns are stored in the morpheme temporary storage unit 80 shown in FIG. As described above, the result of morphological analysis of the record R71 stored in the category-added document storage unit 70 shown in FIG. 2 is the record R81 stored in the morpheme temporary storage unit 80 shown in FIG.

  The bias score calculation unit 30 calculates chi-square values for word combinations, poster ID and word combinations, poster ID and word combinations, and community ID and word combinations. First, the number of occurrences of each combination is calculated. Here, the process of calculating the number of appearances of each combination will be described with reference to FIG. FIG. 8 is a flowchart showing details of processing for calculating the number of appearances of each combination.

  As shown in FIG. 8, the bias score calculation unit 30 forms a set for which the number of co-occurrence is calculated for each record stored in the morpheme temporary storage unit 80 (S1200). The elements of the set are a community ID, a contributor ID, and a word. For example, in the record R81 in FIG. 3, {c1, m1, special feature, programming, language} is an element.

  Then, for the elements of the set formed in step S1200, a cocoon set with 2 elements is created (S1210). For example, in the record R81 of FIG. 3, {{c1, m1}, {c1, special feature}, {c1, programming}, {c1, language}, {m1, special feature}, {m1, programming}, {m1, Language}, {special feature, programming}, {special feature, language}, {language, programming}}.

  Then, the number of appearances is added to the element calculated in step S1210 (S1220). Specifically, each time the element calculated in step S1210 appears, the number of appearances is added to the corresponding record in the co-occurrence list temporary storage unit 90 shown in FIG. At the same time, the number of appearances is also added to the corresponding record in the appearance number temporary storage unit 100 shown in FIG.

  As shown in FIG. 9, in the co-occurrence list temporary storage unit 90, two items of the community ID 901, the contributor ID 902, the word 1903, and the word 2904 have values, and the two values are included. The combination is made up of The number of appearances 905 indicates the number of appearances of each item.

  Further, as shown in FIG. 10, in the appearance number temporary storage unit 100, each item of the community ID 1001, the contributor ID 1002, and the word 11003 indicates the corresponding element name. The appearance number 1004 indicates the number of appearances of each element.

  For example, in the record R701 in the co-occurrence list temporary storage unit 90 shown in FIG. 9, when a bag set is created in step S1210 for all documents, two bag sets having a combination of (community c1, programming) exist. Indicates that it has appeared once. The process for calculating the number of appearances of each combination has been described above.

  Returning to FIG. 7, the description of the technical term extraction process will be continued. After calculating the number of combinations in step S120, the bias score calculation unit 30 calculates chi-square values for the word combinations and the community ID / word combinations. The chi-square value of the word X (or community X or contributor X) and the word Y is calculated by the following formula 1.

Here, N is the total number of elements, and in this embodiment, is the total number of records in the co-occurrence list temporary storage unit 90 of FIG. O ₁₁ , O ₁₂ , O ₂₁ , and O ₂₂ are as follows.

O ₁₁ : [number of elements in which word X and word Y co-occurred] (or [number of elements in which community X and word Y co-occurd] or [number of elements in which poster X and word Y co-occurd] )
The number of appearances of each record in the co-occurrence list temporary storage unit 90 in FIG. 9 corresponds.
O ₂₂ : [the number of elements in which neither word X nor word Y appears] (or [the number of elements in which word Y does not appear in documents other than community X], or [the number of documents not posted by contributor X] Number of elements in which word Y does not appear])
(That is, O ₂₂ = N—O ₁₁ —O ₁₂ —O ₂₁ )
O ₁₂ : [Number of elements in which only the word X appears but the word X does not appear] (or [Number of elements in the community X document in which the word Y did not appear) or [Posted by the contributor X Number of elements in which the word Y did not appear in the number of documents])
O ₂₁ : [Number of elements in which only word Y appears, and word X does not appear] (or [Number of elements in which word Y appears in documents other than community X], or [not posted by contributor X Number of elements in which word Y appears in the number of documents])

Further, the chi-square value of the community X and the contributor Y is calculated in the same manner. in this case,
O ₁₁ : [Number of elements of contributor Y in community X documents]
O ₂₂ : [number of elements other than contributor Y in documents other than community X]
O ₁₂ : [number of elements other than the contributor Y in the community X document]
O ₂₁ : [number of elements of contributor Y in documents other than community X]
It becomes.

O ₁₁ , O ₁₂ , O ₂₁ , and O ₂₂ can be calculated using the co-occurrence list temporary storage unit 90 and the appearance number temporary storage unit 100 of FIGS. 9 and 10. For example, in FIG. 10, the number of elements of the word “programming” is “22”. The number of elements of the community “c1” is “80”. O ₁₁ , O ₁₂ , O ₂₁ , and O ₂₂ for calculating the chi-square value of the community “c1” and the word “programming” are as follows. However, the total number of elements is 1000.

O ₁₁ : 2 (number of appearances of record R91 in FIG. 9)
O ₁₂ : 78 (number of elements not including “programming” in documents of community “c1”: number of appearances of record R101 in FIG. 10−number of appearances of record R91 in FIG. 9 = 80-2)
O ₂₁ : 20 (number of elements including “programming” in documents other than community “c1”: number of appearances of record R102 in FIG. 10−number of appearances of record R91 in FIG. 9 = 22-2)
O ₂₂ : 900 (documents other than O ₁₁ , O ₁₂ , and O ₂₁ = 1000-2-78-20)

Therefore, the chi-square value is
{1000 × (2 × 900−78 × 20) ² } / {(2 + 78) × (2 + 20) × (78 + 900) × (20 + 900)} ≈0.036
It becomes.

The calculation of the chi-square value is executed for combinations of words, combinations of words and posters, combinations of words and communities, and combinations of communities and posters. However, when O ₁₁ is 1 or less, since co-occurrence hardly occurs, it is excluded from calculation. As a result, since the chi-square value as the bias score can be acquired, the chi-square value is stored in the bias score temporary storage unit 110 shown in FIG.

And after calculating a bias score in step S130, the general term extraction part 40 extracts a general term based on the chi-square value calculated in step S130 (S140). First, records satisfying the following two conditions are extracted from the bias score temporary storage unit 110 as records indicating general terms.
(Condition 1) The chi-square value is 1.0 or less, the word 1 item has a value, and the community ID item or the poster ID has a value. (Condition 2) Records satisfying the condition 1 When there is a word 1 item as a key, there must be 3 or more records with the same value in the word 1 item.

A record that satisfies the above conditions can be obtained by the following SQL, for example.
select word 1, count (*) from bias score temporary storage unit
where chi-square ≤ 1.0 AND (community ID is not
null OR contributor ID is not null) AND word 1 is not null group by word 1 having count (*) ≧ 3;

  For example, in FIG. 11, records R111, R112, and 113 correspond to records that satisfy condition 1. Since the condition 2 is also satisfied, the word “special feature” is obtained as a general term. And the obtained word is memorize | stored in the general term temporary storage part 120 shown in above-mentioned FIG.

  Next, in order to eliminate the general terms extracted in step S140 and calculate the bias score again, the bias score temporary storage unit 110 is cleared (S150). Then, the bias score calculation unit 30 executes the calculation of the chi-square value again, excluding the general terms extracted in step S140 (S160). Since the bias score recalculation process in step S160 is the same as the process executed in steps S120 and S130, detailed description thereof is omitted. However, when the co-occurrence list temporary storage unit 90 and the appearance number temporary storage unit 100 are created in step S120, the same word as the general term temporary storage unit 120 is included in the word 1 item or the word 2 item. In that case, the element is not created.

  The result of recalculating the chi-square value in step S160 is stored in the bias score temporary storage unit 110. For example, since the word “special feature” is determined as a general term, the combination of (c1, special feature) in R114 of FIG. 11 is not stored.

  Next, the general term extraction unit 40 extracts general terms under the same conditions as in step S140 (S170). Then, the general terms obtained in step S170 are added to the general term temporary storage unit 120. Then, it is determined whether or not to recalculate the bias score (S180). In step S180, for example, it is assumed that the condition is “recalculate the bias score only once”. In step S180, if the condition “recalculate the bias score only once” is satisfied, step S190 is executed. On the other hand, if the condition “recalculate the bias score only once” is not satisfied in step S180, the processing from step S150 to step S170 is repeated.

  In step S190, the index extracting unit 50 extracts community technical terms (S190). The index extraction unit 50 sorts the data in the bias score temporary storage unit 110 in descending order of bias score for each community in order to extract the technical terms of the community. This can be obtained, for example, by the following SQL.

  select community ID, word 1, chi-square value from bias score temporary storage unit where chi-square value> 1.0 AND community ID is not null AND word 1 is not null order by community ID, chi-square value desc;

For example, in the data stored in FIG. 11, in the order of community ID, word 1, chi-square value,
c1, OO language, 100
c3, search system, 80
Data such as can be obtained. From this data, it can be seen that the technical term of the community c1 is “XX language” and the technical term of the community c3 is “search system”.

  Next, the index extraction unit 50 extracts member technical terms (S200). In step S200, the data in the bias score temporary storage unit 110 is sorted in descending order of bias score for each poster ID. This can be obtained, for example, by the following SQL.

  select contributor ID, word 1, chi-square value from bias score temporary storage section where chi-square value> 1.0 AND contributor ID is not null AND word 1 is not null order by contributor ID, chi-square value dessc;

For example, in the data stored in FIG. 11, in the order of contributor ID, word 1, chi-square value,
m1, OO language, 100
Data such as is obtained. Therefore, it can be seen that the technical term of the member m1 is “XX language”. The index extraction unit 50 stores the obtained data in the index storage unit 130. As described above, the index storage unit 130 stores the data shown in FIG. The score 1304 in FIG. 5 stores a chi-square value. For example, in the data shown in FIG. 11, the record R131 in FIG. 5 is added.

  And the index extraction part 50 judges the relevance degree of a member and a community (S210). In step S210, the data in the bias score temporary storage unit 110 is sorted in descending order of bias score for each community ID and poster ID. This can be obtained, for example, by the following SQL.

  select community ID, contributor ID, chi-square value from bias score temporary storage part where chi-square value> 1.0 AND community ID is not null AND contributor ID is not null order by contributor ID, community ID, chi-square value dessc;

For example, in the data stored in FIG. 11, in the order of community ID, contributor ID, chi-square value,
c1, m9, 70
Data such as is obtained. Therefore, it can be determined that the community c1 and the member m9 are deeply related.

  The index extraction unit 50 stores the technical terms of the community obtained in step S190 in the index storage unit 130 for the community determined to have a deep relationship in step S210 (S220).

  For example, as described above, since the community c1 and the member m9 are deeply related, the technical term “XX language” of the community c1 extracted in step S200 is stored as the technical term of the member m9. The data added here becomes a record R132 in FIG. The score 1304 stores a chi-square value between the community c1 and the member m9.

  The index extraction unit 50 extracts combinations with strong associations between words from the data added up to step S220 and adds them to the index storage unit 130 (S230). A combination having a strong association between words is obtained, for example, by the following SQL. In the following example, the chi-square value threshold is set to 50, but is not limited to this example.

select word 1, word 2, chi-square value from bias score temporary storage unit
where chi-square value> 50 AND word is not null AND word 1 is not null and (word 1 in ('XX language') or word 2 in ('XX language') order by chi-square value dessc;

  As a result obtained by the above SQL, the record R115 in FIG. 11 is extracted, and “C ++” can be obtained as a related word. The obtained related terms are stored in the index storage unit 130 as indicated by records R133 and R134 in FIG.

  The index registration process in the index storage unit 130 has been described above. As described above, the index storage unit 130 shown in FIG. 5 is provided with an approval / non-approval item 1305 for each registered data. When the item of approval / non-approval 1305 is “unapproved”, the member is requested to input confirmation. For example, when the corresponding member logs in to the social network system, an index confirmation screen shown in FIG. 6 is displayed on the confirmation display unit 60 to request confirmation input from the member.

  The details of the technical term extraction processing have been described above. In this embodiment, a chi-square value is calculated as a value representing the relationship between the community and the word, the member and the word, and the word. And even if a word is judged to be a technical term in multiple communities or members, it is judged not to be a technical term (general term) based on a predetermined condition. By calculating the relationship, it is possible to extract technical terms with high accuracy.

  In addition to extracting technical terms based on the strength of the relationship between members and words, it is also possible to extract technical terms of members by considering the relationship between communities and members, and the relationship between communities and words. Become. In other words, technical terms that are not included in the documents posted by the user can also be used as technical terms of their own in relation to the community, so that it is possible to associate a plurality of technical terms that characterize individuals.

[3] Second Embodiment Next, a second embodiment of the present invention will be described. In the first embodiment, words that are closely related to the community are set as words that characterize members that are closely related to the community. However, the first embodiment does not consider the temporal condition of when the word appears. For example, in the first embodiment, even if the associated word never appears within the period posted by the member, it is set as a word characterizing the member. Therefore, the present embodiment is different from the first embodiment in that a word characterizing the member is extracted in consideration of the time when the member posted in the community. Hereinafter, differences from the first embodiment will be particularly described, and detailed description of points similar to those of the first embodiment will be omitted.

[3-1] Functional Configuration of Technical Term Extraction Device The functional configuration of the technical term extraction device 2 according to the present embodiment is substantially the same as the functional configuration of the technical term extraction device 1 according to the first embodiment, and thus detailed description thereof. Is omitted. This embodiment is different from the first embodiment in that the index extraction unit 50 considers the posting period of members in the community when extracting the technical terms of the community. The community terminology extraction process by the index extraction unit 50 will be described in detail in the explanation of the terminology extraction process described later.

[3-2] Details of Terminology Extraction Processing The terminology extraction processing in the present embodiment is different from the first embodiment in terms of community terminology extraction methods. Detailed description of the same processing as that of the first embodiment will be omitted.

  In the technical term extraction process in the present embodiment, the process shown in FIG. 12 is executed in place of the community technical term extraction process in step 190 of FIG. 7 in the first embodiment. FIG. 12 is a flowchart showing details of the process for extracting the technical terms of the community in the present embodiment.

As shown in FIG. 12, the index extraction unit 50 stores, in the index storage unit 130, the technical terms of the community obtained in step S190 based on the following conditions for the community determined to be deeply related in step S210.
(Condition) In the community, the period in which the technical term appears and the member's posting period in the community overlap.

  In order to satisfy the above conditions, the index extraction unit 50 first calculates the appearance period of community technical terms (S2000). For example, the technical term “XX language” of the community c1 appears in the records of the documents d3 and d4 in the morpheme temporary storage unit 80 of FIG. Since the document ID of the morpheme temporary storage unit 80 in FIG. 3 corresponds to the document ID of the category-added document storage unit 70 in FIG. 2, the posting time “December 11, 2008” to “2008” in FIG. “December 14th” is the appearance period.

  Then, the posting period in the community of the member (contributor) is calculated (S2010). In the category-added document storage unit 70 of FIG. 2, the contributor m1 posted in the community c1 from “December 11, 2008” (document d1) to “December 14, 2008” (document d4). I understand that.

  When the period acquired in step S2000 and the period acquired in step 2010 overlap, the technical term of the community is stored in the index storage unit 130. In the above-described example, since the appearance period of the technical term in the community overlaps with the posting period of the member, the technical term “XX language” is stored in the index storage unit 130.

  The second embodiment has been described above. In the present embodiment, the temporal condition of the appearance period of words that is not considered in the first embodiment is considered. As a result, it is possible to further improve the accuracy when setting a word closely related to the community as a word characterizing a member strongly related to the community.

  The preferred embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to such examples. It is obvious that a person having ordinary knowledge in the technical field to which the present invention pertains can come up with various changes or modifications within the scope of the technical idea described in the claims. Of course, it is understood that these also belong to the technical scope of the present invention.

  For example, in the above embodiment, the chi-square value is used as a value indicating the relationship between the community and the member, the member and the word, and the word, but the present invention is not limited to such an example. For example, mutual information may be used to calculate the strength of relations between words.

  Further, when recalculating the bias score excluding general terms, the threshold value and the appearance frequency may be calculated again.

  Moreover, although the word which characterizes a community is extracted for every community, it is not limited to this example, You may make it extract the word which characterizes this community about several related communities.

  Further, for example, each step in the processing of the technical term extraction device 1 of the present specification does not necessarily have to be processed in time series in the order described as a flowchart. That is, each step in the process of the technical term extraction device 1 may be executed in parallel even if it is a different process.

  Further, it is possible to create a computer program for causing hardware such as the CPU, ROM and RAM incorporated in the technical term extraction device 1 and the like to perform the same functions as the components of the technical term extraction device 1 described above. A storage medium storing the computer program is also provided.

DESCRIPTION OF SYMBOLS 1, 2 Technical term extraction apparatus 10 Input part 20 Morphological analysis part 30 Bias score calculation part 40 General term extraction part 50 Index extraction part 60 Confirmation display part 70 Document storage part with category 80 Morphological temporary storage part 90 Co-occurrence list temporary storage part 100 appearance number temporary storage unit 110 bias score temporary storage unit 120 general term temporary storage unit 130 index storage unit

Claims (9)

A morphological analysis unit that performs a morphological analysis on a document input in accordance with a contributor's operation;
A bias score calculation unit that calculates a bias score between words included in the document, between a word and a contributor, between a word and a posting destination group to which the contributor belongs;
A general term extraction unit that extracts a general term included in the document according to a value of the bias score;
An index extraction unit for extracting a keyword indicating an individual characteristic by removing the general term extracted by the general term extraction unit from the document;
A technical term extraction device comprising:   The technical term according to claim 1, further comprising: a storage unit that stores a document input in response to a contributor's operation, a contributor, and a posting destination group to which the contributor belongs. Extraction device. The bias score calculation unit
The technical term extraction device according to claim 1, wherein the bias score is calculated by a chi-square value. The general term extraction unit includes:
Of the combinations of a poster and a word or a posting destination group to which the poster belongs and a word, the bias score is less than or equal to a predetermined value, and a word related to a plurality of posters or a plurality of posting groups The technical term extraction device according to claim 1, wherein the terminology is extracted as a general term. The bias score calculation unit
Excluding the word extracted as the general term, recalculating the bias score,
The general term extraction unit includes:
Of the combinations of a poster and a word or a posting destination group to which the poster belongs and a word, the bias score value is less than or equal to a predetermined value, and a word related to a plurality of posters or a plurality of posting destination groups 5. The technical term extraction device according to claim 4, wherein the terminology is extracted again as a general term. The index extraction unit
The technical term extraction device according to claim 1, wherein a word having a bias score value between a poster and a word that is equal to or greater than a predetermined value is extracted as a keyword indicating an individual characteristic. The index extraction unit
A word indicating the characteristics of the posting destination group is extracted, and the word indicating the characteristics of the posting destination group having a bias score value between the poster and the posting destination group to which the poster belongs belongs to The technical term extraction device according to claim 1, wherein the technical term extraction device is extracted as a keyword indicating a feature. The index extraction unit
Among the words indicating the characteristics of the extracted posting destination group, when a period in which a document including the word is posted corresponds to a period in which the poster has posted in the posting destination group, the word is an individual characteristic. The technical term extraction device according to claim 7, wherein the terminology is extracted as a keyword indicating. Computer
A morphological analysis unit that performs a morphological analysis on a document input in accordance with a contributor's operation;
A bias score calculation unit that calculates a bias score between words included in the document, between a word and a contributor, between a word and a posting destination group to which the contributor belongs;
A general term extraction unit that extracts a general term included in the document according to a value of the bias score;
An index extraction unit for extracting a keyword indicating an individual characteristic by removing the general term extracted by the general term extraction unit from the document;
A program for functioning as a terminology extraction device, comprising:

Images