JP2008033503A - Highlight device and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To easily decide whether an inputted sentence is a sentence of a positive image or a sentence of a negative image. <P>SOLUTION: This highlight device has: a storage means 3 storing a dictionary of words of the positive image and the negative image; a retrieval means 21 retrieving a word of the dictionary from the inputted sentence; and a highlight means 22 performing different highlight of the retrieved word by the image. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention uses an adverb (or adjective) to determine whether a sentence written or spoken by an author or speaker has a positive image or a negative image, and whether it was written or not. The present invention relates to a highlighting device and a program for highlighting (or adjective). For example, a positive image adverb (or adjective) is highlighted in blue and a negative image adverb (or adjective) is highlighted in red. In addition, there is also a function that points out misuse of a user's adverb (or adjective) by using an inconsistent situation where adverbs (or adjectives) of different images are used simultaneously.

  Moreover, in this invention, it can utilize in document browsing systems, such as a WEB browser, and can be integrated and utilized as a toolbar of Internet Explorer.

Conventionally, as a highlighting device, there has been a highlighting device that judges a word in a title as an important word and highlights it in the text (see Patent Document 1).
JP 2004-280176 A

  There is no technique for highlighting an adverb in a text based on a positive or negative image of an author or a speaker, which highlights a word in a conventional title in the text.

  The present invention is intended to solve the above-mentioned problems, and in the sentence written or spoken by the author or speaker, the adverb (or adjective) is used to indicate whether it was written with a positive image or a negative image. The purpose is to highlight the adverbs (or adjectives) differently depending on the image.

  FIG. 1 is an explanatory diagram of a highlight display device of the present invention. In FIG. 1, 1 is an input unit (input unit), 2 is a processing unit (processing unit), 3 is a storage unit (storage unit), 4 is an output unit (output unit), 21 is a search unit (search unit), 22 Is an emphasis display section (emphasis display means).

  The present invention has the following means in order to solve the conventional problems.

  (1): storage means 3 for storing a dictionary of positive image and negative image words, search means 21 for searching words in the dictionary from inputted sentences, and emphasizing the searched words differently depending on the images. And highlighting means 22 for displaying. Therefore, it is possible to easily determine whether the input sentence is a positive image sentence or a negative image sentence by highlighting the word.

  (2): In the highlighting device of (1), the dictionary is an adverb or adjective dictionary. For this reason, the image of a sentence can be easily judged by an adverb or an adjective.

  (3): In the highlighting device of (1) or (2), the highlighting means 22 highlights the inputted sentence differently depending on the image. Therefore, it is possible to easily determine whether the input sentence is a positive image sentence or a negative image sentence.

  (4) In the highlighting device of (2) or (3), the adverb or adjective dictionary is created by registering words in descending order of appearance frequency. Therefore, the necessary adverbs or adjectives can be easily registered in the dictionary.

  (5): In the highlighting device according to any one of (2) to (4), the creation of the adverb or adjective dictionary is performed by rearranging words in the order of meaning and displaying the rearranged words together with their images. And do it. For this reason, it is possible to easily infer an image of an unknown word from the images of previous and subsequent words.

  (6); The highlighting device according to any one of (1) to (5) further includes a contradiction detecting unit that detects the input sentence in which words of a positive image and a negative image appear simultaneously. For this reason, misuse of words, such as an adverb or an adjective, can be pointed out to a user.

  (7): The highlighting device according to any one of (1) to (6), comprising output means 4 for displaying the inputted sentence on the screen, and selecting the highlighting on a toolbar of the screen of the output means 4 The selection part to perform is provided. For this reason, it is possible to highlight the sentence displayed on the screen by instructing the selection unit by clicking or the like.

  (8): In the highlighting device of (7), the highlighting means 22 does not perform the highlighting when the file size of the input sentence is larger than a predetermined value. For this reason, it is possible to prevent a process in which the file is too large and takes too much time.

  (9): In the highlighting device of (7) or (8), text processing of the input sentence is performed in an interpreted programming language capable of character string processing including a regular expression. For this reason, the system construction and maintenance of the highlighting apparatus are easy, and it is convenient for character string processing including regular expressions.

  (10): In the highlighting device according to any one of the above (1) to (9), the dictionary file as the dictionary is provided in the exchangeable storage means 3, and how each word is highlighted in the dictionary file Define what. For this reason, if a file is created in the same way other than adverbs and adjectives, it can be highlighted in the same manner, which is convenient.

  (11): In the highlighting device according to any one of (1) to (10), the highlighting means 22 is provided in a replaceable manner, and the highlighting means 22 is created in an interpreted programming language. For this reason, it is possible to have a program as a text file in an interpreted program language such as perl in addition to the toolbar main body.

  (12): storage means 3 for storing a dictionary of positive image and negative image words, search means 21 for searching words in the dictionary from input sentences, and emphasizing the searched words differently depending on the images. The highlighting means 22 for displaying is a program for causing a computer to function. Therefore, by installing this program in a computer, it is possible to easily provide a highlighting device that can easily determine whether an input sentence is a positive image sentence or a negative image sentence.

  The present invention has the following effects.

  (1): In order to highlight the word searched from the input sentence differently depending on the image by the highlighting means, whether the input sentence is a positive image sentence or a negative image sentence by word emphasis display. Can be easily determined.

  (2): Since the dictionary is a dictionary of adverbs or adjectives, the image of a sentence can be easily determined by adverbs or adjectives.

  (3): Since the input sentence is highlighted differently depending on the image by the highlighting means, it is possible to easily determine whether the input sentence is a positive image sentence or a negative image sentence.

  (4): Since a dictionary of adverbs or adjectives is created by registering words in descending order of appearance frequency, necessary adverbs or adjectives can be easily registered in the dictionary.

  (5): Since the dictionary of adverbs or adjectives is created by rearranging words in order of significance and displaying the rearranged words together with their images, it is easy to create an image of unknown words from the images of previous and subsequent words. It can be analogized to.

  (6) Since the contradiction detection means detects the inputted sentence in which the words of the positive image and the negative image appear at the same time, the misuse of words such as adverbs or adjectives can be pointed out to the user.

  (7): Since the selection unit for selecting highlighting is provided on the toolbar of the screen of the output means, it is possible to highlight the sentence displayed on the screen by instructing the selection unit by clicking or the like.

  (8): When the file size of the input sentence is larger than the predetermined value by the highlighting means, highlighting is not performed, so that it is possible to prevent the file from being too large and time-consuming. .

  (9): Since the text processing of the input sentence is performed in an interpreted programming language capable of character string processing including regular expressions, the system construction and maintenance of the highlighting device is easy, and the character string processing including regular expressions It is convenient to use.

  (10): A dictionary file, which is a dictionary, is provided in the exchangeable storage means, and in order to define how each word is highlighted in the dictionary file, if a file is similarly created in addition to adverbs and adjectives, It can be highlighted and convenient.

  (11): The highlighting means is provided in a replaceable manner, and the highlighting means is created in an interpreted programming language, so that the program as a text file can be written in an interpreted programming language such as perl in addition to the toolbar body. Can have.

  In sentences written or spoken by the author or speaker, it is possible to determine whether an article was written with a positive image or a negative image using an adverb. Then, the adverb is highlighted. For example, a positive image adverb is highlighted in blue and a negative image adverb is highlighted in red. It is also possible to point out the misuse of the user's adverbs by using the contradictory situation where adverbs of different images are used simultaneously in one sentence.

  For example, if you use a negative image adverb, you can be careful not to use such a negative image word. The adverb is directly related to the verb of the main expression of the sentence, and directly expresses the image of the author or the speaker, and is useful for extracting and displaying the image of the author or the speaker.

  In order to identify whether each adverb has a positive or negative image, a dictionary showing whether each adverb has a positive or negative image is created and specified based on this dictionary. This dictionary was constructed by first checking the adverb appearance rate in general sentences and registering it in the dictionary from adverbs with a high appearance rate, thereby efficiently creating a highly comprehensive dictionary.

  In creating this dictionary, teacher data to which a tag indicating whether each adverb has a positive or negative image in a sentence may be created and specified by learning from the data. Also, the perl program (program language) is used for internal text processing, and visual c (program language) is used for the toolbar. In addition, the program is general-purpose, and two files, a list of words to be highlighted and a definition of the notation of the list, are created outside the program, and these two files are changed. This makes it possible to create a toolbar that performs various processes.

(1): Description of Highlight Display Device FIG. 1 is an explanatory diagram of a highlight display device. In FIG. 1, the highlighting device includes an input unit 1, a processing unit 2, a storage unit 3, and an output unit 4. The processing means 2 is provided with a search unit 21 and an emphasis display unit 22.

  The input unit 1 is input means for inputting information such as a keyboard, a mouse, and a reading device. The processing means 2 is processing means (apparatus) that performs search, detection (extraction), word rearrangement, adverb (or adjective) dictionary creation, display processing, and the like. The storage unit 3 is a dictionary (storage means) that stores adverbs (or adjectives) with an evaluation such as a positive image (p), a negative image (n), and a neutral image (0). The output unit 4 is output means for outputting information such as a display device and a printer. The search unit 21 is search means for searching for adverbs (or adjectives) in the dictionary in the input sentence. The emphasis display unit 22 is emphasis display means for emphasizing the searched adverbs (or adjectives) differently depending on the image.

  FIG. 2 is an explanatory diagram of a highlighting device that points out contradiction. In FIG. 2, the highlighting device that points out the contradiction is provided with an input unit 1, a processing means 2, a storage unit 3, and an output unit 4. The processing means 2 includes a search unit 21, an emphasis display unit 22, and a contradiction detection unit 23.

  The contradiction detection unit 23 is a contradiction detection unit that detects and points out that adverbs (or adjectives) of different images are simultaneously used in one sentence (sentence). The highlighting device for pointing out the contradiction in FIG. 2 is obtained by adding a contradiction detection unit 23 to the configuration in FIG. 1, and the other configuration is the same as that in FIG.

(2): Explanation of Highlight Display Processing FIG. 3 is a flowchart of highlight display processing. Hereinafter, a description will be given according to the processes S1 to S3 in FIG.

  S1: A sentence is input by the input unit 1.

S2: The search unit 21 searches for an adverb (or adjective) in the dictionary of the storage unit 3 in the input sentence.
At this time, the search unit 21 has the following two search methods.

  (1) Search for adverbs (or adjectives) in the dictionary as character strings in the input sentence.

  (2) Perform morphological analysis (word segmentation) in the input sentence, find the part of speech of the word, and search for the obtained adverb (or adjective) from the dictionary. Less).

  S3: The emphasis display unit 22 displays the retrieved adverbs (or adjectives) in different emphasis depending on the image and outputs them to the output unit 4.

  FIG. 4 is a flowchart of highlighting processing for indicating a contradiction. Hereinafter, a description will be given according to the processes S11 to S15 of FIG.

  S11: A sentence is input by the input unit 1.

S12: The search unit 21 searches for an adverb (or adjective) in the dictionary of the storage unit 3 in the input sentence.
At this time, the search unit 21 has the following two search methods.

  (1) Search for adverbs (or adjectives) in the dictionary as character strings in the input sentence.

  (2) Perform morphological analysis (word division) in the input sentence, find the part of speech of the word, and search the obtained adverb (or adjective) from the dictionary (the morphological analysis reduces the extraction error).

  S13: The contradiction detection unit 23 detects whether a positive image and a negative image adverb (or adjective) are used at the same time in the input sentence. In this detection, if the positive image and the negative image adverb (or adjective) are used at the same time (there is a contradiction), the process proceeds to step S14. If they are not used simultaneously (there is no contradiction), the process proceeds to the process S15. .

  S14: The emphasis display unit 22 performs display (such as displaying at the beginning of a sentence) where there is misuse (inconsistency) of an adverb (or adjective).

  S <b> 15: The highlighting unit 22 displays the searched adverbs (or adjectives) differently depending on the image and outputs them to the output unit 4.

(3): Explanation of highlighting Highlighting is to change the color of the character, to change the background color of the character or to shade, to change the font of the character (bold, italic, etc.), to underline This can be done by adding a symbol or the like on the character, enclosed in parentheses or enclosed in parentheses.

  For example, if an adverb in a sentence is highlighted in color, the adverb of the positive (positive) image (p) is blue, the adverb of the negative (negative) image (n) is red, and the neutral image (0) Adverbs can be gray or the like.

  In highlighting, the entire sentence or article can be highlighted with an image, and the title of the document or article itself can be highlighted with an image. It can also be displayed by adding symbols such as p, n, 0, etc. before or after the sentence.

(4): Description of the morpheme analysis system In order to divide Japanese into words, a morpheme analysis system performed by the word extraction unit 22 is required. Here, ChaSen will be explained (published on the morphological analysis system Chasen http://chasen.aist-nara.ac.jp/index.html.jp developed at Nara Institute of Technology).

  This splits the Japanese sentence and also estimates the part of speech of each word. For example, if “go to school” is entered, the following results can be obtained.

School Gakkou School Noun-General To He To particle-Case particle-General Go Iku Go Verb-Independence
In this way, each line is divided so that one word is included, and reading and part-of-speech information are given to each word. Here, the divided words are used as extraction units, and the assigned parts of speech are used to specify the extracted expressions.

(5): Explanation of adverb dictionary construction In addition to the degree and behavioral aspects of Japanese adverbs, there are things that express psychological attitudes such as the speaker's feelings and judgments about certain propositions and matters. It is known to do. However, in research on sensitivity information such as evaluation expression extraction and reputation analysis in linguistic processing, adverbs are often excluded from the processing target. Thus, few studies have so far investigated the importance of adverbs in language processing. This is because adverbs are not directly related to the propositional content of sentences in information retrieval, and are often excluded as stop words when indexing is necessary for retrieval, and the same applies to processes such as reputation analysis and opinion extraction. The cause seems to be that the processing has been performed. In addition, since the form of adverbs is not fixed and the appearance rate of each adverb is not so high, even the extraction of important words in reputation analysis is rarely extracted as an important word for determining evaluation. Therefore, in the case of adverbs, unlike other evaluation expressions, it is desirable to define them in the form of a dictionary created by hand in advance, instead of automatic acquisition or automatic classification. Therefore, we built an adverb dictionary describing the speaker's attitude. Here, we show that the dictionary we built is valid as data for reputation analysis. We conducted the following investigations to confirm the effectiveness of the constructed dictionary. First, dictionary coverage was investigated using WEB text. Next, evaluation classification using dictionary data was performed on randomly selected WEB data, and the accuracy was confirmed. Finally, we examined the effectiveness of this adverb dictionary.

  Below, we built an adverb dictionary describing the speaker's attitude. A method for constructing this adverb dictionary will be described.

(1) Target adverbs In building an adverb dictionary, we referred to the following two existing dictionaries. One is an EDR Japanese word dictionary.

(Reference for EDR Japanese Word Dictionary)
Takano Kanno, Yoshio Nakao, Yoko Nagasawa, Ayumi Ogasawara (1995). “Conceptual System in EDR Electronic Dictionary.” Proceedings of the 1st Annual Conference of the Language Processing Society, pp. 197 −200.
The other is ipadic used in ChaSen.

(Literature of ipadic used in ChaSen)
Yuji Matsumoto, Kei Kitauchi, Tatsuo Yamashita, Yoshitaka Hirano, Hiroshi Matsuda, Kazuma Takaoka, Masayuki Asahara (2000). “Instruction Manual for Japanese Morphological Analysis“ Chaya ”version 2.2.1.”.
In these two types of dictionaries, all words defined as adverbs were extracted. As a result, 2592 adverbs were obtained in expression form. In the future, this set of adverbs will be called all adverbs for convenience.

(2) Explanation of adverb tagging Next, we decided to manually add information about the speaker's attitude to the extracted adverb. Speaking of the attitude of a speaker, whether you think it ’s good or bad for something, whether it ’s a positive or negative attitude, whether you ’re certain or not, There are various speaker attitudes, such as attitudes related to confidence. The information we tagged this time is about attitudes related to affirmative negativity. The adverb used when a speaker thinks that an event is desirable or desirable is an adverb with an affirmative rating and is tagged with “p”. Conversely, the adverb used when a speaker thinks that an event is undesirable or undesirable is an adverb with a negative evaluation and is tagged with “n”. Those that are neither plus nor minus, and those that become plus or minus depending on the context, are neutral adverbs and are tagged with “0”. Hereinafter, an adverb with this evaluation tag is called an adverb with evaluation.

  For this tagging, I referred to the “Modern Adverbic Dictionary” (Ryofumi Tobita, Hideko Asada (1994). Tokyodo Publishing). This dictionary evaluates the emotional image of an adverb independent of context based on seven levels of image values. Words that can be evaluated either neutrally or depending on the situation in which the evaluation is used have an image value of 0 and are neither positive nor negative. In this dictionary, the adverb was evaluated in seven levels, but our evaluation was in three levels: p, 0, and n. This is because if the evaluation of this dictionary is adopted as it is, the evaluation classification becomes too fine, and it is determined that the burden of checking the classification accuracy will increase in the situation where the accuracy of the evaluation is not known.

  The target of tagging first is 883 adverbs described in the adverb usage dictionary among all adverbs. For adverbs with ambiguity, the number of meanings of the word and the circumstances in which each meaning of the word was used were judged. For example, when most of the meanings are used as neutral evaluation, this adverb was given a neutral evaluation. In addition, when both positive and negative evaluations exist among multiple meanings, this adverb is also given a neutral evaluation.

(3) Explanation of survey of coverage rate First, we investigated the coverage rate of adverbs with evaluations for all adverbs. Here, the coverage is not the ratio of sentences containing adverbs to the full sentence set, but the ratio of sentences containing adverbs in the newly created dictionary to the set of sentences containing adverbs contained in all adverbs. .

  If the coverage of the created dictionary is high, even if you tag an adverb that has not been tagged yet, you can not expect an effect that is worth the effort. On the other hand, when the coverage is low, it is necessary to add an adverb in addition to the adverb described in the dictionary for tagging.

  For the investigation of the coverage rate, 5,635,346 sentences extracted from the WEB text were prepared. This was first divided in half and used as closed data and open data in the coverage survey. First, string search for all adverbs was performed on closed data. Next, the character string matching search was similarly performed for the adverbs with evaluation. The reason for performing the character string match search is that as many sentences as possible correspond to the sentences that may contain adverbs. Usually, when investigating the appearance ratio for each part of speech, it is performed on the morphological analysis, but some adverbs targeted this time are not recognized as adverbs by the morphological analysis. Therefore, it is expected that the coverage of adverbs will decrease in the survey after morphological analysis.

Table 1 shows the appearance rate when a string match search is performed on all adverbs and evaluated adverbs for closed data.

Table 1: WEB data (half): Adverb appearance rate in 2817672 sentences ┌──────────┬───┬──────────┐
│All adverbs │74.10% │ (2087938/2817672) │
│Evaluated adverb │47.76% │ (1345796/2817672) │
├───────────┼───┼──────────┤
│ Coverage rate for all adverbs │ 64.45% │ (1345736/2087938) │
└───────────┴───┴──────────┘ The ratio of adverbs with evaluations to all adverbs was 64.45%.

  At this point, the coverage was low and it was necessary to add as many new tags as possible to accommodate as many adverbs as possible. In order to select adverbs that should be newly evaluated, adverbs with an appearance ratio of 0.01% or more are extracted from adverbs of all adverbs, and a list of adverbs that are not included in the adverbs with evaluations is created.

Table 2: Adverbs that are included in all adverbs and not included in adverbs with evaluation (top 30)
┌───┬───┬─────┬────┬──┬─────┐
│ adverb │ frequency │ appearance rate │ adverb │ frequency │ appearance rate │
├───┼───┼─────┼────┼──┼─────┤
│Yo │153717│ 5.455461 │Once │9019│ 0.320087 │
│ Ito │ 59405 │ 2.108301 │ Abbreviated │6421│ 0.227883 │
│Eye │ 55248│ 1.960768 │General │6369│ 0.226038 │
│Ika │ 37263 │ 1.322475 │So far │6287│ 0.223127 │
│Chira │ 33609 │ 1.192793 │ All │6124│ 0.217343 │
│ │ 26095 │ 0.926119 │ tat │5716│ 0.202863 │
│ Roll │ 21767 │ 0.772517 │ Both │5423 │ 0.192464 │
│ Early │ 17294 │ 0.613769 │ Certainly │5160 │ 0.183130 │
│10,000 │ 17231 │ 0.611533 │ Firmly │4603 │ 0.163362 │
│ Mochi │ 16504 │ 0.585732 │ Since │4117 │ 0.146114 │
│After │ 10132│ 0.359588 │Why│4105│ 0.145688 │
│ No │ 9848 │ 0.349508 │ Book │ 3964 │ 0.140684 │
│ Everyone │ 9797 │ 0.347698 │ Hata │3854│ 0.136780 │
│Kaku │ 9543│ 0.338684 │Mainly │3819│ 0.135537 │
│Poi │ 9216│ 0.327079 │Good │3803│ 0.134970 │
└───┴───┴─────┴────┴──┴─────┘

Table 3: Adverb conditions to be removed from all adverbs ┌──────────┐
│Consecutive form of adjectives |
│The stem of adjective │
│Two or less hiragana characters │
│Less than 2 katakana characters │
│ 1 Kanji character │
│Noun + particle │
│Common with other parts of speech │
│Some other words │
└───────────┘

Table 4: Web data (half): Adverb appearance rate in 2817672 sentences (after correction)
┌───────────┬───┬──────────┐
│All adverbs (before revision) │74.10% │ (2087938/2817672) │
│All adverbs (after correction) │53.64% │ (1511389/2817672) │
│Evaluated adverb │47.76% │ (1345796/2817672) │
├───────────┼───┼──────────┤
│Coverage rate for all adverbs│89.04% │ (1345736/1511389) │
└───────────┴───┴──────────┘

It can be seen from Table 2 that many adverbs as shown below are included.
・ Adjective (dynamic) verb stems ・ Two or less hiragana characters ・ Two or less katakana characters ・ One kanji character ・ Nouns + particles These adverbs may match part of a word when performing a string match search high. As a result, there is a possibility that an erroneous result that it is included in a very large number of sentences may appear. Therefore, the adverbs corresponding to the conditions in Table 3 existing only in all adverbs were manually removed.

  Table 4 shows the appearance ratio of adverb before and after the correction to remove the adverb that matches the condition from all adverbs. Since the correction is performed on adverbs that appear only in all adverbs, the number of adverbs with evaluation does not change.

  When adverbs that met these conditions were removed from all adverbs, the coverage of all adverbs decreased by 20.46% to 53.64%. When the ratio of adverbs with evaluation to all adverbs was recalculated with this value, the coverage was 89.04%.

  When adverbs with an appearance ratio of 0.01% or more included in all adverbs were extracted, and adverbs that met the conditions in Table 3 were removed from them, the remaining adverbs were 105 words. Table 5 shows examples of remaining adverbs.

Table 6 shows the coverage for all adverbs after adding adverbs. With the addition of adverbs, the coverage of all adverbs with evaluated adverbs is 97.48%. This is a numerical value that is sufficiently practical.

Table 5: New adverbs (top 30)
┌─────┬──┬─────┬┬────┬──┬─────┐
│ adverb │ frequency │ appearance rate │ adverb │ frequency │ appearance rate │
├─────┼──┼─────┼┼────┼──┼─────┤
│ Once │9019 │ 0.320087 │ Again │2117 │ 0.075133 │
│Previous │6287│ 0.223127 │Only │1989│ 0.070590 │
│All │6124│ 0.217343 │Many │1969│ 0.069880 │
│Both │5423│ 0.192464 │Night│1876│ 0.066580 │
│Secure │4603│ 0.163362 │That is │1785│ 0.063350 │
Since │4117│ 0.146114 │Relatively │1734│ 0.061540 │
│Why │4105│ 0.145688 │Soon │1631│ 0.057885 │
│Mainly │3819│ 0.135537 │Only │1592│ 0.056501 │
│Properly │3640│ 0.129185 │Preliminary │1577│ 0.055968 │
│Instead │2558│ 0.090784 │Thanks │1509│ 0.053555 │
│ Many │ 2549 │ 0.090465 │ Carefully │ 1311 │ 0.046528 │
│Original │2469│ 0.087626 │Also │1310│ 0.046492 │
│Like this │2421 │ 0.085922 │ thus │1277 │ 0.045321 │
│ Correctly │ 2299 │ 0.081592 │ Complete │ 1258 │ 0.044647 │
│ One layer │ 2176 │ 0.077227 │ Loose │ 1235 │ 0.043831 │
└─────┴──┴─────┴┴────┴──┴─────┘

Table 6: WEB data (half): 2817772 Adverb appearance rate in sentence (after adverb addition)
┌───────────┬────┬──────────┐
│All adverbs (before revision) │ 74.10% │ (2087938/2817672) │
│All adverbs (after correction) │ 53.64% │ (1511389/2817672) │
│ adverb with evaluation (before addition) │ 47.76% │ (1345796/2817672) │
│ adverb with evaluation (after addition) │ 52.29% │ (1473297/2817672) │
├───────────┼────┼──────────┤
│ Coverage rate for all adverbs │ 97.48% │ (1473297/1511389) │
└───────────┴────┴──────────┘

(4) Additional explanation of adverbs with evaluation In the previous section, adverbs that appeared to be part of the string by mistake were removed from adverbs with an appearance rate exceeding 0.01% extracted from all adverbs. The remaining 105 adverbs were newly tagged.

  When adding tags based on positive, negative, and neutral criteria to newly added adverbs, the classification vocabulary table (National Institute for Japanese Language (Ed.) (2004). Dai Nippon Library) was used. For adverbs existing in the classification vocabulary table, numbers in the classification vocabulary table are assigned, and the adverbs are rearranged in the order of the numbers. Thereby, the adverb with the evaluation tag in advance and the newly added adverb are rearranged in the order of the meaning.

The following paper is an example of dictionary construction using semantic sorting.
Masaki Murata, Kyoko Kanzaki, Kiyotaka Uchimoto, Mao Ai, Hitoshi Isahara (2000). “Semantic sort msort * Example of dictionary construction and semantic tagging corpus creation and information presentation system.” Natural language processing , pp. 51-66.
In this paper, an example of dictionary construction using semantic sorting is given. The method of assigning numbers in the above categorized vocabulary table and rearranging adverbs in the order of those numbers is the “Semantic Sorting” proposed in the above paper. Can be seen as a method using the method of "." Actual examples are listed in Table 7.

  By using the classification vocabulary table, it became possible to refer to adverbs with pre-evaluated tags, and tagging could be done efficiently in a relatively short time. Adverbs with * in Table 7 are adverbs to be newly added. As can be seen from the table, there are already adverbs with tags attached above and below (around), so we were able to tag them with reference to those tags (if the top and bottom were the same, we could do the same tagging). In this case, a tag of “0” was assigned to each adverb.

  All adverbs that are not included in the categorized vocabulary table are judged manually and given a tag.

Table 7: Example of tagging using a classification vocabulary table ┌───────┬─────────┐
│Classified vocabulary table number│Adverb evaluation │
├───────┼─────────┤
│3100030201 │Never 0 │
│3100030201 │Never 0 │
│3100040102 │ Certainly * “0” |
│3100050101 │ Apparently p │
│3100050102 │Maybe 0 │
│3100050103 │Probably n │
│3100050104 │Okata 0 │
│3100050105 │ Certainly * “0” |
│3100050106 │Generally 0 │
│3100050201 │First 0 │
│3100050302 │Okay 0 │
│3100050502 │Exactly n │
│3100050504 │Anything 0 │
└───────┴─────────┘

(5) Explanation of Adverb Dictionary Evaluation a) Adverbic Dictionary Evaluation Experiment An evaluation experiment was conducted to investigate the accuracy of the created adverb dictionary with evaluation. In the evaluation experiment, 1000 sentences are randomly extracted from the WEB text, and when an adverb corresponding to the adverb dictionary is included, an evaluation tag is output and classified into a positive sentence, a negative sentence, and a neutral sentence. Is.

Here, the positive sentence and the negative sentence are defined as follows.
If the writer of the sentence sees the content expressed in the sentence as favorable or good, it will be called a positive sentence. On the other hand, if the writer of the sentence regards the content expressed in the sentence as being unfavorable or bad, it will be called a negative sentence. If it is neither, make a neutral sentence. Based on this evaluation standard, the extracted sentence was judged manually and the correct answer was created.

  Next, using the created dictionary for each sentence, if there is an adverb included in the dictionary in the target sentence, positive “p”, negative “n”, neutral “0” based on the dictionary Classification was performed. Thereafter, whether each of the classifications p, n, 0 by the dictionary is correct or not was determined.

In the evaluation experiment, the following two types were used, using 883 words of adverb before adding data to the dictionary and 988 words after adding data.
1. 1,000 sentences-883 words before addition
2. 1,000 sentences-988 words after addition

Table 8: Results in the pre-adverb dictionary ┌───────┬────────┬────────┬──┐
│ │ Reproducibility │ Accuracy │ F value │
├───────┼────────┼────────┼──┤
│ Total accuracy rate │85% (175/205) │85% (175/205) │85% │
│ Neutral only │89% (151/170) │94% (151/160) │91% │
│Affirmation only │78% (18/23) │55% (18/33) │64% │
│Negation only │50% (6/12) │50% (6/12) │50% │
│Affirmation, Denial only│69% (24/35) │53% (24/45) │60% │
└───────┴────────┴────────┴──┘

Table 9: Results in dictionary after adverb ┌───────┬────────┬────────┬──┐
│ │ Reproducibility │ Accuracy │ F value │
├───────┼────────┼────────┼──┤
│ Total accuracy rate │86% (200/233) │86% (200/233) │86% │
│ Neutral only │89% (166/187) │95% (166/175) │92% │
│Affirmation only │84% (27/32) │60% (27/45) │70% │
│Negation only │50% (7/14) │54% (7/13) │52% │
│Affirmation, Denial only│74% (34/46) │59% (34/58) │65% │
└───────┴────────┴────────┴──┘

b) Results and discussion of evaluation experiment Tables 8 and 9 show the results of the experiment.
Table 8 shows the result before adding a new adverb to the adverb dictionary, and Table 9 shows the classification result when using the dictionary after adding the adverb.

  In Tables 8 and 9, the total correct answer rate was determined regardless of the output in which of the three categories of positive, negative or neutral sentences with adverbs included in the dictionary. Is. Only affirmation and negation are calculated from the accuracy and recall of only affirmation, denial, and neutral judgments using a dictionary whose output was affirmative or negative. Neutral only, positive only, and negative only indicate accuracy and recall for each classification by dictionary. For example, only for affirmation, accuracy and recall are obtained for a sentence in which the corresponding sentence is determined by a positive evaluation and a dictionary.

  The accuracy is a ratio of correct answers among items classified by the dictionary for correct answers. The recall is the percentage of correct answers within the classified item. Further, the F value was calculated as a standard for performance evaluation. Here, the F value is a harmonic average of accuracy and recall.

  Sentences containing adverbs included in the dictionary were 205 sentences in the dictionary before addition and 233 sentences in the dictionary after addition of the adverb.

  The results of classification of speaker attitudes by adverb dictionary were positive, negative or neutral, with an accuracy rate of 85% before addition and an accuracy rate of 86% after addition.

  The F value of those classified as positive or negative by the dictionary was 60% in the dictionary before addition and 65% in the dictionary after addition of the adverb. Considering that the baseline of the three categories is 33%, it can be said that the accuracy is relatively high.

  The determination of affirmation was 64% before the addition and 70% after the addition. On the other hand, the F value in the negative classification is 50% before the addition and 52% after the addition, which is lower than the positive attitude classification.

  Consider an example of incorrect dictionary output. The following example is based on the result of using a dictionary after adding adverbs to 1000 sentences.

  First, we will consider errors that have been classified as either positive or negative. An example of this type of error is shown below. What is written after the dictionary item of each item is affirmative, negative, or neutral judgment output by the dictionary. The correct answer indicates an answer determined manually.

・ No affirmation-Dear Toshio Dear, thank you very much.

(Dictionary: negative, correct: neutral)
-Ken Wilber, a young thinker who has [probably] marked the times, wrote the book "Structure of Evolution" (translated by Taro Matsunaga, Shunjusha).

(Dictionary: negative, correct: neutral)
-Depending on the content, we may not be able to reply.
[Ashikazu] Please note.

(Dictionary: negative, correct: neutral)
-As usual, while concentrating, the consciousness was concentrated in Tanda, and the Toku, Toku and the generator started to move, and when a red fireball was formed, it was raised and energy was sent from the brainstem.

(Dictionary: positive, correct: neutral)
-Since statistics can be national averages, regional ones, and industry-specific ones, [Naturally], as Mr. Hata said, national statistics, statistics in the Kanto region, and the Tokyo Association. Select some of the data to make.

(Dictionary: positive, correct: neutral)
In these sentences, adverbs are used as an expression of confidence in the situation of the speaker or as part of a specific phrase.

  In the former case, these adverbs do not have much work to show the attitude of speaker affirmation or denial, and “natural” and “probably” are functions that indicate how the speaker thinks about the likelihood of a situation. Seems to be strong.

  Adverbs may also indicate the certainty of such a situation, which will be added to the dictionary in the future. It has been found that there are cases where adverbs related to such a certainty do not feel the aspect of attitude such as affirmation or denial.

  On the other hand, “Ashikazu” and “Komo” are adverbs typically found in greeting expressions. These adverbs have little meaning and are used as one of the standard expressions. Even for such expressions, it is difficult to read the attitude of the speaker to the situation.

Next, an example in which affirmative negation is reversed is shown.
-Oh, [Finally] I'm done.

(Dictionary: positive, correct: negative)
-Moreover, the number of applications for reservations is far higher than planned, and in the region, such publications [probably] have come to the news that they may not have come.

(Dictionary: negative, correct answer: positive)
-To the performance full of sensibility beyond technology,
The audience of about 700 people who filled the venue, including school officials, current students, graduates, and parents, was fascinated.

(Dictionary: negative, correct answer: positive)
In spite of the negative sentences, the ones that were judged positive by the dictionary were those that used adverbs that seemed less appropriate. For example, regarding “Finally”, “Finally” is better used to indicate the result of continuous action or action such as achievement. Therefore, a positive attitude can be read from “Finally”. In this case, it seems more appropriate to use an adverb such as “Totou”.

  Also, “probably” indicates the degree of confidence in the situation as described in the absence of affirmation and denial, and is used here to indicate a speaker's attitude different from affirmation and negation. Therefore, from this “probably”, negation cannot be read much.

  “Possibly” is explained as follows in the “Modern Adverbial Dictionary”, which was referred to when tagging a dictionary.

It often shows how to make a guess about the things that are less desirable to realize, with the suggestion of fears and questions.
When guessing the possibility of a favorable matter, “probably” or “probably” is usually used.

  Therefore, it seems that it was more appropriate to use “Maybe” in this case as well.

  In the future, it will be necessary to give information on the certainty of the situation, distinguish whether it is a matter of affirmation or denial of the situation, or whether it is a matter of confidence in the situation, and consider its priority There seems to be.

c) Examination of effectiveness of adverb dictionary Next, the effectiveness of this dictionary was examined. The results will be described.

This dictionary can be used effectively at the following times.
1. When writing a document, check whether it is written negatively.
2. Check that adverbs that are not appropriate for the context are used when creating the document.
3. Automatically determine and display the affirmative / negative of sentences when reading a document.
4. Automatically collect and classify positive and negative sentences.

  First, when creating a document, it is possible to check whether a negative attitude is expressed by using this dictionary. In general, expressing negative attitudes to documents is less desirable because it can hinder the smoothness of communication, except when stating personal feedback such as reviews. By using this dictionary, it is possible to notice this even when the negative attitude is expressed unconsciously, especially for adverbs.

  Next, regarding the indication of adverbs that are not suitable for the context, when using a different method to determine the affirmation and negativity of the sentence, the adverb Take advantage of the high probability that the usage is incorrect or misleading.

  For example, “Finally” and “Probably” were taken up as examples of mistakes in the previous section. However, as in these examples, the affirmation and negativity judged from the context and the affirmation and negativity of adverbs are contradictory. In many cases, the usage is wrong and there are other suitable adverbs. It is then possible to suggest the user to consider other adverbs.

  There are 19 examples out of 1000 sentences that are likely to be used to judge errors in the usage of adverbs, and 19 sentences out of 46 sentences that contain adverbs. Therefore, when only sentences containing adverbs are targeted, it is possible to point out sentences with a probability of 41.30%.

  When reading a document, a system that uses this dictionary and outputs a sentence that contains a negative adverb and a sentence that contains a positive sentence can be considered. As a result, when there is a negative expression, it is possible to determine whether a negative attitude is taken throughout the document or whether it is only primary.

  On the other hand, when affirmative and negative are collected from WEB, sentences and documents expressed as affirmative or negative may be collected using a dictionary. In this case, since 34 sentences are obtained per 1000 sentences, it can be collected with a probability of 3.4%. Although this number is quite low at first glance, it can be used at the initial stage of reputation expression extraction because it is possible to judge affirmation and negativity regardless of the context and field as an advantage of the adverb dictionary.

  Thereby, for example, a use method such as creating a seed for extracting an evaluation expression is conceivable. First, in this dictionary, sentences are divided into positive ones and negative ones, and evaluation expressions are extracted from each group. In the past, there was manual collection as a seed for evaluation expression extraction, but it seems that it will be possible to automate the initial work by using this dictionary.

(6) Conclusion We constructed an adverb dictionary describing the speaker's attitudes and investigated that adverbs are useful as basic data for reputation analysis. The constructed adverb dictionary has a coverage rate of 97.77% in terms of the appearance rate of all adverbs extracted from EDR and ChaSen's ipadic in the 998 word dictionary with 105 words added to the adverb rate of all adverbs. .

  For 998 adverbs in the dictionary, the adverb used when the speaker wants or prefers an event is considered an adverb with an affirmative rating. Each adverb used when it was thought that it was not desirable or not was an adverb with a negative evaluation, and a tag was assigned to each adverb. Those that are neither positive nor negative, and those that are positive or negative depending on the context, are neutral adverbs. In order to confirm the accuracy of the dictionary, sentence classification using a dictionary was performed. The correct answer of classification was created manually and the accuracy of classification was calculated using a dictionary. As a result of experiments on 1000 sentences, the correct answer rate for classification was 86%, and the positive and negative classifications were 65% for F value. It was.

  Although the adverb dictionary has been described above, an adjective dictionary can be created in the same manner.

(6): Explanation of toolbar display The main part of the toolbar (interface part, acquisition and display of WEB information, etc.) is created in the visual c programming language, and text processing (sentence character string processing) that is the part outside the toolbar Create with Perl programming language.

  Perl is an interpreted language (it compiles and runs at run time without compiling in advance. It is slow, but maintenance and system construction is easy), and is useful for string processing including regular expressions. It is.

  In the tool bar body (interface part, WEB information acquisition display, etc.), html (hypertext markup language) is displayed. In an interpreted programming language capable of processing a character string including a regular expression, a new html is created by converting the original html. A new html is displayed on the main body of the toolbar (interface portion, WEB information acquisition display, etc.). Inside the html, a word to be highlighted is colored with an html tag.

  It is also possible to bring the dictionary file to the outside (replaceable), define the word in the file and the color of each word, and highlight the word based on it. Other than adverbs (adjectives, etc.) can be highlighted in the same way if a file is created. For example, an adverb dictionary is highlighted when an adverb dictionary is inserted, an adjective is highlighted when an adjective dictionary is inserted, and both are highlighted when both dictionaries are inserted. It is also possible to highlight a person name by putting a person name dictionary.

  Furthermore, another perl program can be brought outside (replaceable) and highlighted by that program. Since perl is an interpreted language, you can have a program as a text file outside the toolbar itself. Replacing the program allows almost any other kind of html change. This is more than just highlighting. Such changes can be made simply by modifying the outside program.

  Further, when the size of the file of the input sentence to be highlighted is larger than a predetermined value by the highlighting means 22 (this predetermined value can be set freely by the user by providing an input portion), Avoid highlighting. This prevents the html file, which is the input sentence to be highlighted, from being large and taking too much time.

  The present invention can be used by being incorporated as a toolbar of Internet Explorer.

  FIG. 5 is an explanatory diagram of the toolbar display. In FIG. 5, the display screen 30 of the output device 4 shows the title 31 of the data in the top row, the command string 32 for specifying the processing in the second row, and the viewing location in the third row. . In the fourth row, an option area is provided on the tool bar. When the option area is selected (designated), displays such as “headline”, “adverb”, and “atomic symbol” appear. It is also possible to provide a direct display of “headline”, “adverb”, “atomic symbol”, etc. instead of the option area.

  By selecting this “adverb”, the processing of FIG. 3 and FIG. 4 is performed, and the adverb (or adjective) of the sentence (text data) displayed on the lower screen 35 is highlighted differently depending on the image. Further, by selecting “adverb” again, it is possible to return to a normal display that is not highlighted.

  By selecting “headline”, it is possible to highlight an important word appearing in the title of the document (text data) displayed on the screen 35. Also, by selecting “atomic symbol”, the atomic symbol in the document (screen 35) is highlighted.

(7): Description of specific examples Examples of adjectives and adverbs will be described for each evaluation including sentence examples. The adverb includes examples of WEB text. They are written in the form of adverb (adjective) example sentences in the order of 0 (neutral), p (positive), and n (negative), respectively. In addition, the relevant adverbs (adjectives) are enclosed in [].

・ Example of adverb 0
You may also get this warning if, as in & image, the parameter following & in the CGI call is the same as the entity reference defined.
[Sincidentally] I stopped by as I passed by.

Almost no result except [98] Nabisco Cup runner-up.
Yesterday [almost] reading books at home all day.

It was just changed on the 13th, but it was changed suddenly because it was a round trip.
The section manager suddenly died and was called back from the business trip destination.

Gradually, his father was also interested in the train, and from there [gradually] began to be interested in the train.
SL was [slowly] speeding up.

In short, a CyberBook is a book that you read on your computer.
I would like to say that [in short] Japanese politics can't be as it is now.

p
Susakazu [Sukasazu], take it out and put a delicious steak on the plate.
Seibu's player runs [Sukaazu] when he goes to the base in the four balls.

Channels that are full of “helpful know-how” and “delicious information” about “food”.
I baked the cake with [bundant] butter.

Finally, the “digital camera” has finally become a real camera.
The rain that lasted for a week went up.

On the 16th, I went to see the Okinawa Times article, “I found an old grave from inside Kadena US Army along Prefectural Road No. 23 (Kokutan Road) on Chatancho Ippei”.
[Immediately] when I asked to repair my phone.

Because it is communication using the CATV line, it is [absolutely] fast!
If you keep a dog, [by far] it's an Akita dog.

n
At best, only six singers are selected in the year.
I'll be playing [at best] now.

However, faculty-led computer classes tend to simply inform you how to use the software.
This problem has a risk that [if any] could develop into a major case.

Two or three years ago, there was a story that [newly] hired for "ISO", but even if that was not the case, employees would calculate the time spent on "ISO" As you do it will be substantial.
He told me to bother me.

Mashimo (If it's a service [Mashimoshi], it's a good account, and it's usually a muzzle.) This is a habit of course, but it's the forbidden first of a tavern.
If you only have an umbrella, you have forgotten on the train to [Moshishimo].

It was quite quiet until around 22:00, and the vocals laughed as they tried to excite them by saying something different from the situation such as “Yay, everyone is riding”.
The man dragged the child he hated.

・ Example of adjectives 0
[Dazzling] The midsummer sun shines.
Casually his [quick] word hurt her badly.
Difficult to say Other people have [difficult] circumstances.
There are [significant] differences between the two.
Persistent The veteran actor is [persistent] popular.

p
Innocent New student's new uniform appearance is [Innocent].
Nostalgia When I am in the city, my hometown Noyama is [nostalgic].
The compassionate judge handed down the [compassionate] judgment.
He was leaving with a [fresh] impression.
Humble He does not break his [modest] attitude in front of anyone.

n
Unbearable His rant is an [unbearable] insult.
Suspicious Whether he was alone [suspected].
Unsightly His hairstyle is actually [unsightly].
A tenacious guy, like a snake.
It ’s not sad that I have n’t won yet.

(8): Explanation of program installation Input unit (input unit) 1, processing unit (processing unit) 2, storage unit (storage unit) 3, output unit (output unit) 4, search unit (search unit) 21, highlighting The unit (highlight display unit) 22, the contradiction detection unit (contradiction detection unit) 23, and the like can be configured by a program, and are executed by the main control unit (CPU) and stored in the main memory. This program is processed by a general computer (information processing apparatus). This computer is composed of hardware such as an input device as input means such as a main control unit, main memory, file device, display device, and keyboard. The program of the present invention is installed on this computer. In this installation, these programs are stored in a portable recording (storage) medium such as a floppy disk or a magneto-optical disk, and a drive device for accessing the recording medium provided in the computer is used. Alternatively, it is installed in a file device provided in the computer via a network such as a LAN. Then, the program steps necessary for processing are read from the file device into the main memory and executed by the main control unit.

It is explanatory drawing of the highlight display apparatus of this invention. It is explanatory drawing of the highlight display apparatus which points out the contradiction of this invention. It is an emphasis display processing flowchart of the present invention. It is a highlighting process flowchart which points out the contradiction of this invention. It is explanatory drawing of the toolbar display of this invention.

Explanation of symbols

1 Input section (input means)
2 processing unit (processing means)
3. Storage unit (storage means)
4 Output section (output means)
21 Search part (search means)
22 Highlight display section (highlight display means)

Claims (12)

Storage means for storing a dictionary of positive and negative image words;
Search means for searching for words in the dictionary from input sentences;
A highlighting device comprising highlighting means for highlighting the searched word differently depending on the image.   2. The highlighting apparatus according to claim 1, wherein the dictionary is an adverb or adjective dictionary.   The highlighting apparatus according to claim 1 or 2, wherein the highlighting means highlights the inputted sentence differently depending on the image.   The highlighting apparatus according to claim 2 or 3, wherein the adverb or adjective dictionary is created by registering words in descending order of appearance frequency.   5. The highlighting according to claim 2, wherein creation of the adverb or adjective dictionary is performed by rearranging words in order of meaning and displaying the rearranged words together with their images. apparatus.   6. The highlighting apparatus according to claim 1, further comprising a contradiction detecting unit that detects the input sentence in which a word of a positive image and a negative image appear simultaneously. An output means for displaying the input sentence on a screen;
The highlighting apparatus according to claim 1, further comprising a selection unit that selects the highlighting on a toolbar of the screen of the output unit.   8. The highlighting apparatus according to claim 7, wherein the highlighting unit does not perform the highlighting when the size of the input sentence file is larger than a predetermined value.   9. The highlighting apparatus according to claim 7, wherein the text processing of the inputted sentence is performed in an interpreted programming language capable of character string processing including a regular expression. Provide the storage means that can exchange the dictionary file that is the dictionary,
The highlighting apparatus according to claim 1, wherein how to highlight each word in the dictionary file is defined. The highlighting means is provided interchangeably,
The highlighting apparatus according to claim 1, wherein the highlighting means is created in an interpreted programming language. Storage means for storing a dictionary of positive and negative image words;
Search means for searching for words in the dictionary from input sentences;
As a highlighting means for highlighting the searched word differently depending on the image,
A program that allows a computer to function.

Images