JP6653169B2 - Keyword extraction device, content generation system, keyword extraction method, and program - Google Patents

Description

  The present invention relates to a keyword extraction device, a content generation system, a keyword extraction method, and a program.

With the rapid spread of the Internet in recent years, the number of users who use services provided on a Web (Web) site to check information or purchase products sold on the Web site has increased. I have.
In such a case, the user inputs a keyword (hereinafter, referred to as a search keyword) related to information to be searched into a search engine, and searches for a website that provides a service to be used. Then, as a result of the search, the user sequentially accesses, for example, a website displayed at a higher position among the websites displayed on the display unit, and browses the site.

  The search engine, for example, compares the search keyword with the description of the source code of the website, and selects a website displayed as a search result according to the degree of matching between the search keyword and the description of the source code of the website.

For this reason, in recent years, there has been a demand among website operators to know what description should be made on the website to be displayed at the top of the search results.
In response to such demands, a keyword extraction system for extracting keywords for constructing website content has been proposed. In the keyword extraction system, a search keyword is determined in advance for each web page constituting the website. In the keyword extraction system, a search history in a search engine is obtained for each search keyword. In the keyword extraction system, search keywords are selected in descending order of the acquired search histories. As described above, in the keyword extraction system, a keyword whose search engine has performed a large number of searches is selected as a keyword for constructing content (for example, see Patent Document 1).

JP 2006-146446 A

However, in order to be displayed at the top of the search result, it is not sufficient to simply describe the search keyword in a meta (meta) tag or in the content, and the content needs to include a description that easily matches the search keyword.
For this reason, in the technology described in Patent Literature 1, the content only includes a keyword that is frequently searched, and does not necessarily include information that the user wants to obtain. Therefore, even if a keyword with a high number of searches is included, it may not be displayed at the top of the search results.

  The present invention has been made in view of the above points, and has as its object to provide a keyword extraction device, a content generation system, a keyword extraction method, and a program capable of extracting a keyword corresponding to information desired by a user. Aim.

(1) In order to achieve the above object, a keyword extraction device according to an aspect of the present invention includes a search unit that searches a plurality of contents including a main content based on a search keyword, and a search unit that searches the plurality of contents including the main content. A first noise removing unit that removes content of a predetermined domain that does not make sense in the keyword extraction from the content, and the plurality of the plurality of domains from which the content of the predetermined domain has been removed by the first noise removing unit. One of the contents is sequentially selected from the contents, information indicating a link destination is extracted from the selected contents, and the extracted information of the link destination is compared with the information of the selected content to be similar. Main content extraction for extracting main content by removing information from the selected content information. Comprising parts and, and a keyword extraction section that extracts a plurality of keywords from the text of the extracted main content by the main content extractor.

(2) Further, the keyword extraction device according to an aspect of the present invention removes the information described by a predetermined tag from the information of the main content extracted by the main content extraction unit, thereby extracting the keyword. Further comprising a second noise removing unit that removes unnecessary description that does not make sense, wherein the keyword extracting unit removes information described by the predetermined tag by the second noise removing unit. A keyword may be extracted from the text of the main content later.

(3) In order to achieve the above object, a keyword extraction device according to an aspect of the present invention includes a search unit that searches a plurality of contents including a main content based on a search keyword, and a search unit that searches the plurality of contents including the main content. By sequentially selecting one of the contents from the contents and removing information described by a predetermined tag from the selected contents, an unnecessary description that does not make sense in the keyword extraction is removed. (2) a noise extracting unit; and a keyword extracting unit that extracts a plurality of keywords from the text of the content from which information described by the predetermined tag has been removed by the second noise removing unit.

(4) The keyword extracting device according to an aspect of the present invention may further include a first noise removing unit between the searching unit and the keyword extracting unit, wherein the first noise removing unit includes the searching unit. Out of the plurality of contents searched for, a content of a predetermined domain that does not make sense in extracting the keyword may be removed.

(5) Further, the keyword extracting device according to an aspect of the present invention may include sequentially extracting one content from the content in which information described by a predetermined tag is removed from the content by the second noise removing unit. Selecting, extracting information indicating a link destination from the selected content, comparing the extracted link destination information with the information of the selected content, and extracting similar information from the information of the selected content. A main content extracting unit that extracts the main content by removing the main content may be further provided.
(6) Further, the keyword extraction device according to one aspect of the present invention sequentially selects one of the plurality of contents from which the content of the predetermined domain has been removed by the first noise removal unit, The information indicating the link destination is extracted from the selected content, and the information of the extracted link destination is compared with the information of the selected content to remove similar information from the information of the selected content. A main content extraction unit for extracting main content may be further provided.

(7) In the keyword extraction device according to one aspect of the present invention, the search unit searches the content based on the search keyword by limiting a domain in which the content is searched, and the keyword extraction unit A plurality of keywords may be extracted from the text of the content of the limited domain, and a keyword list may be generated based on the extracted result.

(8) Further, in the keyword extraction device according to one aspect of the present invention, the search unit searches for different contents of at least two predetermined domains based on the search keyword. A plurality of keywords may be respectively extracted from the texts of the contents of the different domains, the keywords extracted from the texts of the contents of the different domains may be compared, and a keyword list may be generated based on the comparison result. .

(9) Further, in the keyword extraction device according to one aspect of the present invention, the search unit searches for a content based on the search keyword, and the keyword extraction unit searches for a plurality of keywords extracted from the text of the content. The keyword extracting unit extracts a plurality of keywords from the text of the content, and the extracted keywords are the contents of the site to be evaluated. The keyword list may be generated based on the result of determining whether the keyword is used in the search and the search order of the evaluation target site searched by the search unit.

(10) Further, the keyword extraction device according to one aspect of the present invention includes a text extraction unit configured to extract at least one text from the main content extracted by the main content extraction unit, and a search performed by the search unit based on the text. A search order obtaining unit that obtains the obtained order, and an evaluation result generating unit that performs an evaluation on a target web page on which the sentence is extracted based on the order obtained by the search order obtaining unit. You may make it provide further.

(11) In addition, the keyword extraction device according to one aspect of the present invention includes a suggestion acquisition unit that acquires a predicted word based on a search keyword from a result searched by the search unit, and a plurality of suggestion acquisition units that are acquired by the suggestion acquisition unit. One of the predicted words is selected, and a search order of the selected predicted words is obtained from a result of searching the selected predicted words by the search unit, using the main content extracted by the main content extraction unit. The information processing apparatus may further include a search order obtaining unit, and an evaluation result generating unit that evaluates a web page to be evaluated based on the order obtained by the search order obtaining unit.

(12) In order to achieve the above object, a content generation system according to one aspect of the present invention includes a keyword extraction device according to any one of (1) to (11), and the keyword extraction device extracted by the keyword extraction device. A content generation device that generates predetermined content using a plurality of keywords.

(13) In order to achieve the above object, a keyword extraction method according to an aspect of the present invention provides a keyword extracting method, wherein: a search unit searches for a plurality of contents including a main content based on a search keyword; A first noise removing step of removing, from the plurality of contents searched by the search step, contents of a predetermined domain that does not make sense in extracting a keyword, the main content extracting unit includes: One content is sequentially selected from the plurality of contents from which the content of the predetermined domain has been removed by the removal procedure, information indicating a link destination is extracted from the selected content, and information of the extracted link destination and Comparing the information of the selected content with similar information, Including a main content extraction procedure for extracting the main content is removed from the tree information, the keyword extraction section, and a keyword extraction procedure for extracting a plurality of keywords from the text of the main content extracted by the main content extraction procedure.

(14) In order to achieve the above object, in the keyword extraction method according to one aspect of the present invention, the search unit may search for a plurality of contents including the main content based on the search keyword; By sequentially selecting one of the contents from the plurality of contents searched by the search procedure, and removing information described by a predetermined tag from the selected contents, so that the meaning is extracted in the keyword extraction. A second noise removal procedure for removing unnecessary descriptions that have not been formed, and a keyword extracting unit configured to perform a plurality of processes from the text of the content from which information described by the predetermined tag has been removed by the second noise removal procedure. A keyword extraction procedure for extracting a keyword.

(15) In order to achieve the above object, a program according to one embodiment of the present invention provides a computer with a search procedure for searching for a plurality of contents including a main content based on a search keyword, and the plurality of contents searched by the search procedure. A first noise removing step of removing content of a predetermined domain that does not make sense in the keyword extraction from the contents of the plurality of contents; and a plurality of the plurality of the contents of which the content of the predetermined domain has been removed by the first noise removing procedure. One of the contents is sequentially selected from among the contents, information indicating a link destination is extracted from the selected contents, and the extracted information of the link destination is compared with the information of the selected content to determine similarity. The main content by removing the existing information from the selected content information And in the content extraction procedure, the keyword extraction procedure for extracting a plurality of keywords from the text of the main content extracted by the main content extraction procedure is run.

(16) In order to achieve the above object, a program according to one embodiment of the present invention comprises: a computer having a search procedure for searching a plurality of contents including a main content based on a search keyword; One of the contents is sequentially selected from among the contents, and information described by a predetermined tag is removed from the selected contents, thereby removing unnecessary description that does not make sense in keyword extraction. A second noise removal procedure and a keyword extraction procedure of extracting a plurality of keywords from the text of the content from which the information described by the predetermined tag has been removed by the second noise removal procedure are executed.

  ADVANTAGE OF THE INVENTION According to this invention, the keyword according to the information which a user wants to know can be extracted.

It is a figure showing the operation screen of the keyword extraction device concerning a 1st embodiment. It is a schematic structure figure of the keyword extraction device concerning a 1st embodiment. FIG. 4 is a diagram illustrating an example of information stored in a domain DB according to the first embodiment. It is a figure showing an example of composition of a web page. 5 is a flowchart of a process of the keyword extracting device according to the first embodiment. FIG. 3 is a diagram illustrating an example of a source code of a web page. It is a figure showing the example of composition of the self-webpage and a link destination webpage concerning a 1st embodiment. It is a flowchart of the procedure of the extraction processing of the main content according to the first embodiment. FIG. 3 is a block diagram illustrating a configuration of a keyword extraction unit according to the first embodiment. FIG. 5 is a diagram illustrating an example of a keyword list output by a keyword list output unit according to the first embodiment. 5 is a flowchart of a keyword extraction process according to the first embodiment. It is a schematic structure figure of the keyword extraction device concerning the modification of a 1st embodiment. It is a figure showing an example of information stored in tag DB concerning a modification of a 1st embodiment. It is a flow chart of processing of a keyword extraction device concerning a modification of a 1st embodiment. It is a flowchart of the removal process of the meaningless word which concerns on the modification of 1st Embodiment. It is a schematic structure figure of the keyword extraction device concerning this embodiment concerning a 2nd embodiment. It is a figure showing an example of the information stored in tag DB concerning a 2nd embodiment. It is a flow chart of processing of a keyword extraction device concerning a 2nd embodiment. It is a schematic structure figure of the keyword extraction device concerning the 1st modification of a 2nd embodiment. It is a flow chart of processing of a keyword extraction device concerning a 1st modification of a 2nd embodiment. It is a schematic structure figure of the keyword extraction device concerning the 2nd modification of a 2nd embodiment. FIG. 14 is a diagram illustrating an example of a domain stored in a domain DB according to a second modification of the second embodiment. It is a figure showing the example of the operation screen by the keyword extraction device concerning the 2nd modification of a 2nd embodiment. It is a figure showing the example of comparison of the search result of the keyword by the important keyword extraction device by the 7th extraction method concerning the 2nd modification of a 2nd embodiment. It is a figure showing the example of the operation screen by the keyword extraction device when the 8th extraction method concerning the 2nd modification of a 2nd embodiment is selected. It is a flow chart of processing in the 8th extraction method concerning a 2nd modification of a 2nd embodiment. It is a figure showing the example of the evaluation result by the keyword extraction device when the ninth extraction method concerning the 2nd modification of a 2nd embodiment is selected. It is a schematic structure figure of the keyword extraction device concerning a 3rd present embodiment. 13 is a flowchart of an evaluation process performed by the keyword extraction device according to the third embodiment. It is a figure showing the example of the evaluation result concerning a 3rd embodiment. It is a schematic structure figure of the keyword extraction device concerning a 4th embodiment. It is a flow chart of the evaluation processing which the keyword extraction device concerning a 4th embodiment performs. It is a figure showing the example of the evaluation result concerning a 4th embodiment. It is a lineblock diagram showing the contents generation system concerning a 5th embodiment.

[Summary of the present invention]
First, an outline of the present invention will be described.
In the present invention, a search keyword for a website is searched by a search engine. Note that the search keyword is a keyword assumed to be input to a search engine by a web page viewer. Then, in the present invention, a predetermined number of web pages are selected from the top among the searched web pages. Then, in the present invention, noise is removed from each of the selected predetermined number of web pages (also referred to as contents). Then, in the present invention, a keyword is extracted by analyzing text included in the content from which noise has been removed. Here, the keyword is a keyword included in a web page displayed at a higher rank of a search result as a result of a search performed by a search engine using a search keyword. Each processing will be described later.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Note that the present invention is not limited to the embodiment, and various changes can be made within the scope of the technical idea.

[First Embodiment]
FIG. 1 is a diagram showing an operation screen g101 of the keyword extraction device 1 according to the present embodiment.
In FIG. 1, a square g111 is an input field for a search keyword, a square g112 is an image of a button for starting extraction of a keyword based on the search keyword, and a square g113 is a keyword extraction device based on the search keyword. 1 is a column in which a list of keywords extracted by No. 1 is displayed.

<Configuration of Keyword Extraction Device 1>
FIG. 2 is a schematic configuration diagram of the keyword extracting device 1 according to the present embodiment.
As shown in FIG. 2, the keyword extraction device 1 includes a keyword input unit 11, a search unit 12, a domain DB 13, a first noise removal unit 14, a main content extraction unit 15, a keyword extraction unit 18, and a keyword list output unit 19. . The keyword extracting device 1 is connected to a network 2. The network 2 is, for example, the Internet.

  The keyword input unit 11 is, for example, a keyboard, a mouse, a tablet, or the like. The keyword input unit 11 outputs the search keyword input by the user to the search unit 12.

  The search unit 12 acquires the search keyword output by the keyword input unit 11, and searches for a web page suitable for the acquired search keyword using a search engine. The search unit 12 selects a predetermined number of web pages from the top among the web pages obtained by the search. The predetermined number is, for example, 20 pieces. The search unit 12 outputs information indicating the selected predetermined number of web pages to the first noise removal unit 14. Note that the search result includes the URL (Uniform Resource Locator) address of each web page.

  The domain DB 13 stores domains of websites unnecessary for extracting keywords. Here, the unnecessary domain is a domain of a web site that does not make sense as content such as a site in which a part of a web page is put together.

  The first noise removing unit 14 removes the web pages of the domain stored in the domain DB 13 from the predetermined number of web pages using the information indicating the predetermined number of web pages output by the search unit 12, The information indicating the removed web page is output to the main content extracting unit 15. The information indicating the web page includes the source code of the web page. The information of the web page includes a header, a sidebar, a main content, a footer, and the like. When the predetermined number of web pages do not include the web pages of the domain stored in the domain DB 13, the first noise removing unit 14 outputs information indicating the predetermined number of web pages to the main content extraction unit 15. .

  The main content extraction unit 15 sequentially selects information of one web page from the predetermined number of web pages using the information indicating the predetermined number of web pages output by the first noise removal unit 14 and selects the selected web page. Extract main content from web page information. The method of extracting the main content will be described later. The main content extraction unit 15 outputs the extracted main content to the keyword extraction unit 18 for each web page.

  The keyword extracting unit 18 extracts a plurality of keywords from the main content output by the main content extracting unit 15. The keyword extracting unit 18 performs a sort process on the extracted plurality of keywords as described later, and outputs the sorted keyword list to the keyword list output unit 19. The keyword extraction method and the sorting process will be described later.

  The keyword list output unit 19 is, for example, at least one of an information providing unit on the Web, a display device, a printer device, and a communication device. The keyword list output unit 19 provides the keyword list output by the keyword extraction unit 18 on, for example, the Web.

Next, an example of information stored in the domain DB 13 will be described.
FIG. 3 is a diagram illustrating an example of information stored in the domain DB 13 according to the present embodiment. As shown in FIG. 3, the domain DB 13 stores at least one domain. The domain stored in the domain DB 13 may be updated via the network 2. When a user inputs a keyword, a domain of a website that is unnecessary for extracting the keyword may be input. Then, the keyword extracting device 1 may temporarily store the input domain in the domain DB and extract the keyword.

Next, an example of the configuration of the web page g201 will be described.
FIG. 4 is a diagram illustrating an example of the configuration of the web page g201. The example shown in FIG. 4 is an example in which there are two columns and a menu is on the right. The web page g201 in the example shown in FIG. 4 includes a header g221, a sidebar g222, and a main content g223.

<Processing procedure of keyword extraction device 1>
Next, a processing procedure of the keyword extracting device 1 will be described. FIG. 5 is a flowchart of the process of the keyword extracting device 1 according to the present embodiment.
(Step S1) The keyword input unit 11 acquires a search keyword input by a user.
(Step S2) The search unit 12 searches for a web page suitable for the search keyword output by the keyword input unit 11 using a search engine, and a predetermined number of web pages among the web pages obtained by the search are searched. Select a web page.
(Step S3) The search unit 12 acquires the source code of each of the selected predetermined number of web pages.

(Step S4) The first noise removing unit 14 uses the information on the predetermined number of web pages output by the search unit 12 to remove the web pages of the domain stored in the domain DB 13 from the predetermined number of web pages. I do.
(Step S5) The main content extraction unit 15 sequentially selects one web page information from the web page information output by the first noise removal unit 14, and extracts the main content from the selected web page information.

(Step S7) The keyword extracting unit 18 extracts a plurality of keywords from the main content output by the main content extracting unit 15.
Thus, the processing of the keyword extracting device 1 ends.

<Method of extracting main content>
Next, a method of extracting the main content will be described.
FIG. 6 is a diagram illustrating an example of a source code of a web page. The source code shown in FIG. 6 is a part of the source code of the web page. The source code shown in FIG. 6 is an example for explaining the relationship between the configuration of the web page and the source code, and may not match the actual source code of the web page.
Note that the main content of the web page in the present embodiment is a part necessary for extracting a keyword, for example, a title, an article, a question content, a description of a diagram or a photo, a response to a question, and the like. The unnecessary part in the embodiment is, for example, an advertisement, a menu, or the like.

  As shown by reference numeral g251 in FIG. 6, the source code is described using a plurality of tags. Then, the source code includes header information g261 in which the title of the website is described, header g262 in which the title and description of the website and the web page are described, main content g263, a link destination in the website and other web pages. A menu g264 in which a link to a site and the like are described is included.

FIG. 7 is a diagram illustrating a configuration example of the own web page and the linked web page according to the present embodiment. The own web page is one of the search results in step S2 in FIG.
The configuration of the web page indicated by reference numeral g301 is an example of the configuration of the own web page, which is a two-column configuration, in which a header g311 is arranged on the web page, a main content g313 is arranged on the left, and a menu g312 is on the right. Is arranged.

The configuration of the web page indicated by the reference numeral g321 is a configuration example of the web page of the first link destination described in the own web page, has a two-column configuration, and a header g331 is arranged on the web page. , A main content g333 is arranged on the left, and a menu g332 is arranged on the right.
The configuration of the web page indicated by the reference sign g341 is an example of the configuration of the web page of the second link destination described in the own web page, has a three-column configuration, and a header g351 is arranged on the web page. , A first menu g352 is arranged on the left, a main content g353 is arranged in the middle, and a second menu g354 is arranged on the right.

In FIG. 7, the web page indicated by reference numeral g321 is one of web pages in the same website as the own web page. The web page indicated by reference numeral g321 is one of web pages in a website different from the own web page.
In many cases, the URL address of a web page in the same website as the own web page has the same domain, the address assigned to the home page, and the like. On the other hand, the URL address of a web page in a website different from the own web page often has a different domain, an address assigned to a home page, or the like.

  Here, the distance between the URL address of the own web page and the URL address of a web page in the same website as the own web page is defined as a first Levenshtein distance. The distance between the URL address of the own web page and the URL address of a web page in a website different from the own web page is defined as a second Levenshtein distance. In this case, the first Levenshtein distance is smaller than the second Levenshtein distance, that is, the Levenshtein distance is short. On the other hand, the second Levenshtein distance is larger than the first Levenshtein distance, that is, the Levenshtein distance is far.

  When comparing the source codes of two websites having the same Levenshtein distance, the descriptions of the header g311 and the header g331 are often the same or similar, and the descriptions of the menu g312 and the menu g332 are often the same or similar. . That is, areas where the source codes match or are similar can be considered to be headers and menus (sidebars). The description of the main content g313 is obtained by removing the descriptions of the header g311 and the menu g312 from the source code of the own web page. As described above, the main content extracting unit 15 extracts the description of the main content g313 by removing the descriptions of the header g311 and the menu g312 from the source code of the own web page. Note that the main content extraction unit 15 determines whether or not the source codes match, or whether or not the source codes are similar, using a known similarity estimation method for estimating the similarity between documents.

  Further, when comparing the source codes of two websites having a long Levenshtein distance, the description of the header g311 and the header g351 is often less similar to the description of the header g311 and the header g351. Further, the description of the menu g312 and the first menu g352 is not more similar to the description of the menu g312 and the menu g332, and the description of the menu g312 and the second menu g354 is the description of the menu g312 and the menu g332. Often less similar. As a result, since there is no area where the source code is similar, even if the source codes of the two websites having a long Levenshtein distance are compared, the description of the header g311 and the menu g312 (sidebar) of the own web page can be obtained. It can not be identified. As described above, the description of the main content g313 cannot be extracted by comparing the source code of each of the two websites with a long Levenshtein distance.

  Also, a portion that appears only on a specific web page is likely to be main content. On the other hand, unnecessary parts tend to appear over a plurality of web pages. For example, in the case of an article of a news site, a link destination of another news may be described on the web page, and the configuration of the web page of the other news (see FIG. 3) is similar to that of the web page of the other news. There are many cases. In the other news web page and the web page, for example, information on a link destination, an advertisement, and the like are described in the side bar g222 in FIG. As described above, in the present embodiment, a plurality of web pages are compared, and as a result of the comparison, a common portion is regarded as an unnecessary portion, and a portion which does not appear on another web page is regarded as main content. In this embodiment, one of the searched web pages is selected, and the link destination described in the selected web page is set as the web page used for comparison. In the present embodiment, for example, the Levenshtein distance is used for comparison between web pages.

  For this reason, in the present embodiment, the main content extracting unit 15 extracts at least one URL address of a link destination web page described in the own web page whose Levenshtein distance is close to the own web page. Then, the main content extracting unit 15 obtains the source code of the own web page and the source code of the linked web page having a short Levenshtein distance, and describes a description of an unnecessary area based on the similarity of the obtained source code. The main content is extracted by removing it.

Next, an example of the procedure of the main content extraction processing performed by the main content extraction unit 15 in step S5 of FIG. 5 will be described.
FIG. 8 is a flowchart of a procedure of main content extraction processing according to the present embodiment.

(Step S101) The main content extraction unit 15 sequentially selects information of one unprocessed web page from information of a predetermined number of web pages output by the first noise removal unit 14, and performs steps S102 to S106. Is performed.
(Step S102) The main content extracting unit 15 acquires the source code of the selected web page. Subsequently, the main content extracting unit 15 extracts information indicating a link included in the selected web page. Note that the information indicating a link includes <a href=“... ”>, <base href = ”...”, <Link rel = ”...” Href = ”. link href = “...>” or the like. In the present embodiment, an example in which information indicating a link is described by a tag has been described. However, the description is not limited to this, and any description may be used as long as the information indicates a link.

(Step S103) The main content extraction unit 15 sequentially selects one from the plurality of link destination URL addresses extracted in step S102. The main content extraction unit 15 sequentially calculates the Levenshtein distance between the URL address of the web page selected in step S101 and the URL address described in the tag indicating the link.
(Step S104) As a result of the calculation, the main content extraction unit 15 acquires the source code of at least one linked website whose Levenshtein distance is short. Note that the main content extraction unit 15 may select a plurality of link destinations in ascending order of Levenshtein distance.

(Step S105) The main content extraction unit 15 compares the source code of the web page selected in step S101 with the source code of the link destination web page acquired in step S104.
(Step S106) As a result of the comparison in step S105, the main content extraction unit 15 extracts the main content by removing the description whose source code is close (for example, see Reference Document 1).

(Step S107) When the processing of steps S102 to S106 is completed for all the web pages with respect to the information of the web pages output by the first noise removing unit 14, the main content extraction unit 15 writes the description of the extracted main content into the web page. Each time it is output to the keyword extraction unit 18.
This is the end of the main content extraction processing.

References 1; Mitsuo Yoshida, Mikio Yamamoto, Automatic content extraction from news pages that do not require teacher information, Transactions of the Database Society of Japan 8 (1) 29-34 2009.

<Keyword extraction>
Next, keyword extraction will be described.
FIG. 9 is a block diagram illustrating a configuration of the keyword extracting unit 18 according to the present embodiment. As shown in FIG. 9, the keyword extraction unit 18 includes a morphological analysis unit 181, a term extraction unit 182, and a keyword list generation unit 183.

  The morphological analysis unit 181 acquires text information in the main content output by the main content extraction unit 15 for each web page. The morphological analysis unit 181 performs a morphological analysis on the text information using a known method. When the text is in Japanese, the morphological analysis unit 181 performs morphological analysis using software such as “ChaSen (Chasen)”, “Chamame”, and “MeCab (Wafu)”. The analysis result includes the character string, the type of part of speech of the character string, the type of part of speech, the original form of the character string, the reading, and the like. The morphological analyzer 181 outputs the analyzed result to the term extractor 182.

The term extraction unit 182 uses the analysis result output from the morphological analysis unit 181 to assemble a compound word based on the word arrangement and part-of-speech information. For example, when nouns appear continuously, the term extraction unit 182 integrates the continuous nouns into a compound word. The term extraction unit 182 extracts a noun or a compound word including a plurality of nouns. The term extraction unit 182 determines the minimum unit noun (hereinafter, also referred to as a single noun) or a noun constituting a compound word based on the number of times that each of the nouns appears in a predetermined number of web pages selected by the search unit 12 in a transverse manner. Then, the importance idf is calculated using, for example, an IDF (Inverse Document Frequency) method.
In addition, the term extraction unit 182 calculates the number of applications of nouns or the number of appearances of compound words in the text information of all the predetermined number of web pages selected by the search unit 12.
The term extraction unit 182 outputs the extracted noun or compound word to the keyword list generation unit 183 in association with the calculated importance and the number of appearances.

The keyword list generation unit 183 acquires the noun or compound word output by the term extraction unit 182 and associated with the importance and the number of appearances. The keyword list generation unit 183 generates a keyword list by multiplying the importance and the number of appearances for each noun or compound word, and arranging the nouns or compound words in descending order of the multiplied value. The keyword list generation unit 183 outputs the generated keyword list to the keyword list output unit 19.
The term extraction unit 182 and the keyword list generation unit 183 may be configured to include a module of the programming language Perl, for example, “TermExtract”.

  As described above, in the present embodiment, one search web page is selected one by one from a predetermined number of web pages searched from the top using a search engine. In the present embodiment, the source code of the link destination described in the source code of the selected own web page is compared with the source code of the own web page, and an unnecessary area is determined based on the comparison result. The main content is extracted by removing the description. As described above, in the present embodiment, a keyword can be accurately extracted by extracting, for example, a noun and a compound word using the text of the main content from which the noise component has been removed. Note that the words to be extracted are not limited to nouns and compound words, and may include, for example, adjectives and adverbs.

<Example of keyword list>
Here, an example of the keyword list output by the keyword list output unit 19 will be described.
FIG. 10 is a diagram illustrating an example of a keyword list output by the keyword list output unit 19 according to the present embodiment. As shown in FIG. 10, the keyword list includes the input keyword (the area indicated by reference sign g361), the number of appearances (the area indicated by reference sign g362), the value obtained by multiplying the number of appearances by the importance (the area indicated by reference sign g363), A keyword (a region indicated by reference sign g364) is associated with the keyword.
For example, as shown in FIG. 10, the keyword list output unit 19 multiplies the number of appearances output by the keyword extraction unit 18 by importance for each keyword, and sorts the keywords in descending order of the multiplied value.
As a result, as shown in FIG. 10, the display order of the keywords is the order in which the value obtained by multiplying the number of appearances in the searched web page by the importance is large. For this reason, a keyword extracted when "foot numbness is cold" is input is displayed in the third position in the list because its importance is high even if the number of appearances is, for example, 15th.

The keyword list output by the keyword list output unit 19 only needs to include at least a keyword, and does not need to include the number of appearances and importance.
Further, the display order is not limited to the example shown in FIG. 10, and may be an order in which the number of applications is large, an order in which importance values are large, an order based on another statistical method, or the like.

Next, the keyword extraction process performed by the keyword extraction unit 18 in step S7 of FIG. 5 will be described.
FIG. 11 is a flowchart of keyword extraction processing according to the present embodiment.
(Step S201) The morphological analysis unit 181 performs a morphological analysis on the text information output by the main content extraction unit 15 using a known method.
(Step S202) The term extraction unit 182 assembles a compound word based on the word arrangement and part-of-speech information using the analysis result output from the morphological analysis unit 181.

(Step S203) The term extraction unit 182 extracts a noun and a compound word including a plurality of nouns.
(Step S204) The term extraction unit 182 calculates the importance of each of the noun and the compound word using, for example, the IDF method.

(Step S205) The term extraction unit 182 calculates the number of appearances of nouns and compound words in the text information of each of the input web pages. Subsequently, the term extraction unit 182 associates each of the nouns or compound words extracted in step S203 with the calculated number of appearances and the importance calculated in step S204, and outputs them to the keyword list generation unit 183. .

(Step S206) The keyword list generation unit 183 multiplies the noun or compound word output by the term extraction unit 182 by the importance and the number of appearances, and sorts the noun or compound word in descending order of the multiplied value. , Generate a keyword list. The keyword list generation unit 183 outputs the generated keyword list to the keyword list output unit 19.
Thus, the keyword extraction processing ends.

<Example of using keyword list>
A usage example of the keyword list thus extracted will be described.
For example, when the company A sets up a web page of the product B, the description of the product B is usually described on the web page. However, such description does not always cover information that a user who is actually interested in the product B wants to know. When such a web page, that is, content is created, the content described on the web page is evaluated by a search engine, and as a result, the content is not often displayed at the top of the search result.
For this reason, the web page creator extracts, using, for example, a mind map or the like, a search keyword that is assumed to be used by the user regarding the product B for the search. Then, the extracted search keywords are input to the keyword extracting device 1 to obtain a keyword list.
The web page creator creates a web page of the product B using the keywords listed in the keyword list. As a result, the web page creator can create a web page containing much information that the user wants to know about the product B. A web page that contains a lot of information that the user wants to know is displayed at the top of the search results as a result of the evaluation of the content of the web page by the search engine, and the user wants to know a lot of information. Since it is included, the staying time of the user is prolonged, and an effect of purchasing a product is obtained.

  In the example described above, the web page regarding the product is described, but the present invention is not limited to this. By producing pamphlets, catalogs, instruction manuals, and the like using the keyword list, it is possible to include a lot of information that the user wants to know.

Next, a case will be described as an example where a trend researcher wants to know the trend of sweets sold at a convenience store.
When a search is performed by inputting a search keyword of “convenience store” or “sweets” into a search engine, as of May 28, 2015, about 1.8 million search results are obtained. It is difficult for researchers to read all of these, and it is difficult to spot trends by simply reading the entire web page of some searched results.
On the other hand, by inputting the search keywords “convenience store” and “sweets” into the keyword extraction device 1, the keyword extraction device 1 generates a keyword list relating to sweets in a convenience store that has been talked about by users of the Internet. Can be. Thereby, in the present embodiment, the trend can be known by the trend investigator viewing the generated keyword list. In addition, by generating a keyword list regularly (for example, once a month) by the keyword extracting device 1, a trend researcher can also know a change in a keyword, that is, a change in a trend.

  In the present embodiment, an example has been described in which the search unit 12 selects a predetermined number of web pages from the search result from the top, but the present invention is not limited to this. For example, when a predetermined number of web pages are selected from the search result and the number of web pages after removing unnecessary domains by the first noise removal unit 14 is less than a desired number (for example, 10 or more), The search unit 12 may select a number in which the number of web pages after being removed by the first noise removal unit 14 is equal to or greater than a desired number.

  Further, in the present embodiment, an example has been described in which the main content extraction unit 15 calculates the Levenshtein distance based on the closeness of the distance between the own web page and the link destination. However, the present invention is not limited to this. The main content extraction unit 15 may calculate the closeness of the distance between the web page and the link destination using, for example, the 3-gram (n-gram) method.

As described above, the keyword extraction device 1 according to the present embodiment includes the search unit 12 that searches for a plurality of contents (for example, 20) including the main content based on the search keyword, and the search unit 12 among the plurality of contents searched by the search unit. And a first noise removing unit 14 for removing content of a predetermined domain that does not make sense in the keyword extraction, and a plurality of contents from which the content of the predetermined domain has been removed by the first noise removing unit. One piece of content is sequentially selected, information indicating a link destination is extracted from the selected content, and information of the extracted link destination is compared with information of the selected content, and similar information is determined. A main content extraction unit 15 for extracting main content by removing from the information; Comprising a keyword extraction unit 18 for extracting a plurality of keywords from the text of the extracted main content by part, a.
Note that the predetermined domain is a domain that does not make sense as content in keyword extraction.

  With this configuration, in the present embodiment, first, by removing a web page of a predetermined domain, an unnecessary web page having no meaning as content in extracting a keyword is removed from a search result. In the present embodiment, after removing unnecessary web pages, an unnecessary area is determined from each of a plurality of web pages searched using the search keyword based on the similarity with the source code of the link destination in the own web page. (Eg, headers, footers, menus, etc.) are removed. As a result, in the present embodiment, the main content can be accurately extracted only from the web pages from which unnecessary web pages have been deleted. As a result, according to the present embodiment, it is possible to extract a search keyword, that is, a keyword corresponding to information that the user wants to know.

[Modification of First Embodiment]
Next, a modified example of the first embodiment will be described.
An example in which the keyword extracting device 1 further includes a tag DB 16 and a second noise removing unit 17 will be described.

<Configuration of Keyword Extraction Device 1A>
FIG. 12 is a schematic configuration diagram of a keyword extraction device 1A according to a modification of the present embodiment. Note that functional units having the same functions as those of the keyword extracting device 1 are denoted by the same reference numerals, and description thereof is omitted.
As shown in FIG. 12, the keyword extracting device 1A includes a keyword input unit 11, a search unit 12, a domain DB 13, a first noise removing unit 14, a main content extracting unit 15, a tag DB 16, a second noise removing unit 17, a keyword extracting unit. 18 and a keyword list output unit 19.

  The main content extraction unit 15 sequentially selects information of one web page from the web page information output by the first noise removal unit 14, and extracts main content from the selected web page information. The main content extraction unit 15 outputs the description of the extracted main content to the second noise removal unit 17 for each web page.

  The tag DB 16 stores tags used to delete items including unnecessary terms (hereinafter, meaningless words) from the main content of the web page. The tags stored in the tag DB 16 may be updated via the network 2.

  The second noise removing unit 17 refers to the tag DB 16 and removes meaningless words from the main content output by the main content extracting unit 15. Thereby, the second noise removing unit 17 removes noise of meaningless words from the main content of the web page for each web page. The second noise removing unit 17 outputs the main content from which the meaningless words have been removed to the keyword extracting unit 18. A method of removing meaningless words will be described later. In the present embodiment, an example in which meaningless words and the like are removed using a tag will be described. However, the present invention is not limited to this, and other methods may be used to remove meaningless words and the like. In this case, the tag DB 16 may store information used for removing meaningless words and the like.

  The keyword extracting unit 18 extracts a plurality of keywords from the main content output by the second noise removing unit 17. The keyword extracting unit 18 performs a sort process on the extracted plurality of keywords as described later, and outputs the sorted keyword list to the keyword list output unit 19. The keyword extraction method and the sorting process will be described later.

Next, an example of information stored in the tag DB 16 will be described.
FIG. 13 is a diagram illustrating an example of information stored in the tag DB 16 according to a modification of the present embodiment. As shown in FIG. 13, the tag DB 16 stores at least one tag. For example, “<h1> to </ h1>” used in <class> is a tag indicating a heading. “<Div class =“ usrInfo ”> to </ div>” is a tag indicating identifier information for identifying a user on a web page such as a bulletin board. Tags indicating such unnecessary items are set in advance by the designer of the keyword extracting device 1A.

<Processing Procedure of Keyword Extraction Apparatus 1A>
Next, a processing procedure of the keyword extracting device 1A will be described. FIG. 14 is a flowchart of the process of the keyword extracting device 1A according to the modification of the present embodiment. Note that the same processes as those of the keyword extraction device 1 are denoted by the same reference numerals, and description thereof will be omitted.

(Steps S1 to S5) The keyword extraction device 1A performs the processing of steps S1 to S5, and proceeds to the processing of step S6.
(Step S6) The second noise removing unit 17 refers to the tag DB 16 and removes meaningless words from the main content output by the main content extracting unit 15.
(Step S7) The keyword extracting unit 18 extracts a plurality of keywords from the main content output by the second noise removing unit 17.

<Speechless processing>
Next, the process of removing meaningless words performed by the second noise removing unit 17 in step S6 of FIG. 14 will be described.
FIG. 15 is a flowchart of a meaningless word removal process according to a modification of the present embodiment.

(Step S301) The second noise removal unit 17 sequentially selects the main content of one unprocessed web page from the main content of each web page output by the main content extraction unit 15, and performs the processes of steps S302 to S303. Do.
(Step S302) The second noise removing unit 17 removes the description of the tag corresponding to the meaningless word from the source code of the main content of the web page selected in step S301 by referring to the tag DB16. Remove semantic words. Here, the description of the tag is a description between the start tag and the end tag.

(Step S303) The second noise removing unit 17 removes the remaining tag information from the description of the main content removed in Step S302. The remaining tag information is, for example, a tag indicating designation of a character size, designation of a character color, line feed, and the like.

(Step S304) For the main content for each web page output by the main content extraction unit 15, the second noise removing unit 17 completes the processing of steps S302 to S303 for all the web pages, The text information is output to the keyword extracting unit 18 for each web page.
This is the end of the main content extraction processing.

<Examples of nonsense word items to be removed>
Here, examples of items that are removed as meaningless words will be described.
First, a question and answer web page where you can write answers to questions will be described as an example. In addition to questions and answers, such question and answer web pages include, for example, the number of views, the number of answers, thanks for the answers, the identity of the respondent, the date and time of the answer, the information indicating the person who gave the best answer, etc. include.
Effective contents for extracting a keyword are, for example, a text of a question and a text of an answer. Therefore, in the web page of the question, the number of views, the number of answers, thanks for the answer, the identification information of the answerer, the date and time of the answer, the information indicating the person who gave the best answer, and the like are nonsense words. These items are described by a tag having a predetermined class name, an item tag (for example, <h2> to </ h2>) in the tag of the predetermined class.

Further, for example, a dictionary page on the web includes an outline, a text, a table of contents, a description of the type of term, a description indicating that the content is insufficient, a reference, a reference list-related item, and the like.
Effective contents for extracting a keyword are, for example, outline text and body text. For this reason, in the dictionary page, the table of contents, the description of the types of terms, the description indicating that the content is insufficient, the references, the reference list related items, and the like are nonsense words.

  For example, the manufacturer of the keyword extracting device 1A analyzes and extracts the source code in advance and describes the tags in which the items corresponding to these meaningless words are described, and stores the extracted tags in the tag DB16.

As described above, in the keyword extraction device 1A of the present embodiment, by removing the information described by the predetermined tag from the information of the main content extracted by the main content extraction unit 15, meaning is obtained in keyword extraction. And a second noise removing unit 17 for removing unnecessary descriptions that have not been performed. The keyword extracting unit 18 includes a main content text after the information described by a predetermined tag is removed by the second noise removing unit. Extract keywords from.
The information described by the predetermined tag is an unnecessary term that does not make sense in extracting the keyword.

With this configuration, in the present embodiment, meaningless words can be removed by removing information described by a predetermined tag from main content. As a result, in the present embodiment, keywords can be accurately extracted from the main content from which the meaningless words have been removed.
In addition, for example, when words that are assumed to be unnecessary are registered in the database and words that are considered to be unnecessary are removed from the main content, valid words may be removed from the content in some cases. On the other hand, by removing information described by tags as in the present embodiment, meaningless words can be accurately removed.

[Second embodiment]
In the first embodiment, the example in which the keyword extracting device 1 (or 1A) includes the first noise removing unit 14 and the main content extracting unit 15 has been described, but in the second embodiment, the first noise removing unit 14 and the main content An example in which the extraction unit 15 is not provided and the second noise removal unit is provided will be described. In the present embodiment, the noise removed by the second noise removing unit is a meaningless word and an unnecessary description based on the tag.

<Configuration of Keyword Extraction Device 1B>
FIG. 16 is a schematic configuration diagram of a keyword extraction device 1B according to the present embodiment according to the present embodiment.
As shown in FIG. 16, the keyword extraction device 1B includes a keyword input unit 11, a search unit 12B, a tag DB 16B, a second noise removal unit 17B, a keyword extraction unit 18, and a keyword list output unit 19. The keyword extracting device 1B is connected to the network 2. Note that functional units having the same functions as those of the keyword extracting device 1 or 1A are denoted by the same reference numerals and description thereof is omitted.

  The keyword input unit 11 is, for example, a keyboard, a mouse, a tablet, or the like. The keyword input unit 11 outputs the search keyword input by the user to the search unit 12B.

  The search unit 12B searches for a web page suitable for the search keyword output by the keyword input unit 11 using a search engine, and among the web pages obtained by the search, for example, a predetermined number of web pages from the top. select. The search unit 12B outputs information indicating the selected predetermined number of web pages to the second noise removal unit 17B.

  The tag DB 16B stores a tag used to delete an item of an unnecessary term (nonsense word) and a tag used to delete an unnecessary description from the main content of the web page. I have. FIG. 17 is a diagram illustrating an example of information stored in the tag DB 16B according to the present embodiment. As shown in FIG. 17, at least one tag is stored in the tag DB 16B. For example, “<h1> to </ h1>” used in <class> is a tag indicating a heading. “<Div class =“ usrInfo ”> to </ div>” is a tag indicating identifier information for identifying a user on a web page such as a bulletin board.

  The “<meta>” tag is a tag indicating meta information regarding a document (web page), and the “<div id =“ header ”> to </ div> tags are tags indicating header information. The “<script type> to </ script>” tag is a tag for describing the script. The “<link href =“ ”>” tag and the “<a href=“”>” tag indicate the link destination. Tag. These are unnecessary descriptions that do not make sense as contents in the web page. Other tags used to delete unnecessary descriptions include a tag indicating an advertisement, a tag indicating a button, a tag indicating a footer, a tag indicating a notice, and the like. Tags indicating such unnecessary items and descriptions are set in advance by the designer of the keyword extracting device 1B. The tags stored in the tag DB 16B may be updated via the network 2.

  The second noise removing unit 17B removes nonsense words and unnecessary descriptions from the information (source code) of a predetermined number of web pages output by the search unit 12B with reference to the tag DB 16B for each web page. I do. The second noise removing unit 17B outputs the content from which the meaningless words have been removed to the keyword extracting unit 18.

  The keyword extracting unit 18 extracts a plurality of keywords from the main content output by the second noise removing unit 17B. The keyword extracting unit 18 performs a sorting process on the extracted keywords as described later, and outputs the sorted keyword list to the keyword list output unit 19.

<Processing procedure of keyword extraction device 1B>
Next, a processing procedure of the keyword extracting device 1B will be described. FIG. 18 is a flowchart of the process of the keyword extracting device 1B according to the present embodiment. Note that the same processes as those of the keyword extraction device 1 (or 1A) are denoted by the same reference numerals and description thereof is omitted.

(Steps S1 to S3) The keyword extraction device 1B performs the processing of steps S1 to S3, and proceeds to the processing of step S15.

(Step S15) The second noise removing unit 17B removes the meaningless words and unnecessary descriptions from the information of the predetermined number of web pages output by the search unit 12B by referring to the tag DB 16B for each web page. I do. Subsequently, the second noise removing unit 17B advances the processing to step S7.

  In the example described above, an example has been described in which the second noise removing unit 17B removes meaningless words and unnecessary descriptions from the source code of the web page, but the present invention is not limited to this. The second noise removing unit 17B may remove at least one of a tag of an item corresponding to a meaningless word and a tag corresponding to an unnecessary description.

  As described above, in the present embodiment, one web page is sequentially selected from a predetermined number of web pages from the top searched for using a search engine. In the present embodiment, the content necessary for keyword extraction is extracted by removing the tag of the description of the item corresponding to the meaningless word and the tag of the unnecessary description from the source code of the selected web page. As a result, in the present embodiment, it is possible to obtain, from a plurality of web pages searched using the search keyword, the content in which the meaningless words as noise and the unnecessary description are removed. As described above, in the present embodiment, a keyword can be accurately extracted by extracting a noun and a compound word using the text from which the noise component has been removed.

  As described above, the keyword extracting device 1B of the present embodiment includes the search unit 12B that searches for a plurality of (predetermined numbers, for example, 20) contents (for example, web pages) including the main content based on the search keyword, By sequentially selecting one content from the plurality of contents searched by the section and removing information described by a predetermined tag from the selected content, unnecessary contents that do not make sense in keyword extraction are removed. The apparatus includes a second noise removing unit 17B for removing the description, and a keyword extracting unit 18 for extracting a keyword from the text of the content from which information described by a predetermined tag has been removed by the second noise removing unit.

With this configuration, in the present embodiment, at least one of the meaningless words and the unnecessary description can be removed by removing the information described by the predetermined tag from the main content. As a result, in the present embodiment, keywords can be accurately extracted from the content from which the meaningless words or unnecessary descriptions have been removed.
For example, when words that are assumed to be unnecessary are registered in a database and words that are assumed to be unnecessary are removed from the contents, valid words may be removed from the contents in some cases. On the other hand, by removing information described by tags as in the present embodiment, meaningless words can be accurately removed.

[First Modification of Second Embodiment]
Next, an example in which the keyword extracting device 1B further includes a domain DB 13 and a first noise removing unit 14 will be described.

<Configuration of Keyword Extraction Device 1C>
FIG. 19 is a schematic configuration diagram of a keyword extracting device 1C according to a first modification of the present embodiment. Note that functional units having the same functions as those of the keyword extracting apparatuses 1, 1A, and 1B are denoted by the same reference numerals and description thereof is omitted.
As shown in FIG. 19, the keyword extracting device 1C includes a keyword input unit 11, a search unit 12, a domain DB 13, a first noise removing unit 14, a tag DB 16B, a second noise removing unit 17B, a keyword extracting unit 18, and a keyword list. An output unit 19 is provided. Note that the keyword extracting device 1C may include, for example, a main content extracting unit 15 (see FIG. 2) between the second noise removing unit 17B and the keyword extracting unit 18.

The first noise removal unit 14 acquires information indicating a predetermined number of web pages output by the search unit 12. The first noise removing unit 14 uses the information indicating the web page of a predetermined number obtained by removing the web pages of a domain that is stored from the web pages of a predetermined number of the domain DB 13, after removal of The information indicating the web page is output to the second noise removing unit 17B. When the predetermined number of web pages do not include the web pages of the domain stored in the domain DB 13, the first noise removing unit 14 sends information indicating the predetermined number of web pages to the second noise removing unit 17B. Output.

  The second noise removing unit 17B deletes the meaningless words and unnecessary descriptions from the information (source codes) of the plurality of web pages output by the first noise removing unit 14 by referring to the tag DB 16B. The second noise removing unit 17B outputs the content from which the meaningless words and unnecessary descriptions have been removed to the keyword extracting unit 18.

<Processing procedure of keyword extracting device 1C>
Next, a processing procedure of the keyword extracting device 1C will be described. FIG. 20 is a flowchart of the process of the keyword extracting device 1C according to the first modification of the present embodiment. The same processes as those of the keyword extracting device 1, 1A, or 1B are denoted by the same reference numerals, and description thereof will be omitted.

(Steps S1 to S3) The keyword extracting device 1C performs the processing of steps S1 to S3, and proceeds to the processing of step S4.
(Step S4) The first noise elimination unit 14 uses the information of the predetermined number of web pages output by the search unit 12 to convert the web pages of the domain stored in the domain DB 13 from the predetermined number of web pages. Remove.
(Step S15) The second noise removing unit 17B deletes the meaningless words and unnecessary descriptions from the information (source codes) of the plurality of web pages output by the first noise removing unit 14 with reference to the tag DB 16B. .

  As described above, in the keyword extracting device 1C of the present embodiment, the first noise removing unit 14 is further provided between the searching unit 12 and the keyword extracting unit 18, and the first noise removing unit is searched by the searching unit. From the plurality of contents (for example, web pages), the content of a predetermined domain that does not make sense in extracting the keyword is removed.

  With this configuration, in the present embodiment, by removing a web page of a predetermined domain, unnecessary web pages that do not make sense as content when extracting a keyword can be removed from a search result. As a result, in the present embodiment, the main content can be accurately extracted only from the web pages from which unnecessary web pages have been deleted.

[Second Modification of Second Embodiment]
Next, a modified example of the keyword extracting device 1B will be described. In the present modification, the search unit searches for a web page of a domain in which the search keyword output by the keyword input unit 11 is predetermined using a search engine.

<Configuration of Keyword Extraction Device 1D>
FIG. 21 is a schematic configuration diagram of a keyword extraction device 1D according to a second modification of the present embodiment.
As shown in FIG. 21, the keyword extraction device 1D includes a keyword input unit 11D, a search unit 12D, a domain DB 13D, a tag DB 16B, a second noise removal unit 17B, a keyword extraction unit 18D, and a keyword list output unit 19. Note that functional units having the same functions as those of the keyword extracting devices 1, 1A, 1B, and 1C are denoted by the same reference numerals and description thereof is omitted. Note that the keyword extracting device 1D may include the first noise removing unit 14 between the search unit 12D and the second noise removing unit 17B, for example, between the first noise removing unit 14 and the keyword extracting unit 18D. May be provided with a main content extraction unit 15.

  The keyword input unit 11D acquires a search keyword and a search target domain input by the user, and outputs the acquired search keyword and information indicating the search target domain to the search unit 12D. The search target domain will be described in an eighth extraction method.

  The domain of the website from which the keyword is extracted is stored in the domain DB 13D. The domain of the website from which keywords are extracted is, for example, a domain of a website that provides a mechanism in which a user publishes a question, solicits an answer, and resolves the question. In the following description, such a site is referred to as a Q & A site. The Q & A site is, for example, Yahoo! (Registered trademark) Chiebukuro (registered trademark), tell me! goo (registered trademark), remark Komachi (registered trademark), OKWave (registered trademark), and the like.

  The search unit 12D searches for a search keyword output by the keyword input unit 11D for a domain stored in the domain DB 13D using a search engine. The search unit 12D selects, for example, a predetermined number of web pages from the top among the web pages obtained by the search, and outputs information indicating the selected predetermined number of web pages to the second noise removal unit 17B.

  When an important keyword extraction method for comparing a plurality of search results is selected, the keyword extraction unit 18D compares the extracted important keywords and generates a keyword list based on the comparison result. The keyword extracting unit 18D outputs the generated keyword list to the keyword list output unit 19.

Next, an example of a domain stored in the domain DB 13D will be described.
FIG. 22 is a diagram illustrating an example of domains stored in the domain DB 13D according to the second modification of the present embodiment. As shown in FIG. 22, the domain DB 13D stores a Q & A site name and information on the domain of the Q & A site in association with each other.

<Example of operation procedure and example of operation screen by keyword extraction device 1D>
Next, an example of an operation procedure and an example of an operation screen by the keyword extraction device 1D will be described.
FIG. 23 is a diagram illustrating an example of an operation screen by the keyword extraction device 1D according to the second modification of the present embodiment.
In the example shown in FIG. 23, the image of the area indicated by reference sign g400 is an image of an area for selecting an important keyword extraction method. An image g400 of an “extraction tool” button for selecting an extraction method, an image g402 of a “co-occurrence word extraction” button for selecting a first extraction method, and a second image g400 of an area g400 for selecting a keyword extraction method. An image g403 of a “co-occurrence word extraction Q & A site 1” button for selecting an extraction method is included. Further, the image g400 includes an image g404 of a "co-occurrence word extraction Q & A site 2" button for selecting a third extraction method, and an image g404 of a "co-occurrence word extraction Q & A site 3" button for selecting a fourth extraction method. An image g405 and an image g406 of a “co-occurrence word extraction Q & A site 4” button for selecting the fifth extraction method are included. Further, the image g400 includes an image g407 of a “co-occurrence word extraction (combination)” button for selecting the sixth extraction method, and a “co-occurrence word extraction (comparison)” button for selecting the seventh extraction method. The image g408 includes an image g409 of an “extraction of co-occurring words (search in site)” button for selecting the eighth extraction method, and an image g410 of an “excess or insufficient keyword in page” button for selecting the ninth extraction method. Have been. Note that a co-occurrence word is a keyword that is different from a certain word that is frequently used in a sentence when the certain word is used in the sentence, and is a keyword extracted in the present invention. The processing of the first to ninth extraction methods will be described later.

In FIG. 23, the image of the area indicated by reference sign g420 is the image of the input area of the search keyword. FIG. 23 shows an example in which the second extraction method is selected. The image g420 of the search keyword input area includes an image g421 of the search keyword input space, an image g422 of the search button, and a predetermined number of images g423 of the web page to be selected.
Further, in FIG. 23, the image of the area indicated by reference numeral g430 is an image indicating the extracted result. The image g430 indicating the extracted result is a partial image of the extracted result, and the remaining search results are displayed when the user scrolls the search results using the scroll button (image g431). Is done.

Here, an outline of the first to ninth extraction methods will be described.
In the first extraction method, a search keyword is input to a search engine to perform a search, and a predetermined number of search result sites are selected from the top of the search results. Then, in the first extraction method, important keywords (co-occurring words) are extracted from the selected site.
In the second extraction method, a search keyword is searched for the Q & A site 1 to extract an important keyword.
In the third extraction method, a search keyword is searched for the Q & A site 2 to extract an important keyword.
In the fourth extraction method, a search keyword is searched for the Q & A site 3 to extract an important keyword.
In the fifth extraction method, a search keyword is searched for the Q & A site 4 to extract an important keyword.

In the sixth extraction method, search keywords are searched for all the Q & A sites 1 to 4 to extract important keywords.
In the seventh extraction method, search keywords are searched for all of the Q & A sites 1 to 4 to extract important keywords, and the first extraction method further extracts important keywords. Then, in the seventh extraction method, the important keywords extracted by searching all the Q & A sites 1 to 4 are compared with the important keywords extracted by the first extraction method.
In an eighth extraction method, important keywords are extracted by the first extraction method, and whether or not the extracted important keywords are included in a site to be evaluated (hereinafter, referred to as an evaluation site). The evaluation is based on the frequency of use.
In the ninth extraction method, keywords are extracted from the evaluation site, and important keywords are further extracted by the first extraction method. Then, in the ninth extraction method, important keywords and excessive keywords that are insufficient for the site to be evaluated are extracted and evaluated.

First, the processing of the first extraction method will be described.
It is assumed that the domain DB 13D stores at least the domains 11 to 14 associated with the Q & A sites 1 to 4 and the address (domain) of the search engine.
When the first extraction method is selected, the search unit 12D searches for the input search keyword using the search engine of the search engine address stored in the domain DB 13D. Subsequently, after removing unnecessary domains from the search result, the search unit 12D selects, for example, the top 20 web pages. Subsequently, the second noise removing unit 17B removes a tag indicating a meaningless word or the like. Subsequently, the keyword extracting unit 18D extracts keywords from the information of the top 20 web pages from which the meaningless words and the like have been removed, and calculates the number of appearances, importance, and the like of the extracted keywords. Subsequently, the keyword list output unit 19 outputs the extracted keywords in a list format, for example, as shown in FIG.

Next, the processing of the second to fifth extraction methods will be described.
When the second extraction method is selected, the search unit 12D selects the domain 11 associated with the Q & A site 1 stored in the domain DB 13D, and the search keyword input using the selected domain 11 Search for.
Similarly, when the n-th (n is 3 to 5) extraction method is selected, the search unit 12D selects the domain 1 (n) associated with the Q & A site n stored in the domain DB 13D. Then, the input search keyword is searched using the selected domain 1 (n).
Subsequently, the search unit 12D selects, for example, the top 20 web pages from the search result. Subsequently, the second noise removing unit 17B removes a tag indicating a meaningless word or the like. The information of the web page after the removal by the second noise removing unit 17B includes at least the text of the question part of the Q & A site, and may include the text of the part such as times. Subsequently, the keyword extracting unit 18D extracts keywords from the information of the top 20 web pages from which the meaningless words and the like have been removed, and calculates the number of appearances, importance, and the like of the extracted keywords. Subsequently, the keyword list output unit 19 outputs the extracted keywords in a list format, for example, as shown in FIG. That is, the difference between the second extraction method to the fifth extraction method and the first extraction method is that the search target web page of the first extraction method is not limited, but the second extraction method to the fifth extraction method are different. Is that the search target web page is limited to the Q & A site.

Next, the processing of the sixth extraction method will be described.
When the sixth extraction method is selected, the search unit 12D selects all the domains 11 to 14 corresponding to the Q & A sites 1 to 4 stored in the domain DB 13D, and selects the selected domain 11 Search for the entered search keyword using all of the domains 14. Subsequently, the search unit 12D selects a predetermined number of web pages from the top from each of the search results of the domains 11 to 14. Subsequently, the second noise removing unit 17B removes a tag indicating a meaningless word or the like. Subsequently, the keyword extracting unit 18D extracts a keyword from information of the top web page from which the meaningless words and the like have been removed, and calculates the number of appearances, importance, and the like of the extracted keyword. Subsequently, the keyword list output unit 19 outputs the extracted keywords in a list format, for example, as shown in FIG.

  As described above, in the keyword extraction device 1D according to the second modification of the present embodiment, the search unit 12D limits the domain in which the content is searched based on the search keyword (for example, limits the search domain to the Q & A site). The keyword extracting unit extracts a plurality of keywords from the text of the content of the limited domain, and generates a keyword list based on the extracted result.

  With this configuration, according to the present embodiment, a keyword (co-occurrence word) corresponding to the search keyword used in the Q & A site can be extracted.

Next, a seventh extraction method will be described.
When the seventh extraction method is selected, the search unit 12D selects all the domains 11 to 14 corresponding to the Q & A sites 1 to 4 stored in the domain DB 13D, and selects the selected domain 11 Search for the entered search keyword using all of the domains 14. Further, the search unit 12D searches the search keyword input using the search engine, and outputs the result of searching all the domains 11 to 14 and the result of the search using the search engine to the second noise removal unit 17B. Output. Subsequently, the second noise removing unit 17B removes a tag indicating a meaningless word or the like. Subsequently, the keyword extracting unit 18D extracts a keyword from each of the information of the top web pages from which the meaningless words and the like have been removed, and calculates the number of appearances, importance, and the like of the extracted keyword.
In this case, as shown in FIG. 24, the keyword extracting unit 18D compares the result of searching all the domains 11 to 14 (image g380) with the result of searching using the search engine (image g370), and compares the results. A keyword list is generated based on the result.

FIG. 24 is a diagram illustrating a comparative example of a keyword search result obtained by the keyword extraction device 1D using the seventh extraction method according to the second modification of the present embodiment.
As shown in FIG. 24, each search result includes an image of the number of appearances (g372, g382), an image of the value obtained by multiplying the number of appearances by importance (g373, g383), and an image of the extracted important keyword (g374, g374). g384).
Further, as shown in FIG. 24, when the result of searching all the domains 11 to 14 (image g380) and the result of searching using the search engine (image g370) are different, the different important keywords are different. The display method may be changed. The keyword extracting unit 18D may perform, for example, the color of the character, the thickness of the character, the type of the font, and combining a colored marker with the character. In the example shown in FIG. 24, the keyword indicated by g385 indicates that the search rank is, for example, 61 or less among the keywords that have a high frequency of appearance of Q & A sites but are not frequently used in the top sites searched by the search engine. Is shown. The keyword indicated by reference sign g386 indicates that the search ranking is, for example, from the 31st to the 60th among keywords that are frequently used in the Q & A site but are not often used in the top sites searched by the search engine. .

  In the present embodiment, an example in which the result of searching all the domains 11 to 14 and the result of searching using a search engine are compared in the seventh extraction method has been described, but the present invention is not limited to this. For example, the keyword extracting unit 18D may generate a keyword list by comparing results searched in the domain 11 and the domain 12.

  As described above, according to the second to seventh extraction methods, keywords are extracted from the Q & A site. It is highly likely that the meaning of the information that the person using the search engine wants to know is included in the question column of the Q & A site. Therefore, there is a high possibility that a person who wants to know information inputs a keyword included in the question column of the Q & A site as a search keyword when searching using a search engine. Therefore, if a co-occurrence word, which is an important keyword included in the Q & A site, is used in, for example, its own site, a person who wants to know information is displayed as a top search result when searching using a search engine. Can be increased.

  As described above, in the keyword extraction device 1D according to the second modification of the present embodiment, the search unit 12D includes at least two domains (for example, the Q & A sites 1 to 4 and the search At least two of the engines search for different contents, and the keyword extracting unit 18D extracts a plurality of keywords from each of the texts of the contents of the different domains, and compares the keywords extracted from the respective texts of the contents of the different domains. Then, a keyword list is generated based on the comparison result.

With this configuration, according to the present embodiment, an extracted keyword (co-occurrence word) corresponding to a search keyword used in a Q & A site is compared with a keyword (co-occurrence word) extracted by a search engine. be able to.
For example, a keyword (co-occurrence word) extracted from a question on the Q & A site is likely to include a keyword that the user wants to know most. On the other hand, a keyword (co-occurrence word) extracted from a web page searched by a search engine may not be a keyword of information that the user wants to know most, even if the keyword is frequently used for content. Therefore, by comparing these keywords (co-occurrence words) and creating a content including the keyword (co-occurrence word) that the user wants to know most, it is possible to provide information that the user wants to know. Will be possible.

Next, an eighth extraction method will be described.
In the eighth extraction method, when a search is performed with the input search keyword, it is determined whether the keyword included in the top site of the search result is written in the evaluation target site (for example, the company's own site) without any excess or shortage. .

The operation screen in the eighth extraction method will be described.
FIG. 25 is a diagram illustrating an example of an operation screen of the keyword extraction device 1D when the eighth extraction method according to the second modification of the present embodiment is selected. FIG. 25 is a diagram showing an image g420A of the input area of the search keyword and an image g440 of the search result extracted.
As shown in FIG. 25, the image g420A of the search keyword input area includes an image g421 of a search keyword input field, an image g422 of a search button, an image g423 for selecting a predetermined number, and an image g424 of a search target domain input field. It is included.

The search result image g440 includes an image g441 indicating a search keyword, an image g442 indicating a search domain, an image g443 of the number of appearances, an image g444 of a value obtained by multiplying the number of occurrences by importance, and an extracted important keyword (common Image g445 is included. Further, the image g440 of the search result shows an image g446 indicating the ranking of the important keyword in the web page in the evaluation site where the important keyword is used, and the address of the web page in the evaluation site where the important keyword is used. An image g447 is included.
If an important keyword is not included in the web page in the domain (it is not used), the order is set to, for example, 50th or higher, "50+" is displayed in the image g436, and a blank is displayed in the image g447. It may be.

Next, the processing procedure of the eighth extraction method will be described.
FIG. 26 is a flowchart of a process in the eighth extraction method according to the second modification of the present embodiment.

(Step S401) The user selects the eighth extraction method, inputs a search keyword in the search keyword input field (image g421), and further inputs a domain to be evaluated in the domain input field (image g424). Subsequently, the keyword input unit 11D acquires the input search keyword and the domain to be evaluated. The domain to be evaluated is a domain for searching for a web page including the extracted important keyword (co-occurrence word), and is, for example, a domain of a site of a company to be evaluated.

(Step S2) The search unit 12D searches for a search keyword using a search engine. Note that the search method may be the Q & A site described in the second to fifth extraction methods.

Subsequently, the search unit 12D performs Step S3. Subsequently, the second noise removing unit 17B performs the process of step S15. Subsequently, the keyword extracting unit 18D performs the process of step S7.
(Step S402) The keyword extracting unit 18D sequentially selects the extracted important keywords (co-occurrence words).
(Step S403) The keyword extracting unit 18D determines whether the selected important keyword is included in the web page of the input domain. When the keyword extracting unit 18D determines that the selected keyword is included in the web page of the input domain (step S403; YES), the process proceeds to step S404. When the keyword extracting unit 18D determines that the selected keyword is not included in the web page of the input domain (step S403; NO), the process returns to step S402.

(Step S405) The keyword extracting unit 18D acquires an address of a web page including the keyword selected in step S402.
(Step S405) The keyword extracting unit 18D determines whether or not all important keywords extracted in step S6 have been selected. When determining that all important keywords have been selected (step S405; YES), the keyword extracting unit 18D proceeds to step S406 and determines that all important keywords have not been selected (step S406). S405; NO), and returns to the process of step S402.

(Step S406) The keyword extracting unit 18D generates a keyword list by associating an important keyword with an address.
This is the end of the processing of the eighth extraction method.

  As described above, according to the eighth extraction method, when a person who wants to know information searches with a search engine using a search keyword that is assumed to be input at the time of search, the search result is included in the top sites in the search results It can be determined whether or not the keyword is also used for a site to be evaluated (for example, its own site).

Next, a ninth extraction method will be described.
In the ninth extraction method, first, important keywords are extracted by the first extraction method. Further, a keyword having a high appearance frequency (use frequency) is extracted from the information on the domain of the site to be evaluated or the address of the web page (hereinafter referred to as information on the evaluation site). Then, in the ninth extraction method, the extracted important keywords are compared with the keywords on the evaluation site to be extracted, and the important keywords that are lacking in the evaluation site are extracted.

FIG. 27 is a diagram illustrating an example of an evaluation result by the keyword extraction device when the ninth extraction method according to the second modification of the present embodiment is selected.
As shown in FIG. 27, the image g450 of the evaluation result includes an image g451 indicating the search keyword, an image g452 indicating the information of the evaluation site, an image g453 indicating the keyword extracted from the evaluation site, and the top rank searched by the search engine. An image g454 indicating an important keyword extracted from the site and an image g455 indicating an excess / shortage keyword are included.
In the image g453 indicating the keyword extracted from the evaluation site, the keyword (image g4533) is displayed in descending order of the value (image g4532) obtained by multiplying the number of appearances (image g4531) by the importance. In addition, the image g454 indicating the important keyword extracted from the top site includes an image g4541 of the number of appearances, an image g4542 of a value obtained by multiplying the number of appearances by the importance, and an image g4543 of the important keyword (co-occurrence word). I have.

  For example, the image g455 indicating the excess / shortage keyword includes an image g4551 of a list of shortage 1 (an important keyword that should be added to the evaluation site; a keyword with a low frequency on the evaluation site), and a shortage 2 (a keyword added to the evaluation site). Image g4552 of a list of important keywords; keywords not used on the evaluation site). Further, the image g455 indicating the excess / shortage keyword includes an image g4553 of a list of excess 1 (a keyword that is high in the evaluation site but low in frequency in the top site) and an excess 2 (a keyword that is not used in the top site but is used in the evaluation site. The image g4554 of the list of keywords with high frequency is included. Since the missing keyword is a co-occurrence word extracted from a higher-level site by a search engine, it is information that a purchaser or a user of a product wants to know about information on a product or service. On the other hand, excessive keywords may be excessive information for buyers and users of products.

Here, an example of a method for detecting a missing keyword and a method for detecting an excessive keyword will be described.
The keyword extracting unit 18D sequentially selects one of the important keywords extracted from the top sites. Then, the keyword extracting unit 18D detects a missing keyword by sequentially comparing the selected important keyword with the keyword extracted from the evaluation site.
The keyword extracting unit 18D sequentially selects one of the keywords extracted from the evaluation site. Then, the keyword extracting unit 18D detects an excessive keyword by sequentially comparing the selected keyword with an important keyword extracted from the top site.

Next, a usage example of the ninth extraction method will be described.
The site operator inputs the address of the company's own site as an evaluation site, for example. Then, by using the evaluation results to know the keywords that are lacking in the company site, the company site can be improved.
The evaluation result shown in FIG. 27 is an example, and the present invention is not limited to this. Any information may be used as long as the information is based on a result of comparison between an evaluation site and a top site.

  As described above, in the second modification of the present embodiment, when searching for a keyword (co-occurrence word), the user can select the domain of the web page to be searched. Thus, the user can, for example, extract a keyword by each extraction method and compare the obtained extraction results. Further, as described above, in the second modification of the present embodiment, a plurality of Q & A sites are searched to extract keywords, and therefore, keywords (co-occurring words) in a well-balanced manner are extracted from sentences that have been a topic at each Q & A site. Can be extracted.

  The search unit 12D may extract a keyword (co-occurrence word) corresponding to the search keyword from the question on the Q & A site. In this case, the keyword extracting device 1D includes a main content extracting unit 15, and extracts a question text from the main content extracted by the main content extracting unit 15.

  As described above, in the keyword extraction device 1D according to the second modification of the present embodiment, the search unit 12D searches for content using the search engine based on the search keyword, and the keyword extraction unit 18D searches for the text of the content. The keyword extraction unit extracts a plurality of keywords from the text of the content by searching based on the plurality of keywords extracted from the search result and searches the ranking of the search result of the site to be evaluated. The keyword list is generated based on the result of determining whether or not the content is used in the content of the site and the search order of the evaluation target site searched by the search unit.

  With this configuration, according to the present embodiment, it is possible to provide whether or not the keyword (co-occurrence word) extracted by the search engine is included in the evaluation site, and the ranking when the search engine performs the search.

[Third embodiment]
In the present embodiment, a keyword extraction device 1E capable of evaluating the quality of a web page will be described.
The keyword extracting device 1E of the present embodiment extracts the main content of the input web page, and extracts a predetermined number of sentences from the extracted main content. Then, the keyword extracting device 1E searches the extracted text by a search engine, and evaluates the web page based on the search result.

<Configuration of Keyword Extraction Device 1E>
FIG. 28 is a schematic configuration diagram of a keyword extraction device 1E according to the present embodiment. Note that functional units having the same functions as those of the keyword extracting devices 1, 1A, 1B, 1C, and 1D are denoted by the same reference numerals and description thereof is omitted.
As shown in FIG. 28, the keyword extraction device 1E includes a keyword input unit 11E, a search unit 12E, a main content extraction unit 15, a tag DB 16, a second noise removal unit 17E, a text extraction unit 20, a search rank acquisition unit 21, and an evaluation result. A generation unit 22 and an evaluation result output unit 23 are provided.

  The keyword input unit 11E acquires information indicating the address of the web page input by the user, and outputs the information indicating the acquired address of the web page to the search unit 12E and the search order acquisition unit 21.

The search unit 12E inputs the address of the web page output by the keyword input unit 11E to a search engine, and searches for the web page. The search unit 12E acquires the source code of the searched web page, and outputs the acquired source code to the main content extraction unit 15.
Further, the search unit 12E acquires the text output by the text extraction unit 20, and sequentially selects one of the acquired documents. The search unit 12E sequentially inputs the selected sentences to a search engine to search. The search result includes the source code. Then, the search unit 12E sequentially outputs the selected text and the search result to the main content extraction unit 15.

  The second noise removing unit 17E removes meaningless words and the like from the main content output by the main content extracting unit 15 with reference to the tag DB16. The second noise removing unit 17 </ b> E outputs the main content from which the meaningless words and the like have been removed to the text extracting unit 20. The meaningless words include advertisements related to the text used for the search. The second noise removing unit 17E sequentially outputs the main content from which the meaningless words and the like have been removed and the selected text to the search order obtaining unit 21.

  The text extraction unit 20 extracts a predetermined number of texts (texts) from the main content output by the second noise removal unit 17E. The predetermined number may be one or more, may be a fixed value, or may be a number set according to the total number of characters of the main content. The text extraction unit 20 outputs the extracted text to the search unit 12E. When the extracted text has a predetermined number of characters or more, the text extraction unit 20 may extract a predetermined number of characters from the head of the document and treat the text as one text.

  The search order obtaining unit 21 sequentially obtains the selected sentence output by the second noise removing unit 17E and the main content in the search result from which the meaningless words and the like have been removed. In addition, the search order obtaining unit 21 obtains information indicating the address of the web page output by the keyword input unit 11E. The search order obtaining unit 21 obtains the order of the web page in the search result using the obtained main content and the address of the web page, and sequentially associates the obtained order with the selected text to the evaluation result generation unit 22. Output.

  The evaluation result generation unit 22 assigns scores to the rankings output by the search ranking acquisition unit 21 and evaluates each sentence. The evaluation result generation unit 22 sums up the points of each sentence, generates an evaluation result according to the total score, and outputs the generated evaluation result to the evaluation result output unit 23. Note that the points for the rankings and the evaluation results will be described later.

  The evaluation result output unit 23 is, for example, at least one of an information providing unit on the Web, a display device, a printer device, and a communication device. The evaluation result output unit 23 provides the evaluation result output by the evaluation result generation unit 22 on, for example, the Web.

<Evaluation procedure>
Next, a procedure of an evaluation process performed by the keyword extracting device 1E will be described.
FIG. 29 is a flowchart of an evaluation process performed by the keyword extraction device 1E according to the present embodiment.
(Step S501) The keyword input unit 11E acquires information indicating an address of a web page input by a user.
(Step S502) The search unit 12E acquires the source code of the web page.
Subsequently, the main content extraction unit 15 performs the process of step S5, and after the process ends, proceeds to step S503.

(Step S503) The text extracting unit 20 extracts a predetermined number of texts (texts) from the main content output by the second noise removing unit 17E.
(Step S504) The search unit 12E acquires the text output by the text extraction unit 20, and sequentially selects one of the acquired documents. Subsequently, the search unit 12E sequentially inputs the selected sentences to a search engine to search. Subsequently, the main content extraction unit 15 extracts main content from the search result output by the search unit 12E. Subsequently, the second noise removing unit 17E removes nonsense words including advertisements from the main content output by the main content extracting unit 15.

(Step S505) The search order obtaining unit 21 sequentially obtains the selected text output by the second noise removing unit 17E and the main content in the search result from which the meaningless words have been removed. Subsequently, the search order obtaining unit 21 obtains the order of the web page in the search result.
(Step S506) The evaluation result generation unit 22 assigns a score to the ranking output by the search ranking acquisition unit 21, and evaluates each sentence. Subsequently, the evaluation result generation unit 22 sums up the points of each sentence and generates an evaluation result according to the total points. Subsequently, the evaluation result output unit 23 outputs the evaluation result output by the evaluation result generation unit 22.
Thus, the evaluation processing ends.

<Example of score and evaluation result>
Next, an example of a score and an evaluation result will be described.
FIG. 30 is a diagram illustrating an example of an evaluation result according to the present embodiment. The example shown in FIG. 30 is an example of an evaluation result for two web pages. As shown in FIG. 30, the output determination result includes the address of the web page, the total score, the obtained sentence + rank, advice, and the evaluation date.

First, the allocation will be described.
Generally, a user of a search engine browses search contents from the top of the search results. For example, when a search result browses a search result of the first place and obtains information that the user wants to know, there are many cases where another search result is not browsed. And, in many cases, the user of the search engine does not browse the search results whose search results are 20th or less. Therefore, it can be said that the higher the search result is, the more the sentence used in the search is superior to other web pages. In addition, when the ranking is low, it means that the sentence used for the search is also used for another web page, and thus it can be said that the superiority to other web pages is low.

When five sentences are selected, the evaluation result generation unit 22 assigns 20 points (= 100/5) to each sentence. Then, the evaluation result generation unit 22 assigns, for example, 20 points to the first place, 16 points to the second place, 12 points to the third place,...
When the total score of the five sentences is 100, the evaluation result generation unit 22 determines that the judgment result is “」 ”or“ Very Good ”, and when the total score is less than 100, the judgment result is Is determined to be “×” or “No Good”.
Note that the above-described scoring and determination are merely examples, and the present invention is not limited to this.

  The evaluation result surrounded by reference numeral g501 in FIG. 30 is the evaluation result for the web page “http: //www.abcdef.html”. The five extracted sentences are sentences 1 to 5. As a result of inputting each sentence to the search engine and searching, the respective rankings are third, first, first, twentieth or higher, and first. Then, the total score is 72 points, and the judgment is “x”. Also, the advice is "I strongly recommend that you copy or rewrite."

The evaluation result surrounded by reference numeral g502 in FIG. 30 is the evaluation result for the web page “http: //www.abcdfg.html”. The evaluation result enclosed by reference sign g502 is an example of the result of extracting and evaluating ten sentences from the main content. In the example extracted from this web page, among the extracted ten sentences, the first place is two places, the top 20 places are eight places, and the total score is 20 points. And the advice is "I strongly recommend that you copy or rewrite."
The text of the advice may be stored in advance in the evaluation result generation unit 22 in association with the total score.

  In the example illustrated in FIG. 28, the keyword extracting device 1E may not include the tab DB 16 and the second noise removing unit 17E. In this case, the main content extracting unit 15 may output the extracted main content to the text extracting unit 20 and output the main content of the search result to the search order obtaining unit 21.

  As described above, in the keyword extraction device 1E according to the present embodiment, the sentence extraction unit 20 that extracts at least one sentence from the main content extracted by the main content extraction unit 15 and the order searched based on the sentence by the search unit 12E. And an evaluation result generation unit 22 that evaluates a web page to be evaluated from which sentences have been extracted based on the ranking acquired by the search ranking acquisition unit. .

  With this configuration, according to the present embodiment, main content is extracted from a web page to be evaluated, and at least one sentence is extracted from the extracted main content. In the present embodiment, the extracted text is searched using a search engine, and the web page is evaluated based on the ranking of the website including the document used for the search. Thus, according to the present embodiment, the web page operator can obtain the evaluation result of the quality of the web page of the company only by inputting the web page address into the keyword extracting device 1E.

[Fourth embodiment]
In this embodiment, an example in which a keyword is extracted and evaluated using a suggestion function of a search engine will be described.
First, the suggestion function will be described.
The suggestion function is a function in which when a search engine is used to search for a word, the search engine proposes a word that is highly likely to be searched by a user of the search engine. For example, when "cheap SIM" is input to the search engine, candidates such as "cheap sim", "cheap sim comparison", and "cheap sim tethering" are proposed. As described above, the suggested words are frequently searched by users of the search engine, that is, information that the users want to know in many cases.
In the present embodiment, words suggested for a search word are collected, and the evaluation sites are evaluated according to the frequency at which the collected words are included in an evaluation site (for example, a company's own site).

<Configuration of Keyword Extraction Device 1F>
FIG. 31 is a schematic configuration diagram of a keyword extraction device 1F according to the present embodiment.
As shown in FIG. 31, the keyword extraction device 1F includes a keyword input unit 11F, a search unit 12F, a main content extraction unit 15, a suggestion acquisition unit 24, a search order acquisition unit 21F, an evaluation result generation unit 22F, and an evaluation result output unit 23. Is provided. The keyword extracting device 1F is connected to the network 2. Note that functional units having the same functions as those of the keyword extracting apparatuses 1, 1A, 1B, 1C, 1D, 1E, and 1F are denoted by the same reference numerals and description thereof is omitted. The keyword extracting device 1F may include the second noise removing unit 17 (or 17B, 17E) and the tag DB 16 between the main content extracting unit 15 and the search order obtaining unit 21F.

  The keyword input unit 11F acquires a search keyword input by the user, and outputs the acquired search keyword to the search unit 12F. The keyword input unit 11F acquires information on the evaluation site input by the user, and outputs the acquired information on the evaluation site to the search order acquiring unit 21F.

The search unit 12F inputs the search keyword output by the keyword input unit 11F to a search engine, and outputs a word proposed when the search keyword is input (hereinafter, a predicted word) to the suggestion acquisition unit 24. The predicted words include at least a search keyword, such as a combination of the search keyword and another word, a compound word including the search keyword, and the like.
The search unit 12F selects one of the predicted words output from the suggestion acquisition unit 24, and inputs the selected predicted word to a search engine to search. Then, the search unit 12F sequentially outputs the search results to the main content extraction unit 15.

The suggestion acquisition unit 24 acquires the predicted words output by the search unit 12F, and outputs the obtained predicted words to the search unit 12F and the evaluation result generation unit 22F.
The main content extraction unit 15 extracts the main content from the source code as a result of the search performed by the search unit 12F using the predicted words, and outputs the extracted main content to the search order obtaining unit 21F.

  The search order acquiring unit 21F acquires the main content output by the main content extracting unit 15 and the information of the evaluation site output by the keyword input unit 11F. The search order obtaining unit 21F obtains the order of the evaluation site in the search result, and sequentially outputs the obtained order to the evaluation result generating unit 22F in association with the predicted word.

  The evaluation result generation unit 22F evaluates each predicted word using the order and the predicted word output by the search order obtaining unit 21F. The evaluation result generation unit 22F generates an evaluation result based on the evaluation result, and outputs the generated evaluation result to the evaluation result output unit 23. Alternatively, the evaluation result generation unit 22F generates an evaluation result using the predicted words output by the suggestion acquisition unit 24, and outputs the generated evaluation result to the evaluation result output unit 23. The evaluation result will be described later.

<Evaluation procedure>
Next, a procedure of an evaluation process performed by the keyword extracting device 1F will be described.
FIG. 32 is a flowchart of an evaluation process performed by the keyword extraction device 1F according to the present embodiment.
(Step S601) The keyword input unit 11F acquires a search keyword input by a user and information on an evaluation site.
(Step S602) The search unit 12F inputs the search keyword output by the keyword input unit 11F to the search engine. Subsequently, the suggestion obtaining unit 24 obtains the proposed predicted words. It should be noted that the number of predicted words to be acquired may be all presented or may be a predetermined number.

(Step S603) The search unit 12F, the search order acquisition unit 21F, and the evaluation result generation unit 22F perform the processing of steps S604 to S606 for each predicted word.
(Step S604) The search unit 12F selects one of the predicted words output from the suggestion acquisition unit 24, and inputs the selected predicted word to a search engine to search.

(Step S605) The search order obtaining unit 21F obtains the order of the evaluation site output by the keyword input unit 11F in the search result output by the search unit 12F.
(Step S606) The evaluation result generation unit 22F makes a determination on each predicted word using the order and the predicted word output by the search order obtaining unit 21F. For example, the evaluation result generation unit 22F may evaluate “exclusive” when the ranking is 1st to 10th, and determine “unexcavated” when the ranking is 11th or less. Good.

(Step S607) When the processing of steps S604 to S606 ends for the predicted word acquired in step S602, the search unit 12F, the search order obtaining unit 21F, and the evaluation result generation unit 22F proceed to the processing of step S608. .
(Step S607) The evaluation result generation unit 22F generates an evaluation result based on the evaluation result. Subsequently, the evaluation result output unit 23 provides the evaluation result output by the evaluation result generation unit 22F, for example, on the Web.
Thus, the evaluation processing ends.

<Example of evaluation result>
Next, an example of an evaluation result will be described.
FIG. 33 is a diagram illustrating an example of an evaluation result according to the present embodiment.
As shown in FIG. 33, the image g600 indicating the evaluation result includes an image g601 of the search keyword, an image g602 indicating the evaluation site, and an image g603 indicating the search result and the evaluation result.
The image g603 indicating the search result and the evaluation result includes an image g6031 indicating the predicted word, an image g6032 indicating the determination result, and an image g6033 indicating the order.
As shown in an image g6032 showing the determination result, advice such as “exclusive”, “coexistence”, “undiscovered”, “improved”, etc. is given according to the order in which the predicted word (image g6031) is input to the search engine and searched. Is shown. Note that the determination result illustrated in FIG. 33 is an example, and the evaluation result generation unit 22F may assign a label of the determination result for every tenth rank, for example, the first to tenth ranks, the eleventh to 20th ranks, and the like. Good. In addition, the evaluation result generation unit 22F color-codes each of the words “exclusive”, “coexistence”, “undiscovered”, and “improved”, color-codes in the order, and assigns a marker to each word or character. Alternatively, the types of words or characters may be different.

  The operator of the evaluation site constructs, for example, its own site so that the evaluation site is searched at a higher rank even when the search is performed using the predicted words by using such an evaluation result. This has the effect of increasing the number of accesses to the company's site.

  Note that, in the examples shown in FIGS. 32 and 33, an example has been described in which a predicted word is searched for by a search engine and the ranking of the evaluation site is also obtained. The keyword extraction device 1F may acquire a predicted word corresponding to the search keyword and output the obtained predicted word as an evaluation result.

  As described above, in the keyword extraction device 1F according to the present embodiment, the suggestion obtaining unit 24 that obtains a predicted word based on the search keyword from the search result obtained by the search unit 12F, and the plurality of predictions that are obtained by the suggestion obtaining unit. A search order obtaining unit 21F that obtains a search order of the selected predicted word using the main content extracted by the main content extraction unit 15 from a result of selecting one of the words and searching the selected predicted word by the search unit. And an evaluation result generation unit 22F that evaluates a web page to be evaluated based on the ranking acquired by the search ranking acquisition unit.

  With this configuration, according to the present embodiment, when a search using a search engine is performed using predicted words that are considered to be frequently input, based on the ranking of the evaluation site to be evaluated, Evaluate the quality of the evaluation site. Thus, according to the present embodiment, for example, the evaluation of the company site is performed using not only the search keyword input for evaluation by the company site operator but also the search keyword used by the user. Can be.

[Fifth Embodiment]
In the present embodiment, an example will be described in which the content generation device 3 is connected to any one of the keyword extraction devices 1, 1A to 1F.
FIG. 34 is a configuration diagram illustrating the content generation system 5 according to the present embodiment.
As shown in FIG. 34, the content generation system 5 includes a keyword extraction device (any one of 1, 1A to 1F) and a content generation device 3. The content generation system 5 is connected to the network 2.
In the following example, the keyword extraction device 1A will be described as an example.

The keyword list output unit 19 of the keyword extracting device 1A is a communication device.
The keyword extracting device 1A extracts a plurality of keywords based on the input search keywords, and outputs to the content generating device 3 information of a keyword list obtained by sorting the extracted keywords.

The content generation device 3 includes a content template storage unit 31, a content generation unit 32, and a content output unit 33.
The content model storage unit 31 stores a content model. Note that the content template is a web page template, catalog template, pamphlet template, instruction manual template, and the like. For example, a template is stored for each product.

  The content generation unit 32 generates content using the extracted plurality of keywords and the content templates stored in the content template storage unit 31, and outputs the generated content to the content output unit 33. Here, the content is a web page, a catalog, a pamphlet, an instruction manual, or the like.

  The content output unit 33 is, for example, at least one of an information providing unit on the Web, a display device, a printer device, and a communication device. The content output unit 33 provides the content output by the content generation unit 32, for example, on the Web.

  In the present embodiment, an example has been described in which content is generated using a content template and a plurality of extracted keywords, but the present invention is not limited to this. The content generation device 3 uses a plurality of keywords extracted by the keyword extraction device (any one of 1, 1A, 1B, 1C, and 1D) to generate a content by a program that automatically generates a well-known sentence. May be generated. In this case, the number of keywords used for one sentence may be set in advance. Further, the number of times of use in the content for each keyword may be set based on, for example, the degree of importance or the number of applications.

  As described above, the content generation system 5 of the present embodiment uses the keyword extraction device (any one of 1, 1A to 1F) and a plurality of keywords extracted by the keyword extraction device to perform a predetermined process. A content generation device 3 for generating content.

  With this configuration, in the present embodiment, content can be generated using a plurality of keywords extracted by the keyword extraction device (any one of 1, 1A to 1F). As a result, according to the present embodiment, it is possible to provide content using information that the user wants to know.

In the first embodiment and the second embodiment described above, the keyword extraction device (any one of 1, 1A to 1F) searches the network 2 for a web page corresponding to a search keyword. Although described, it is not limited to this. For example, a web page corresponding to the search keyword may be searched from a server (not shown) connected to the keyword extraction device (one of 1, 1A to 1F). In this case, the server stores information about a plurality of web pages corresponding to the search keyword.
In the above-described first to fourth embodiments, the domain DB 13 (or 13D) and the tab DB 16 (or 16B) may be on the network 2.

  Further, a part or all of the keyword extracting device (one of 1, 1A, 1B, 1C, 1D, 1E, and 1F) or the content generating device 3 in the above-described embodiment is realized by a computer. You may. In that case, a program for realizing the functions of these devices may be recorded on a computer-readable recording medium, and the program recorded on this recording medium may be read and executed by a computer system. Good. Here, the “computer system” is a computer system built in the recognition data transmission device, and includes hardware such as an OS and peripheral devices. The “computer-readable recording medium” refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, and a CD-ROM, and a storage device such as a hard disk built in a computer system. Further, the "computer-readable recording medium" is a medium that holds the program dynamically for a short time, such as a communication line for transmitting the program through a network such as the Internet or a communication line such as a telephone line, In this case, a program holding a program for a certain period of time, such as a volatile memory in a computer system serving as a server or a client, may be included. Further, the above-mentioned program may be a program for realizing a part of the above-mentioned functions, or may be a program which can realize the above-mentioned functions in combination with a program already recorded in a computer system.

1, 1A, 1B, 1C, 1D, 1E, 1F: keyword extraction device, 2: network, 3: content generation device, 5: content generation system, 11, 11D, 11E, 11F: keyword input unit, 12, 12B, 12D, 12E, 12F: search unit, 13, 13D: domain DB, 14: first noise removal unit, 15: main content extraction unit, 16, 16B: tag DB, 17, 17B, 17E: second noise removal unit, 18 , 18D: Keyword extraction unit, 19: Keyword list output unit, 181: Morphological analysis unit, 182: Term extraction unit, 183: Keyword list generation unit, 20: Sentence extraction unit, 21, 21F: Search order acquisition unit, 22, 22F ... Evaluation result generation unit, 23 ... Evaluation result output unit, 24 ... Suggest acquisition unit, 31 ... Content model storage unit, 32 ... Ceiling generation unit, 33 ... content output unit

Claims (11)

A search unit for searching a plurality of contents including a main content based on a search keyword;
A first noise removing unit that removes, from the plurality of contents searched by the search unit, content of a predetermined domain that does not make sense in extracting a keyword,
One of the plurality of contents from which the contents of the predetermined domain have been removed by the first noise removing unit is sequentially selected, and information indicating a link destination is extracted from the selected contents, and the extracted link is extracted. A main content extraction unit that extracts the main information by removing the similar information by comparing the previous information and the information of the selected content from the information of the selected content,
A keyword extraction unit that extracts a plurality of keywords from the text of the main content extracted by the main content extraction unit,
A keyword extraction device comprising: A second noise elimination for removing unnecessary description that does not make sense in the keyword extraction by removing information described by a predetermined tag from the information of the main content extracted by the main content extraction unit. Part, further comprising:
The keyword extracting unit includes:
Extracting a keyword from the text of the main content after the information described by the predetermined tag is removed by the second noise removing unit;
The keyword extracting device according to claim 1. A search unit for searching a plurality of contents including a main content based on a search keyword;
A first noise removing unit that removes, from the plurality of contents searched by the search unit, content of a predetermined domain that does not make sense in extracting a keyword,
Sequentially selecting one of the plurality of contents from which the content of the predetermined domain has been removed by the first noise removing unit , and removing information described by a predetermined tag from the selected content; A second noise removing unit that removes unnecessary descriptions that do not make sense in keyword extraction;
Sequentially selecting one of the contents from the content from which the information described by the predetermined tag has been removed from the content by the second noise removing unit, extracting information indicating a link destination from the selected content, A main content extraction unit that extracts information of a link destination and information similar to the information of the selected content by comparing the information of the selected content from the information of the selected content to extract a main content.
A keyword extraction unit that extracts a plurality of keywords from the text of the main content extracted by the main content extraction unit ,
A keyword extraction device comprising: The search unit searches for different content of at least two predetermined domains based on the search keyword,
The keyword extracting unit extracts a plurality of keywords from each of the texts of the content of the different domains, compares the keywords extracted from the texts of the content of the different domains, and generates a keyword list based on the comparison result. The keyword extracting device according to any one of claims 1 to 3 . A sentence extracting unit for extracting at least one sentence from the main content extracted by the main content extracting unit;
A search order obtaining unit that obtains the order searched based on the sentence by the search unit;
An evaluation result generation unit that performs an evaluation on a target web page on which the sentence is extracted based on the ranking acquired by the search ranking acquisition unit;
The keyword extraction device according to any one of claims 1, 2, and 3 , further comprising: A suggestion acquiring unit for acquiring a predicted word based on a search keyword from a result searched by the search unit;
Selecting one of the plurality of predicted words acquired by the suggestion acquiring unit, and using the main content extracted by the main content extracting unit from a result of searching the selected predicted word by the search unit; A search order obtaining unit for obtaining the search order of the selected predicted word;
An evaluation result generation unit that evaluates a web page to be evaluated based on the ranking acquired by the search ranking acquisition unit,
The keyword extraction device according to any one of claims 1, 2, and 3 , further comprising: A keyword extraction device according to any one of claims 1 to 5 ,
Using the plurality of keywords extracted by the keyword extraction device, a content generation device that generates predetermined content,
Content generation system comprising: A search procedure in which a search unit searches for a plurality of contents including a main content based on a search keyword;
A first noise removing step in which the first noise removing unit removes, from the plurality of contents searched by the searching procedure, content in a predetermined domain that does not make sense in extracting a keyword;
A main content extracting unit sequentially selecting one of the plurality of contents from which the content of the predetermined domain has been removed by the first noise removal procedure, and extracting information indicating a link destination from the selected content; And extracting the main information by extracting the information similar to the information of the extracted link destination and comparing the information of the selected content, from the information of the selected content, and extracting the main content.
A keyword extracting step of extracting a plurality of keywords from the text of the main content extracted by the main content extracting step,
Keyword extraction method including A search procedure in which a search unit searches for a plurality of contents including a main content based on a search keyword;
A first noise removing step in which the first noise removing unit removes, from the plurality of contents searched by the searching procedure, content in a predetermined domain that does not make sense in extracting a keyword;
The second noise removing unit sequentially selects one of the plurality of contents from which the content of the predetermined domain has been removed by the first noise removing procedure , and is described by a predetermined tag from the selected content. A second noise removal procedure for removing unnecessary information that does not make sense in keyword extraction by removing the information that
The main content extracting unit sequentially selects one of the contents from which the information described by a predetermined tag has been removed from the content by the second noise removal procedure, and indicates a link destination from the selected content. A main content extraction procedure of extracting information, extracting extracted link destination information and information similar to the information of the selected content by comparing the information of the selected content, and extracting the main content by removing the information of the selected content;
Keyword extraction section, and a keyword extraction procedure of extracting a plurality of keywords from the text of the main content that is extracted by the main content extraction procedure,
Keyword extraction method including On the computer,
A search procedure for searching for multiple content, including the main content, based on the search keyword;
A first noise removal procedure for removing, from the plurality of contents searched by the search procedure, content of a predetermined domain that does not make sense in extracting a keyword;
The one content is sequentially selected from the plurality of contents from which the content of the predetermined domain has been removed by the first noise removal procedure, information indicating a link destination is extracted from the selected content, and the extracted link is extracted. A main content extracting step of extracting the main information by removing the similar information by comparing the previous information and the information of the selected content from the information of the selected content,
A keyword extraction procedure for extracting a plurality of keywords from the text of the main content extracted by the main content extraction procedure,
A program that executes On the computer,
A search procedure for searching for multiple content, including the main content, based on the search keyword;
A first noise removal procedure for removing, from the plurality of contents searched by the search procedure, content of a predetermined domain that does not make sense in extracting a keyword;
Sequentially selecting one of the plurality of contents from which the content of the predetermined domain has been removed by the first noise removal procedure , and removing information described by a predetermined tag from the selected content; A second noise removal procedure for removing unnecessary descriptions that do not make sense in keyword extraction;
Sequentially selecting one of the contents from the content from which the information described by the predetermined tag has been removed from the content by the second noise removal procedure, extracting information indicating a link destination from the selected content, A main content extraction step of extracting information that is similar to the extracted link destination information by comparing the information of the selected content with information of the selected content, and extracting the main content by removing the information from the information of the selected content;
A keyword extraction procedure for extracting a plurality of keywords from the text of the main content that is extracted by the main content extraction procedure,
A program that executes

Images