JP5490082B2 - Internet site information analysis method and apparatus - Google Patents

Description

  The present invention relates to an Internet site information analysis method and apparatus for analyzing information published on an Internet website and acquiring / providing trend information and the like.

  On the Internet, an enormous amount of information released by various people is stored in a Web site and is still increasing. Here, the Web site means an information source such as a bulletin board (BBS: Bulletin Board System), a homepage, and a Web log called a blog.

  In recent years, efforts to obtain new trend information by analyzing articles accumulated on Web sites have been actively conducted. For example, a reputation analysis technique that analyzes how positive or negative the content of an opinion is, a technique that evaluates trends in the frequency of appearance and the degree of attention (Burst degree) of a given keyword in time series, Services using various technologies are actually provided, such as a search engine optimization technology that allows a specific Web site to be displayed at the top of search results when a search engine is used.

  For example, as disclosed in Patent Document 1, text information (reputation information) is collected every predetermined period by accessing a website, and the usage status of the collected keywords is quantified, and the quantified keyword By monitoring usage, select keywords that will become trend keywords from the extracted keywords, predict trend keywords that are likely to be used in search engines in the near future, and relate to those trend keywords There is a trend prediction device that provides information to be used. This is characterized in that it has better real-time performance than the trend keyword is obtained by analyzing the actual use of the search keyword input to the search engine.

JP 2006-227965 A

  However, the trend prediction apparatus of Patent Document 1 analyzes the affirmative / negative of text information including the keyword, the frequency of occurrence of the keyword, and the like, and selects the keyword itself as the trend keyword. It was not possible to substantiate the changes in the interests of contributors that spread around the keyword.

  Further, a correlation analysis method has been proposed as a method of collecting peripheral information of a specific keyword, and a service such as an associative search for acquiring another keyword associated with the keyword has been performed. Similarly, changes in the interests of contributors spreading around the keyword could not be grasped in practice.

  In addition, there is a request to know in real time an active website where useful information can be obtained, but no method or service that can meet such a request has been proposed and has not yet been put into practical use. .

  The present invention was made in view of the above background art, and based on a lot of site information published on the Internet, an analysis that can effectively and accurately know the meaning, background, and tendency of the information. Providing devices and analysis methods, especially website activity analysis that extracts active websites where useful information can be obtained in real time, and changes in the interests of contributors spreading around certain keywords It is an object of the present invention to provide an Internet site information analysis method and apparatus capable of performing co-occurrence information analysis and grasping new trends accurately from articles accumulated on a Web site.

The present invention relates to an Internet site information analysis method for automatically accessing a Web site existing on the Internet by a computer system to collect and analyze the text information, and the computer system allows the text information and each Web site to be analyzed. An information collecting step for collecting site update date information, a word dividing step for dividing the sentence information into words, a word that is identical or similar to a predetermined keyword from the word group, and a sentence including the word a related information posts calculation step of calculating the number of information as related information posts, the proportion of the relevant information posts to total sentence information collected from each Web site, related information posted calculated for each Web site a related information post rate calculation step of the rate, and the reference date to analyze the update date information based on Internet to perform the update frequency calculation step of calculating the update frequency of each Web site, the process consisting of the relevant information Post rate and with said update frequency relative comparison output site activity analysis output step for each Web site This is a site information analysis method.

In addition to the above steps, an information collecting step for collecting sentence information of the website every time a predetermined period elapses, a word dividing step for dividing the sentence information into words, and an investigation from the word group A search object information extracting step for extracting sentence information including a word that is the same as or similar to the target keyword, and a co-occurrence keyword for extracting a co-occurrence keyword that is included in the words constituting the search target information and is a keyword different from the keyword. A keyword occurrence step, a co-occurrence keyword score calculation step for calculating a score for each co-occurrence keyword based on the frequency of occurrence of the co-occurrence keyword in the survey target information, and the co-occurrence keyword as the co-occurrence keyword score. Sorting step for rearranging in order to create a co-occurrence information list, and the co-occurrence information list obtained every predetermined period , It may be provided and co-occurrence information analysis output step of outputting represents the time series.

Also, the present invention provides an Internet site information analyzing apparatus configured to analyze a web site that is configured by a computer system, accesses the web site existing on the Internet, collects the text information, and analyzes the web site information. Information collecting means for collecting the word information, word dividing means for dividing the sentence information into words, a word that is the same as or similar to a predetermined keyword is extracted from the word group, and the number of sentence information including the word is determined. Related information posting number calculating means for calculating the number of related information postings, and a related information posting rate by calculating, for each website, the ratio of the number of related information postings to the number of text information collected from each website. Information post rate calculation means, update date information collection means for collecting the update date of each website, and analysis An update frequency calculation means for calculating the update frequency of each website based on the reference date and the update date information, and a site for outputting the related information posting rate for each website and the update frequency in a relative comparison An internet site information analysis apparatus comprising activity analysis output means.

  The activity analysis output means outputs the related information posting rate and the update frequency in a two-dimensional graph.

In addition to the above configuration, information collecting means for collecting the sentence information of the Web site every time a predetermined period elapses, word dividing means for dividing the sentence information into words, and a survey target from the word group Search target information extracting means for extracting sentence information including a word that is the same as or similar to the keyword of the keyword, and other words that are included in the words constituting the search target information and exclude the word that is the same as or similar to the keyword of the search target A co-occurrence keyword extracting means for extracting the co-occurrence keyword, a co-occurrence keyword score calculating means for calculating a score for each co-occurrence keyword based on the frequency of occurrence of the co-occurrence keyword in the survey target information, Sort means for rearranging co-occurrence keywords in the order of the co-occurrence keyword scores to create a co-occurrence information list, and the co-occurrence information list obtained every predetermined period , May be provided and co-occurrence information analysis output means for outputting represents the time series.

  According to the present invention, it is possible to analyze an enormous amount of information released and accumulated on a website and easily obtain accurate trend information.

In particular, according to the first to third aspects of the present invention, it is easy to make websites actively transmitting information through site activity analysis that calculates the posting rate and update frequency of information related to a predetermined keyword for each website. If the information is collected by focusing on Web sites that should be noted as information sources, useful trend information can be obtained efficiently.

In addition, by carrying out the co-occurrence information analysis to analyze a time series variation of another keyword co-occurring in a given keyword, also tangibly to grasping changes in the interest of contributors spread around the predetermined keyword be able to.

It is a figure which shows the structure of the whole network system which has arrange | positioned one Embodiment of the internet site information analysis apparatus of this invention. It is a flowchart which concerns on the site activity analysis which is 1st embodiment of the internet site information analysis method of this invention. It is a figure which shows the example of text information and update frequency calculation in this embodiment. It is a flowchart which concerns on the step which performs the number calculation of related articles in this embodiment. It is a figure which shows the example of the calculation result list | wrist of a related article contribution rate in this embodiment. It is a graph which shows the example of the output format of a site activity analysis result in this embodiment. It is a flowchart which concerns on the co-occurrence information analysis which is other embodiment of the internet site information analysis method. It is a figure which shows the example which extracts investigation object information from text information in other embodiment. It is a flowchart which concerns on the step which performs the score calculation of each co-occurrence keyword in other embodiment. It is a figure which shows the example of the output format of a co-occurrence information analysis result in other embodiment.

  Hereinafter, an embodiment of a network system in which the Internet site information analysis apparatus 10 of the present invention is arranged will be described with reference to FIG. In this network system, a crawler 14 that regularly collects text information in RSS (Rich Site Summary) format from a designated website and a website 12 on which many people publish information such as comments and opinions. And an article database 16 in which information collected by the crawler 14 is stored as a database, an analyzer 18 having a program for performing an analysis, which will be described later, an evaluation expression corresponding to the evaluation axis to be analyzed, and an evaluation score thereof. The set evaluation expression dictionary group database 20, the analysis result database 22 for storing the analysis results, and the personal computer owned by the user 26 are connected, and a desired analysis result is extracted from the analysis result database 22 to the user. 26 are distributed on the Internet. .

Hereinafter, the Internet site information analyzing method for performing site activity analysis according to an embodiment of the present invention will be described with reference to FIGS. 2-6. First, an outline will be described using the flow shown in FIG. In step S210, text information and update date information are collected from each Web site. At this time, in the Internet site information analysis apparatus 10, the crawler 14, the article database 16, and the site activity analysis analyzer 18b function as the information collection unit and the update date information acquisition unit. Next, in step S220, the collected sentence information is decomposed into words (parts of speech). In the Internet site information analyzing apparatus 10, the site activity analysis analyzer 18b functions as this word dividing means. Then, in step S230, the sentence information including the same or similar to the predetermined keyword in the divided word group, that is, the number of related information posts is calculated. In the Internet site information analysis apparatus 10, the site activity analysis analyzer 18b functions as the related information posting number calculation means. In step S240, the related information posting rate, which is the ratio of the number of related information posts to the number of text information collected from the website, is calculated, and the calculation result is stored. In the Internet site information analysis apparatus 10, the site activity analysis analyzer 18b and the site activity analysis result database 22b function as the related information posting rate calculation means.

  Step S250 calculates the update frequency of each Web site based on the reference date for analysis and update date information, and stores the calculation result. The Internet site information analysis apparatus 10 uses the site activity analysis analyzer 18b. The site activity analysis result database 22b serves as the update frequency calculation means.

  In step S260, in response to a request from the user 26, each Web site to which two calculated values of the related information posting rate and the update frequency are assigned is displayed on a graph and output. In the Internet site information analysis apparatus 10, the site activity analysis result database 22b and the site activity analysis display framework 24b provided in the portal server 24 function as this site analysis output means.

  Next, each step of the site activity analysis will be described in detail. As shown in FIG. 3, each Web site includes a plurality of text information and information on the last update date of each Web site. In step S210, for example, for the Web site 1, the text information a1 and a2 and the update date information “last update date: September 11” are collected. Further, in step S220, the sentence information is decomposed into words (parts of speech) such as nouns, adjectives and verbs.

  Further detailed processing of step S230 will be described with reference to FIG. In step S230, when a predetermined keyword to be investigated is given in step S231, a word group similar to the keyword is extracted using a thesaurus which is a kind of thesaurus. In step S232, sentence information including any word group that is the same as or similar to the keyword, that is, related information is extracted for each Web site. In step S233, the extracted related sentence information is counted and the number is cumulatively calculated. In step S234, which is a determination step, it is determined whether or not calculations have been performed for all websites. If NO, steps S232 to S233 are repeated for the next website. End and go to the next step 240.

  Thus, in step S230, for example, when the keyword “car” is given, common words, abbreviations, formal names, and other words such as “light four”, “hybrid car”, and “automobile” are extracted as similar words in step S231. Therefore, it is possible to analyze the contents of related information and calculate the number of posts without omission.

  In step S240, the posting rate of related information is calculated for each Web site, and the calculation result is stored in the site activity analysis result database 22b. The related information posting rate is calculated by dividing the total number of sentence information collected from each Web site by using the denominator and the number of related information for a predetermined keyword in the denominator. For example, the Web site 1 shown in FIG. 3 has two pieces of sentence information a1 and a2, and the related information of the keyword “car” is one piece of sentence information a1 including the word “hybrid car”. Therefore, the related information posting rate for the keyword “car” on the Web site 1 is calculated as 0.5. In this way, the related information posting rate is calculated for each keyword and for each Web site, and the calculation results are systematically stored in the site activity analysis result database 22b as shown in the list of FIG.

  In step S250, the update frequency of each Web site is calculated, and the calculation result is stored in the site activity analysis result database 22b. In the calculation example of FIG. 3, 1 is added to the difference between the reference date for analysis and the last update date of the Web site, and the reciprocal is defined as the update frequency. According to this definition, in the case of Web site 1, the update frequency is 1.0 because both the reference date for analysis and the last update date are September 11 (the same day). In the case of the Web site 2, if the same calculation is performed, the update frequency is 0.011. That is, the frequently updated Web site 1 has a high update frequency, and the Web site 2 that has been left unupdated for a long period shows a low update frequency.

  In step S260, the relative information posting ratio assigned for the predetermined keyword in step S240 is plotted on the horizontal axis, the update frequency assigned in step 250 is plotted on the vertical axis, and the relative positioning of each website is represented in a graph. The site activity analysis information is provided in PULL type (the user 26 extracts information as needed). For example, as shown in FIG. 6, the website located in the upper right of the graph is a group of websites that have a lot of information related to “cars” and are frequently updated. If you access a website that actively transmits information, you will find that you can get useful information about "cars". Conversely, the website located at the lower left of the graph is a group of websites with little information on “cars” and a low update frequency. Therefore, a website with low activity such as website 8 is accessed. However, it turns out that useful information about "cars" is unlikely to be obtained.

According to the Internet site information analysis method of the site activity analysis (steps S210 to S260) according to this embodiment described above, it is possible to easily know a website that is actively transmitting information, and as an information transmission source. If information is collected focusing on those Web sites to be noticed, useful trend information can be obtained efficiently.

Next, an embodiment of an Internet site information analysis method for performing co-occurrence information analysis will be described with reference to FIGS. First, an outline will be described using the flow shown in FIG. In step S310, sentence information is collected every time a predetermined period elapses from each Web site. In the Internet site information analyzing apparatus 10, the crawler 14, the article database 16, and the co-occurrence information analyzing analyzer 18c function as this information collecting means. Next, in step S320, the collected sentence information is broken down into words (parts of speech). In the Internet site information analyzing apparatus 10, the co-occurrence information analyzing analyzer 18c functions as the word dividing means. In step S330, sentence information including the same or similar to the predetermined keyword in the divided word group, that is, survey target information is extracted. At this time, in the Internet site information analysis apparatus 10, the co-occurrence information analysis analyzer 18c functions as this investigation target information extraction means. In step S340, other words excluding the same or similar words as the predetermined keyword, that is, co-occurrence keywords are extracted from the words constituting the extracted survey target information. In the Internet site information analysis apparatus 10, the co-occurrence information analysis analyzer 18c functions as this co-occurrence keyword extracting means.

  In step S350, a score is calculated for each co-occurrence keyword based on the frequency with which the extracted co-occurrence keyword appears in the survey target information. In the Internet site information analysis apparatus 10, the co-occurrence information analysis analyzer 18c functions as this co-occurrence keyword score calculation means. Next, in step S360, a co-occurrence information list in which the co-occurrence keyword groups are rearranged in the order of the co-occurrence keyword scores is created and stored. In the Internet site information analysis apparatus 10, the co-occurrence information analysis analyzer 18c and the co-occurrence information analysis result database 22c serve as the sorting means. In step S370, in response to a request from the user 26, the co-occurrence information list created every time a predetermined period elapses is output in a time series. In the Internet site information analysis apparatus 10, the co-occurrence information analysis result database 22c and the co-occurrence information analysis display framework 24c included in the portal server 24 function as the co-occurrence information analysis output means.

  Next, each step of the co-occurrence information analysis will be described in detail. As shown in FIG. 8, each Web site has a plurality of pieces of text information. In step S310, for example, if it is Web site 1, sentence information a1 and a2 are collected. Further, in step S320, the sentence information is decomposed into words (parts of speech) such as nouns, adjectives and verbs.

  In step S330, when a predetermined keyword to be investigated is given, a word group similar to the keyword is extracted using a thesaurus which is a kind of thesaurus, and either one of the same or similar word groups as the keyword is extracted. Text information including, that is, survey target information is extracted. For example, in the example of FIG. 8, when the keyword “digital camera” is given, common names such as “digital camera”, “digital still camera”, and “digital video camera”, abbreviated names, and other words are extracted as similar words. Then, text information a1, a2, and b2 including “digital camera” and similar words are extracted as investigation target information. In this way, it is possible to extract the investigation object information that should be originally investigated without omission.

  In step S340, other words excluding words that are the same as or similar to the predetermined keyword, that is, co-occurrence keywords are extracted from the words constituting the extracted survey target information. For example, in the case of sentence information a1, “Company A”, “Company B”, and “Performance” correspond to the co-occurrence keywords. Here, when co-occurrence keywords such as “(A) Co., Ltd.”, “A Co., Ltd.”, “A”, and “Company A” are separately extracted, all are synonymous with “Company A” and there is no problem. Collect them as co-occurrence keywords and go to the next step.

  Further detailed processing of step S350 will be described based on the flow of FIG. In step S351, the number of all sentence information collected is counted. For example, in the example of FIG. 8, if information collection is performed only on the Web sites 1 and 2, the total number of text information is counted as 5. In step S352, the number of text information corresponding to the survey target information is counted. For example, in the example of FIG. 8, out of the total number 5 of text information, the number of survey target information for the keyword “digital camera” is counted as 3. In step S353, the number of sentence information including the same word as the co-occurrence keyword extracted in step 340 is counted for each co-occurrence keyword. For example, in the example of FIG. 8, the number of text information including the co-occurrence keyword “Company A” is counted as 3 out of 5 text information. In step S354, the number of investigation target information including the same word as the co-occurrence keyword extracted in step S340 is counted for each co-occurrence keyword. For example, in the example of FIG. 8, the number of pieces of investigation target information including the co-occurrence keyword “Company A” is counted as two out of three pieces of investigation target information for “digital camera”.

  In step S355, the score of the co-occurrence keyword is calculated for each co-occurrence keyword. Here, the score of the co-occurrence keyword is obtained by dividing the sum of the count number of S351 by the count number of Step S354 by the sum of the count number of S352 and the count number of S353. It is preferable to define that converted into a logarithm. For example, in the example of FIG. 8, the score of the co-occurrence keyword “Company A” is 5 which is the count number in step S351, 3 which is the count number in step S352, 3 which is the count number in step 353, and the count number in step S354. Is used to obtain a rating of 0.152. In step S356, which is a determination step, it is determined whether or not all co-occurrence keywords have been calculated. If NO, steps S353 to S355 are repeated for the next co-occurrence keyword. S350 ends.

  Note that step S350 of the present embodiment has a feature that the correlation chain element is also incorporated in the score calculation. Originally, when there is a correlation between the keyword “digital camera” and the co-occurrence keyword “Company A”, It should be considered that there is a correlation between the two. However, if attention is paid to the chain of correlations, there is a problem that the amount of calculation becomes enormous, and generally, the processing up to that point has not been performed. In this embodiment, by incorporating the count number in step S351 and the count number in step S353 into the calculation formula, not only the strength of the correlation between “digital camera” and “Company A” but also “liquid crystal television” and “ Regarding the strength of the correlation of “Company A”, the difference can be relatively reflected in each score.

  In step S360, a co-occurrence information list in which the co-occurrence keyword groups are rearranged in the order of the co-occurrence keyword scores is created and stored in the co-occurrence information analysis result information database 22c. Further, in step S370, the co-occurrence information list created every time a predetermined period elapses in response to a request from the user 26 is displayed in time series, and the PULL type (the user 26 as necessary) Information is taken out). FIG. 10 shows an example in which the score calculation is performed for all websites including the websites 1 and 2. For example, in the case of the keyword “digital camera”, the co-occurrence keyword “product W” had a low score and was not ranked as of July 2007, but it rose to the second place in August 2007. Therefore, it can be seen that in the field of “digital camera”, “product W” has become the center of the contributor's topic. Further, the co-occurrence keyword “Company B” was ranked second in July 2007, but was ranked down to fifth in August 2007. Therefore, it can be seen that the poster has stopped paying attention to “Company B”.

According to the co-occurrence information analysis described above (steps S310 to S370) , a poster who spreads around a predetermined keyword by analyzing changes in another keyword co-occurring with the predetermined keyword in time series. Can change the interests of the actual situation.

The present invention is not limited to the above embodiment, the Internet site information analyzer 10, in addition to the analyzer or analysis process having a working means or steps according to the above sites activity analysis, the above occurrence information You may make it the structure which provides the working means or step which concerns on an analysis .

  The display frame of the analysis result output in step S260 may be any graph as long as it can visually recognize the relative relationship between a plurality of characteristic values of each data. It is not limited. The graph scale should be displayed in logarithmic form, or a legend should be attached to display the results of multiple analysis.

  Further, the calculation formula of the related posting rate defined in step S240, the calculation formula of the update frequency defined in step S250, and the calculation formula of the co-occurrence keyword score defined in step S350 are the items to be investigated and analyzed for each field As long as it is defined in view of individual circumstances, the calculation formula of the above embodiment is not limited. For example, in the case of the co-occurrence keyword score calculation formula defined in step S350, the content to be analyzed in detail, such as changing the base value of the logarithm or substituting a square value for the number of specific information, is the characteristic value. Another calculation formula may be defined so as to appear prominently.

  A series of processing operations as in the above embodiment is constructed as a program, installed in a server computer used as the Internet site information analysis apparatus 10 and executed by a control means such as a CPU, and the program is transmitted via a network. May be distributed. Further, the constructed program is stored in a portable storage medium such as a hard disk device, a flexible disk, or a CD-ROM connected to various computers used as the Internet site information analysis apparatus 10, and is installed and executed on the computer. You may do it.

DESCRIPTION OF SYMBOLS 10 Internet site information analyzer 12 Website 14 Crawler 16 Article database 18 Analyzer 20 Evaluation expression dictionary group database 22 Analysis result database 24 Portal server 26 User

Claims (3)

In an Internet site information analysis method for automatically accessing a website existing on the Internet by a computer system to collect and analyze the text information,
By the computer system,
An information collecting step of collecting the sentence information and update date information of each website;
A word dividing step for dividing the sentence information into words;
Extracting a word that is the same as or similar to a predetermined keyword from the word group, and calculating a related information posting number calculating step for calculating the number of text information including the word as a related information posting number;
A related information posting rate calculating step of calculating a ratio of the number of related information postings to the number of text information collected from each Web site and calculating the related information posting rate for each Web site;
An update frequency calculating step for calculating an update frequency of each Web site based on a reference date for analysis and the update date information;
A site activity analysis output step of outputting the related information posting rate for each website and the update frequency relative to each other;
Internet site information analysis method and performing a process consisting of. In an Internet site information analyzing apparatus configured by a computer system, accessing a Web site existing on the Internet by the computer system , collecting the sentence information, and performing analysis,
Information collecting means for collecting text information of the website;
Word dividing means for dividing the sentence information into words;
A related information posting number calculating means for extracting a word that is the same as or similar to a predetermined keyword from the word group and calculating the number of pieces of text information including the word as the related information posting number;
A related information posting rate calculating means for calculating a ratio of the number of related information postings to the number of text information collected from each Web site for each Web site to be a related information posting rate;
Update date information collecting means for collecting the update date of each website;
Update frequency calculation means for calculating the update frequency of each website based on the reference date for analysis and the update date information;
Site activity analysis output means for relatively comparing and outputting the related information posting rate for each Web site and the update frequency;
Internet site information analyzer characterized by comprising 3. The Internet site information analysis apparatus according to claim 2, wherein the site activity analysis output means outputs the related information posting rate and the update frequency in a two-dimensional graph.

Images