JP4266023B2 - Document arrangement method based on link relationship - Google Patents

Description

  The present invention relates to the organization of documents existing on a network, and is particularly suitable for a case where there are not only character information but also a large number of documents in various forms such as images and sounds, and those documents change drastically. Related to simple document organization technology.

  For example, WWW (World Wide Web, hereinafter referred to as the web) is a rapidly growing Internet resource. There are a large number of documents (also called web pages) on the web, as in 2000, a survey of over 2 billion pages. In addition, the Web has not only a large amount of existing documents but also a feature that the change of documents is very fast.

  According to a survey by the Web Archive Organization, the information on the Web increases by 10% every month, and the life of one document (from the time the document is created until it is no longer maintained) is about 75 days. is there.

  Currently, several search services for searching for information existing on the web are provided. In this search service, information indicating the network location of a document obtained as a result of the search, for example, a URI (Uniform Resource Identifier) or URL (Uniform Resource Locator) and a sentence explaining the content of the web page are searched. Provided to the person.

  Also, in recent years, reflecting the broadband era, document content has shifted from text to video / audio, etc., or simply from browsing content to document providing services.

  However, since the conventional search service provides a search service based on the status of the web at a certain point in time, what kind of situation the document is in chronological order, for example, popularity is beginning to appear, There was a problem that it was unclear whether it was a classic or a less popular one. For example, there was no way to look up “recently popular web pages” from the web.

  In the case of the Web, authors rarely delete outdated documents or update the contents of documents frequently. Therefore, calculating the degree of popularity of a document based on the number of other documents that are simply linked to the document (number of linked links), that is, calculating the degree of popularity, is that the degree of popularity is hardly reduced. There was also a problem of not.

  Reflecting on the broadband era, documents centered on text and those containing non-text and services such as images. However, there was no way to organize documents that correspond to these changes.

  In view of the above problems, an object is to solve the problem that the popularity of documents based on the number of simple linked links increases but does not decrease. Another object of the present invention is to make it possible to obtain information indicating how the popularity of a document is in time series. Another object is to make it possible to organize documents in response to the migration of document contents and the like.

  According to one aspect of the present invention, in a popularity degree calculation method for calculating a popularity degree that is a degree of popularity of a document on a network, a link relationship is extracted from the document and updated or collected within a first period. The extracted document is extracted as an object for calculating the degree of popularity, and the degree of popularity of each extracted document is calculated.

  By calculating the popularity for the documents collected within the first period or the documents updated within the first period, the old document is omitted from the target for calculating the popularity, and thus the popularity of the document. Solves the problem of increasing but not decreasing. In order to calculate a significant degree of popularity, it is desirable that the first period is a certain period of time, for example, about 150 days.

  Here, the popularity degree may be calculated based on the link relation and document position information indicating the position of the document on the network. Thereby, since it is not necessary to see the contents of the document, the popularity can be calculated quickly.

  In the above method, the popularity change degree indicating the direction and degree of change of the popularity degree of the document may be calculated based on the popularity degree calculated within the second period. As a result, it is possible to obtain information indicating the status of the popularity of the document in time series.

Here, it is desirable that the second period is not a very long period, for example, several weeks in order to see the change in popularity.
In the above method, a regression equation with respect to time of the popularity calculated within the second period may be calculated, and the popularity change may be calculated based on the regression equation. In this case, the popularity change degree may be determined based on a regression coefficient of the regression equation, or a trend of transition of the popularity degree with respect to time may be determined based on an intercept of the regression equation. .

Further, when calculating the regression equation, a ranking based on the popularity of the extracted document may be used instead of the popularity.
According to another aspect of the present invention, in a document relationship determination method for determining a relationship between documents on a network, a link relationship is extracted from a first document, and the first relationship is determined based on the link relationship. It is determined whether or not the second document linked from the document is a non-text document related to the contents of the first document. This makes it possible to organize documents according to the type of non-text media such as images, which has increased in recent years.

  In the above method, a character string in the vicinity of a portion linked from the first document to the second document is extracted from the first document, and based on the character string, the second document is converted into the first document. The method may further include determining whether the document is a related non-text document related to the content of one document. For example, when the character string is a character string indicating that the second document is in a non-text format, such as MPEG, video, streaming, etc., the second document is a non-text document related to the contents of the first document. It can be estimated that there is.

  The method may include determining that the second document is not a non-text document related to the contents of the first document when the extension is not a specific extension. Since the extension indicates the document format of the second document, it can be determined whether or not the extension is a non-text document.

  In the above method, the second document is a non-text related to the content of the first document based on whether the second document has been used more than a predetermined number of times in the first document. It may be determined whether or not it is a document. For example, a bullet or the like is an image, but these material-based images for document creation are often used repeatedly in one document. It can be presumed that the second document having a large number is not related to the contents of the first document.

  In the above method, when there is a third document having a file name similar to the file name of the second document in the first document, the file name of the second document is the third document. When the document name is younger than the file name of the document, it may further include not registering the second document in the database as a non-text document related to the content of the first document.

  For example, if the first document is a photo book, it contains many images. If all these images are registered as non-text documents related to the contents of the first document, there is a possibility that it will be complicated. However, in this case, since the file names of the image files are often similar to each other, only the document with the youngest file name in the dictionary order among the file names of the plurality of documents is only a non-text related to the contents of the first document. By registering it as a document, it is possible to eliminate such complications.

  In the above method, when there is a third document linked from the second document, the document position information indicating the position of the first document on the network and the document position information of the second document are included. The method may further include determining whether or not the second document is a non-text document related to the content of the first document. Whether the second document is a non-text document related to the contents of the first document based on the document position information of the first document and the document position information of the third document. It may be possible to further include determining.

  For example, the first document may include a non-text document that is not related to the content of the document, such as a banner advertisement, as the second document. In such a case, the document location information of the second document and the document location information of the third document that is the link destination of the second document are the same server address as the document location information of the first document. Or, since it often does not have a domain, a non-text document that is not related to the content of the first document such as an advertisement banner can be excluded based on the document position information of each document.

  According to still another aspect of the present invention, in a service type determination method for determining a type of service provided by a document on a network, a tag specifying user input is extracted from the document, and the user input And determining a type of service provided by the document based on a tag designating. This also makes it possible to organize the documents according to the types of services provided by the documents in response to changes in the contents of the documents in recent years. As a tag for designating user input, for example, when the language in which a document is described is HTML, a form tag is used.

  The method may further include determining that the document does not provide a service if the document does not include a tag designating user input. This is because if there is no user input field in the document, it is unlikely that the document provides a service.

  Further, the method may further include determining a type of service provided by the document based on display of a button included in the document. Furthermore, it may further include determining the type of service provided by the document based on the input field in addition to the button display. This is because, in many cases, the format of an input field such as a button is determined by the service provided by the document.

  More specifically, for example, when the document includes a button having a display indicating that a product is to be purchased, it further includes determining the type of service provided by the document as a store. Also good. This is because a document that provides a service for selling a product often includes such a button in order to receive an order for the product.

  For example, when the document includes a user input area and a button having a display indicating a search, it may further include determining the type of service provided by the document as a search.

  Further, the same operation and effect as the above-described method can be obtained also by an apparatus including means for realizing the procedure performed in the method according to each aspect of the present invention. Further, by causing a computer to execute a program that causes a computer to perform the same control as the procedure performed in each method of the present invention described above, it is possible to obtain the same operations and effects as those described above. Also, it is possible to obtain the same operation and effect as the method described above by causing a computer to read and execute the program from a computer-readable recording medium in which the above-described program is recorded.

  As described above in detail, the present invention calculates the popularity indicating the degree of popularity for documents collected or updated within the first period, and further calculates within the second period. A popularity change degree indicating a degree of change in the popularity degree is calculated based on the obtained popularity degree. As a result, it is possible to obtain information indicating the status of the document in time series while solving the problem that the popularity of the document increases but does not decrease.

  Further, according to the present invention, it is possible to organize various documents such as a document that provides non-text content and a service based on a link relationship and tags between documents.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 shows the principle of the present invention. The document organizing apparatus according to the present invention calculates the popularity indicating the degree of popularity of the document based on the link relationship, and further shows how the popularity changes in time series. Calculate the popularity change shown. Then, each document is organized based on the calculated popularity and popularity change.

  As shown in FIG. 1, the document organizing apparatus 10 includes a popularity degree calculating unit 11 and a popularity degree transition calculating unit 12. The popularity calculation means 11 calculates the popularity indicating the degree of popularity of each document based on the link relationship between documents on the network collected in the first period. Here, the popularity degree calculation means 11 calculates the popularity degree for the documents collected within the first period or the documents updated within the first period. This solves the problem that the popularity of the document increases but does not decrease.

  The popularity degree transition calculating unit 12 calculates a popularity change degree indicating the direction and degree of change in the popularity degree based on the popularity degree calculated by the popularity degree calculating unit 11 within the second period. The popularity level transition calculation unit 12 may use the popularity ranking obtained by ranking each document based on the popularity instead of the popularity when calculating the popularity change. This makes it possible to analyze how the popularity of documents on the network changes over time.

  In recent years, reflecting the era of broadband Internet, document contents (contents) can be changed from text to non-text such as images, videos, and voices. In addition, services such as search and registration can be provided from documents that simply read information. The emphasis is shifting to documents. However, for example, in a conventional search service, as a search result, simply providing the searcher with information indicating the position of the document on the network and a sentence explaining the content of the document, what kind of non-text the document is The searcher does not know whether the content is included or what service is being performed on the document unless the document is accessed.

  Also, when organizing such non-text content, if the non-text content included in the document is simply determined based on the file extension, such as banners and bullets (dots) included in the document. There is also a problem in that non-text content that is not related to the content of a document is also organized as content related to the document.

  Therefore, as shown in FIG. 1, the document organizing apparatus 10 according to the present invention further includes a related non-text content determination unit 13 and a service type determination unit 14. The related non-text content determination means 13 determines non-text content related to the content of the document among the non-text content included in each document based on the link relationship between the documents, and determines that the related non-text content is related to the content of the document. Organize the non-text content that has been made corresponding to the document.

  The service type determination unit 14 provides a service for a document based on a tag included in each document, for example, a tag for designating user input used when creating an input field, for example, a form tag in the case of HTML. If the document provides a service, the service type is determined, and the determined service type is organized in association with the document. Thus, for example, in a search service, in addition to information indicating the position of the document on the network and a sentence explaining the content of the document, the non-text content related to the content of the document and the document are provided as search results. It is possible to provide information about the service being provided as information about the document.

  Hereinafter, embodiments of the present invention will be described. In addition, although the case where the above-mentioned document organization apparatus is applied to the document search apparatus which searches a document on a network is demonstrated, it is not the meaning which limits the application range of this invention.

  FIG. 2 shows the configuration of the document search apparatus according to the embodiment of the present invention. The document search apparatus 100 collects documents from the network and organizes the collected documents. As the network, a LAN (Local Area Network) such as an intranet or a dedicated line, and a WAN (Wide Area Network) such as a public line or the Internet can be considered. The document search apparatus 100 searches for a document in accordance with an instruction from a user of a terminal (not shown) connected directly or via a network (not shown), and provides a search result to the user.

  When the document search apparatus 100 is a server that provides services and data to a terminal via a network, the user terminal includes a browser 108, and the user transmits information transmitted from the document search apparatus 100 using the browser 108. It is good also as browsing.

  As shown in FIG. 2, the document search apparatus 100 includes a collection unit 101, a popularity calculation unit 102, a popularity transition calculation unit 103, a related non-text content determination unit 104, a service type determination unit 105, a page classification unit 106, a search A service unit 107, a document table 111, a link relation table 112, a popularity degree table 113, a popularity degree change table 114, a non-text content table 115, and a service type table 116 are provided. The collection unit 101, the popularity calculation unit 102, the popularity transition calculation unit 103, the related non-text content determination unit 104, the service type determination unit 105, the page classification unit 106, and the search service unit 107 include, for example, software described by a program Corresponding to the component, it is stored in a specific program code segment of the memory of the computer that implements the document search apparatus 100.

  Here, as a language for describing a document existing on the network, that is, a web page, for example, HTML (HyperText Markup Language), XHTML (eXtensible HyperText Markup Language), XML (eXtensible HyperText Markup Language), SGML (Standard Generalized Markup Language) A language that can embed a link relationship in a document is considered. In the present invention, in addition to a text document described in the above language, images, moving images, sounds, and the like are also handled as documents. Hereinafter, the description may be made assuming that the language for describing the text document is HTML, but this is not intended to limit the present invention.

  The collection unit 101 collects documents published on the network, and adds a document ID (IDentification information) that identifies the document to the collected documents. Further, the collection unit 101 analyzes the link relationship of the collected documents. The collection unit 101 stores document position information indicating the position of the collected document on the network in the document table 111, and stores information related to the link relationship between the collected documents in the link relationship table 112.

  Here, for example, URI (Uniform Resource Identifier) can be considered as the document position information. The URI is a comprehensive concept, and currently, a URL (Uniform Resource Locator) that specifically uses a part of the URI function is widely used. Hereinafter, the document position information may be described assuming that it is a URL, but this is not intended to limit the present invention.

  The popularity calculation unit 102 calculates the popularity indicating the degree of popularity based on the link relationship of the documents collected by the collection unit 101 periodically (or irregularly), and calculates the calculation result. Stored in the popularity degree table 113. When calculating the popularity, the popularity calculation unit 102 selects, among the documents collected by the collection unit 101, a document collected within the first period or a document updated within the first period. The document for which the degree is to be calculated. Here, the first period needs to be a somewhat long period because a meaningful result as popularity cannot be obtained in a very short period. For example, as the first period, 150 days before the day of calculating the popularity degree can be considered.

  As a result, it is possible to omit a document that has been created and has not been updated and is left as an object for calculating popularity. In other words, simply calculating the popularity of a document in time series can solve the problem that the popularity is increasing monotonously.

  Based on the popularity calculated by the popularity calculation unit 102 within the second period, the popularity transition calculation unit 103 calculates a popularity change degree indicating the direction and degree of change in popularity for each document, and the calculation result Is stored in the popularity change table 114. Here, if the second period is too long, it is not possible to grasp the short-term popularity fluctuation, and therefore it is necessary to have a relatively short period, for example, several weeks. For example, as the second period, 14 days before the day for calculating the popularity change degree can be considered.

  More specifically, for example, the popularity transition calculating unit 103 acquires, for each document, the popularity calculated within the second period from the popularity table 113 and linearizes the acquired popularity with respect to time. The regression equation is calculated, and the regression coefficient of the linear regression equation is obtained as the popularity change degree. Also, the popularity level transition calculation unit 103 may use the popularity ranking obtained by ranking each document based on the popularity instead of the popularity when calculating the popularity change. This makes it possible to analyze how the popularity of documents on the network changes over time.

  The related non-text content determination unit 104 determines the type of each document based on the extension of the file name included in the document position information of each document and the character string before and after the embedded portion of the link in the document. To do. Further, the related non-text content determination unit 104 determines whether or not the non-text content included in each document is related to the contents of each document based on the link relationship between the documents. Then, the related non-text content determination unit 104 stores the non-text content determined to be related to the contents of each document in the non-text content table 115 in association with the document. As a result, it is possible to remove non-text content not related to the contents of the document from the non-text contents included in each document and organize the non-text contents related to the contents of the document in correspondence with the documents. Become.

  The service type determination unit 105 determines the type of service provided by the document based on the information describing the input field included in each text document, and associates the determined service type with the document in the service type table 116. To store. This makes it possible to organize the types of services provided by each document in correspondence with the document.

  The page classification unit 106 classifies each document based on a related field or the like. Since various classification techniques already exist for the document classification method, a detailed description thereof will be omitted in this embodiment.

  The search service unit 107 searches a document on the network according to a user instruction and provides a search result to the user. At that time, the search service unit 107 acquires information about the document obtained as a result of the search from the popularity degree table 113 and the popularity degree change table 114, and uses the information and the document position information to explain the contents of the retrieved document. In addition, the degree of popularity and popularity change are provided to the user. As a result, the user can use the information provided on the search result output screen to determine the popularity of the retrieved document, whether it is beginning to appear, whether the popularity is beginning to appear, or the popularity is decreasing. I can know.

  Further, the search service unit 107 acquires information on the document obtained as a result of the search from the non-text content table 115 and the service type table 116, and information on the non-text content related to the content of the searched document and the searched information. Information regarding the service type provided in the document may also be provided to the user. As a result, the user accesses (views) what non-text content the document obtained as a result of the search includes or what service is provided for the document. Even without this, it is possible to know from the information provided on the search result output screen.

  When the user requests to provide information related to the popularity of one or more documents, the search service unit 107 acquires the information related to the documents from the popularity table 113, the popularity change table 114, and the like. The information may be provided in time series. Thereby, the user can analyze the transition of the popularity of a certain document.

  Hereinafter, the data structure of each table will be described with reference to FIGS. First, the data structure of the document table 111 will be described with reference to FIG. As shown in FIG. 3, the document table 111 stores document position information and a corresponding document ID for each document. As a result, the document position information of each document is converted into a document ID, and in the subsequent processing, it is possible to manage information related to the link relationship and the like of each document using the document ID.

  Next, the data structure of the link relationship table 112 will be described with reference to FIG. The link relationship table 112 stores link relationship information for each document. As shown in FIG. 4, the link relationship information includes the date and time (or date) when the document was collected, the updated date and time (or date), the document ID of the document that is the link source, and the link destination. The document ID of the document is included as an item. In the following description, the document ID of a document that is a link source is referred to as a link source ID, and the document ID of a document that is a link destination is referred to as a link destination ID. If it is difficult to obtain the update date / time of each document, the collection date / time may be handled instead of the update date / time.

  Next, the data structure of the popularity degree table 113 will be described with reference to FIG. The popularity degree table 113 stores popularity degree information for each document. As shown in FIG. 5, the popularity information is the result of sorting documents based on the date and time (or date) when the popularity was calculated, the document ID of the document, the calculated popularity, and the popularity. The degree rank is included as an item.

  Next, the data structure of the popularity change table 114 will be described with reference to FIG. The popularity change table 114 stores popularity change information for each document. The popularity change information includes the document ID of the document, the regression coefficient (slope) and intercept obtained as a result of calculating the linear regression equation for the popularity, and the regression obtained as a result of calculating the linear regression equation for the popularity ranking. The coefficient (slope) and intercept are included as items.

  Next, the data structure of the non-text content table 115 will be described with reference to FIG. The non-text content table 115 includes, for a document having a link destination, the document ID of the document and the document ID of the non-text content linked to the document (hereinafter referred to as related non-text content ID). And the file type of the non-text content.

  Finally, the data structure of the service type table 116 will be described with reference to FIG. As shown in FIG. 8, the service type table 116 stores a sentence document ID and a service type provided by each document for each document.

  Hereinafter, processing performed by each unit configuring the document search apparatus 100 will be described with reference to FIGS. 9 to 15. Note that the description of the processing performed by the page classification unit 106 is omitted as described above.

  First, the collection unit 101 continuously collects documents from the network, analyzes the link relationship between the collected documents, and stores the collection and analysis results in the document table 111 and the link relationship table 112. The popularity calculation unit 102 calculates the popularity of documents collected or updated regularly, for example, every day, within a certain period before the calculation date. Note that each day is merely an example, and is not intended to limit the present invention. Hereinafter, the procedure of the process of calculating the popularity will be described with reference to FIG.

  As shown in FIG. 9, first, the popularity calculation unit 102 is activated every day at a fixed time. When the popularity calculation date for calculating the popularity is d1, the popularity calculation unit 102 determines a date d2 N days before d1, for example, 150 days before as a calculation target start date (step S11). Note that 150 days is merely an example. N may be a period long enough to obtain a meaningful result as popularity.

  Subsequently, the popularity calculation unit 102 extracts link relationship information whose collection date or update date is between the calculation target start date d2 and the calculation date d1 from the link relationship table 112 (step S12). By restricting the collection date or update date of a document for which popularity is to be calculated within a certain period of time, documents that have been created and are left unupdated are excluded from the target for calculating popularity. Is possible.

  When there is link relation information having the same link source ID among the extracted link relation information, the popularity calculation unit 102 leaves the link relation information having the latest collection date or update date, and other same link source IDs. The link relation information having is deleted (step S13). Thereby, it is possible to prevent the degree of popularity from being calculated for the same document.

  The popularity calculation unit 102 calculates the popularity of each document based on the extracted link relation information (step S14). More specifically, the popularity calculation unit 102 refers to the link relationship and the degree of similarity between the character strings indicating the document position information of the link source document and the link destination document without referring to the content of the document. The popularity of each document is calculated based on a certain similarity. Hereinafter, a procedure for calculating the degree of popularity will be described.

The basic idea when calculating popularity is as follows.
1. Documents that are frequently linked from documents that have dissimilar document location information are popular.

  For example, generally, a plurality of documents provided in the same site are linked to other documents in the site, but the document position information of these documents is similar to each other. Therefore, it can be estimated that the popularity of documents linked from documents whose character strings indicating document position information are similar to each other is low.

  2. A document linked from many documents is a document having a higher popularity, and a document linked from a document having a higher degree of popularity that has dissimilar document position information is higher.

  For example, famous directory services and public offices are linked from many documents, but documents linked from such documents are linked from sites opened by individuals and entry pages of their contents. This is because it is considered to be more popular than existing documents. Further, many documents and documents provided in a service (site) having a mirror site are often linked in the site. Since the document location information of documents in one site is almost similar, for example, with the same domain, the concept that “the popularity of documents linked from documents with similar document location information is high” was introduced. By doing so, it becomes possible to eliminate the fact that the popularity of a document linked many times within the site becomes high.

  3. Whether or not the document position information is similar indicates that the server address, path, and file name are all the smallest, and the document position information indicates that the document in the mirror site or the same server has a high similarity. Define from a string.

  By introducing the above three concepts, all link relationships are not handled equally, but weights are given to the link relationships. More specifically, the weight is given to the link relationship as the reciprocal of the similarity between the document position information of the link source document and the link destination document.

Hereinafter, the procedure for calculating the degree of popularity will be described in more detail.
DOC = {p1, p2,... PN}, a set of documents whose popularity is to be calculated
Wp is the popularity of document p,
Ref (p) is a set of documents linked to document p.
Refed (p), which is the document set of the link source of document p
The similarity between the document position information of document p and document q is expressed as sim (p, q),
If the difference is diff (p, q) = 1 / sim (p, q),
Assuming that a link is established from document p to document q, the weight lw (p
, q) is defined by the following equation (1).

  As can be seen from the equation (1), lw (p, q) indicates that the lower the similarity sim (p, q) between the URLs of the document p and the document q, and the number of links from the document p to the document p. The smaller the number, the larger.

  The popularity Wq of the document q is Cq as a constant (a lower limit of popularity, and a different value may be given depending on the document) for each document pεDOC.

It is defined as the solution of the simultaneous linear equations shown in equation (2). The popularity calculation unit 102 calculates the popularity of each document by solving the simultaneous linear equations. In addition, about the solution method of such simultaneous linear equations, since many existing algorithms exist, description is abbreviate | omitted. A method for calculating the similarity sim (p, q) of the document position information in the equation (1) will be described later. It can be read from the formulas (1) and (2) that the above-mentioned concept is realized. That is, if the similarity of the document position information is low from the equation (1), the link relation weight lw is large. Then, the popularity Wq of the document linked from the document having a large link relation weight lw from the equation (2) becomes high. That is, the popularity of a document that is frequently linked from a document having document position information with a low similarity is high. Moreover, the popularity is higher as the document is linked from many documents from the equation (2). Furthermore, it can be seen from the equation (2) that the popularity of documents linked from documents with high popularity W increases.

  Next, the similarity sim (p, q) between the document position information of the documents p and q in the equations (1) and (2) will be described. Hereinafter, description will be made assuming that the document position information is a URL, but this is not intended to limit the present invention.

In general, the URL of a document is composed of three types of information: a server address, a path, and a file name. For example, the URL of the WWW document,
http://www.flab.fujitsu.co.jp/hypertext/news/1999/product1.html is the server address (www.flab.fujitsu.co.jp), path (hypertext / news / 1999), file name It consists of three types of information (product1.html).

  Further, the server address is further hierarchized by “.”, And gradually increases as it goes backward. For example, if the server address is www.flab.fujitsu.co.jp, it represents the hierarchy of Japan (jp), company (co), Fujitsu (fujitsu), laboratory (flab), and machine (www) from the back. ing.

The link relationship weight calculation method according to this embodiment is based on the following concept.
1. In many cases, since similar documents are put in the same directory, the contents of document position information with the same path on the same server are often similar.
2. The document position information of the document in the mirror site provided for distributing access and the document in the original site has high similarity. For example, only the server address part is different, and the remaining paths and file names are often the same.
3. Document position information that is different from the server address, path, and file name has a low similarity.

  In the present embodiment, the similarity between the document position information of two given documents p and split q is defined by a combination of the above-described three types of server address, path, and file name. As the similarity sim (p, q), for example, the domain similarity sim-domain (p, q) and the fusion similarity sim-merge (p, q) described below can be considered.

  The domain similarity sim-domain (p, q) is calculated based on domain similarity. The domain is the latter half of the server address and represents a company or organization. For US servers whose server address ends with .com, .edu, .org, etc., the second from the end of the server address, and for servers in other countries where the server address ends with .jp, .fr, etc., after the server address The third one corresponds to the domain. For example, the domain of www.fujitsu.com is fujitsu.com, and the domain of www.flab.fujitsu.co.jp is fujitsu.co.jp.

The domain similarity between the document p and the document q is defined by the following equation (3).
sim-domain (p, q) = 1 / α (when p and q are the same domain)
= 1 (for domains with different p and q)
... (3)
Here, α is a constant and takes a real value larger than 0 and smaller than 1. By introducing the concept of sim-domain (p, q), documents having different domains can be easily searched. In other words, documents having the same domain are not easily searched.

As sim (p, q), a fusion similarity sim-merge (p, q) obtained by fusing the above three types of information is defined as follows.
sim-merge (p, q) = (similarity of server addresses) + (similarity of paths) + (similarity of file names)
Hereinafter, the calculation method of each term on the right side will be described.

  The similarity of server addresses is 1 + n when the address hierarchy is viewed from the back and if there is a match to n levels. For example, since www.fujitsu.co.jp and www.flab.fujitsu.co.jp match up to three levels, the fusion similarity is 4. Since www.fujitsu.co.jp and www.fujitsu.com do not match each other at one level (match 0 level), the fusion similarity is 1.

  The path similarity is compared for each element delimited by “/” of the path from the top, and the level up to the matching level is regarded as the similarity. For example, /doc/patent/index.html and /doc/patent/1999/2/file.html match up to two levels, so the similarity is 3.

The similarity of the file name is set to 1 when the file names match.
Also by this sim-merge (p, q), the popularity of a document linked from a document with a similar URL is lower than that of a document linked from a document with a similar URL. Therefore, by introducing the concept of sim (p, q) or diff (p, q) in lw (p, q), the amount of servers (sites) and individuals with a large amount of documents is simply large. Can solve the problem of high popularity.

  After calculating the popularity, the popularity calculation unit 102 acquires the popularity ranking by sorting the documents in descending order of popularity (step S15). Changes in the popularity ranking over time may increase or decrease. Therefore, the problem that the change in the time series of popularity by the conventional calculation method was increasing can be solved by paying attention to the time series change of the popularity ranking instead of the popularity. Become. Finally, the popularity calculation unit 102 stores the calculated popularity and popularity ranking in the popularity table 113 together with the document ID and popularity calculation date of each document (step S16), and ends the process.

  For example, when providing the user with a result of searching for a document, each document may be sorted or ranked based on the popularity calculated as described above. Further, when providing information about a document, the degree of popularity of the document may be provided to the user (described later).

  Hereinafter, the features of the present invention in calculating the degree of popularity will be described with reference to FIG. FIG. 10A is a diagram showing a temporal change in popularity calculated by a conventional calculation method. In FIG. 10A, the horizontal axis indicates time, and the vertical axis indicates popularity. Since the author rarely deletes or updates a document once created on the web, it is based on the number of other documents (linked number) that are simply linked to the document as in the prior art. When the degree of popularity of the document is calculated, the degree of popularity does not decrease and increases as shown in FIG.

  FIG.10 (b) is a figure which shows the time change of the popularity calculated by the calculation method concerning this invention. Also in FIG. 10B, the horizontal axis indicates time, and the vertical axis indicates popularity. According to the present invention, in order to calculate the popularity for a document collected or updated within a certain period between the calculation target start date and the popularity calculation date, the document is created once and then for a long period of time. Documents that are left unattended are not subject to popularity ratings. Therefore, for example, the popularity of a document whose link source is a document that has been left unattended for a long period of time is calculated to be lower than in the past. This solves the problem of increasing popularity in the past.

  Also, for example, since the top page of a site that has just been published on the web is linked to many documents from the site, etc., the popularity of the top page is initially calculated to be high. If left unupdated, the popularity of the top page will drop, and the popularity will be transient.

  The popularity of the document shown in FIG. 10B has increased rapidly at first, but after a certain amount of time has passed, the popularity has started to decrease, and continues to decrease thereafter. From this, it can be seen that the epidemic of this document ended only temporarily.

  FIG.10 (c) is a figure which shows the time change of the popularity ranking based on the popularity calculated by the calculation method concerning this invention. In FIG. 10C, the horizontal axis represents time, and the vertical axis represents popularity ranking. The popularity ranking is information indicating the relative popularity of the document as viewed from the whole document for which popularity is calculated, and therefore increases even if the popularity is calculated by the conventional calculation method due to its nature. It's hard to think about continuing. Therefore, it is possible to solve the problem that the popularity has been increasing conventionally by determining the popularity of the document based on the temporal change of the popularity ranking.

  Further, according to the temporal change in the popularity ranking based on the popularity calculated by the calculation method according to the present invention, the document shows the transition of the average ranking as viewed from the whole document for which the popularity is calculated. In the case of showing, as shown in the graph of FIG. 10B, the popularity ranking changes substantially constant even if time passes. Also, when the popularity of the document is increasing, the popularity ranking also increases with the increase in popularity. On the other hand, when the popularity of the document is decreasing, the popularity ranking is lowered as the popularity decreases. In general, the popularity of documents starts from an initial increase period, then goes through a stable period and then decreases. In this case, as shown in FIG. 10C, the popularity ranking rises in the increase period, becomes almost constant in the stable period, and falls in the decrease period. Becomes Yamagata.

  Next, a procedure of processing for calculating the degree of popularity change will be described with reference to FIG. When the popularity calculating unit 102 calculates the popularity, the popularity transition calculating unit 103 acquires the popularity calculated within a certain period from the popularity table 113, and calculates the popularity change that is a temporal change in the popularity. To do.

  First, the popularity level transition calculation unit 103 determines the date d3 from the popularity level calculation date d1 to M days, for example, 14 days before as the calculation target start date (step S21). Note that the 14th is merely an example. If M is set too long, it will not be possible to grasp short-term fluctuations in popularity.

  Subsequently, the popularity level transition calculation unit 103 acquires the popularity level or the popularity level ranking calculated from the calculation target start date d3 to the popularity level calculation date d1 for each document from the popularity level table 113 (step S22). The popularity transition calculating unit 103 calculates a linear regression equation with respect to time of popularity or popularity ranking for each document, and obtains a regression coefficient and an intercept b of the linear regression equation (step S23). When the linear regression equation is calculated based on the popularity, the regression coefficient a corresponds to the popularity change degree. When the linear regression equation is calculated based on the popularity ranking, the value obtained by dividing the regression coefficient a by the intercept b, a / B corresponds to the popularity change.

Hereinafter, the calculation method of the linear regression equation will be described in detail. Assuming that the values of popularity or popularity ranks on the dates (d3, d3 + 1,..., D1) from the dates d3 to d1 are (w0, w1,... WM-1), a linear regression equation r = A (d1-d3) + b
Is calculated by the method of least squares. here,
a is a regression coefficient, and is calculated by the following equation.

a = (M * Iw-I * W) / (M * I2-I2)
Moreover, b is an intercept and is calculated by the following formula.
b = (I * Iw-W * I2) / (I2-M * I2)
Here, Iw, W, I, and I2 are calculated by the following equations, respectively.

  Finally, the popularity transition calculation unit 103 stores the calculated regression coefficient a and intercept b of each document together with the document ID in the popularity change table 114 (step S24), and ends the process.

  When the linear regression equation is calculated based on the popularity, the popularity is increasing if the regression coefficient a of the linear regression equation is positive. The larger the absolute value, the faster the popularity increases. Show. Further, when the intercept b is higher than a relatively high value, it indicates that the popularity is stable at a high level, and when the intercept b is less than a relatively low value, the popularity is stable at a low level. It shows that.

  On the other hand, when the linear regression equation is calculated based on the popularity ranking, the popularity is increasing if the regression coefficient a is negative, and the higher the absolute value, the faster the popularity increases. . In addition, when the intercept b is below a relatively low value, it indicates that the popularity is stable at a high level, and when the intercept b is above a relatively high value, the popularity is stable at a low level. It shows that.

  The degree of popularity change of each document is provided to the user together with information indicating the document position information, title, and contents of the document when providing information about the document to the user. When provided, the degree of popularity change may not be provided as a numerical value, but may be provided using an icon illustrating the direction and degree of change in popularity (described later).

  Next, processing for determining related non-text contents related to the contents of each document will be described with reference to FIG. Documents often include non-text content such as images and sounds in addition to text content. Among the non-text contents included in the document, there are non-text contents that are not related to the contents of the document, such as a banner advertisement. The related non-text content determination unit 104 determines whether the non-text content included in the document is related to the content of the document based on the link relationship.

For this purpose, first, the non-text content determination unit 104 refers to the link relationship table 112 and extracts link relationship information in which the link destination ID is stored.
If there is link relation information having the same link source ID among the extracted link relation information, only the link relation information having the latest collection date or update date is adopted, and the others are deleted. This is to prevent the same processing from being performed on the same document.

  Hereinafter, a document set including the link source documents S specified by the link source ID included in the extracted link relation information is referred to as a link source document set. A document identified by the link destination ID included in the extracted link relation information (that is, the link destination document) is referred to as a determination target document C.

  The procedure from step S31 to step S40 is performed for each determination elephant document C included in each link source document S. First, the non-text content determination unit 104 extracts a link character string A that exists in the vicinity of a portion linked from each link source document S to the determination target document C (step S31).

  For example, in the case of a document using HTML, the non-text content determination unit 104 may extract 100 bytes before and after the untag (<a>) as the link character string A. Subsequently, the related non-text content determination unit 104 determines whether or not the link character string A is a specific character string (step S32).

  The specific character string is, for example, a character string indicating a non-text format such as “MPEG”, “moving image”, “streaming”, “video”, “audio”, “mp3”, or a format name of a moving image. . Assume that a table defining these specific character strings is provided in advance in the document search apparatus 100 (not shown).

  If the related non-text content determination unit 104 determines that the link character string A is a specific character string (step S32: Yes), the non-text content related to the content of the link source document S is determined as the determination target document C. And proceed to step S40. The related non-text content determination unit 104 stores the document ID of the determination target document C in the non-text content table 115 as the related non-text content ID together with the type of the determination target document C and the document ID of the link source document S. Then, the process for the determination target document C is terminated.

  When the related non-text content determination unit 104 determines that the link character string A is not a specific character string (step S32: No), the file of the determination target document C included in the document position information of the determination target document C is further included. It is determined whether or not the extension of the name is a specific extension (step S33).

In the current web, for example, the following can be considered as specific extensions. The description of each extension is omitted because it is obvious to those skilled in the art. This illustration is not intended to limit the present invention.
-For music content mp3, wma, wav
・ In the case of video content ram, rm, rv, rmm, wmv, avi, asx, qt, mov, mp
eg, mpg, fla, swf
・ Image content jpg, jpeg
The related non-text content determination unit 104 can also determine whether the determination target document C is non-text content based on such an extension. Assume that the table for defining these specific extensions is provided in advance in the document search apparatus 100 (not shown). When the related non-text content determination unit 104 determines that the extension of the file name included in the document position information of the determination target document C is not a specific extension (step S33: No), the determination target document C is a non-text content. If not, the process for the document is terminated.

  If the related non-text content determination unit 104 determines that the extension of the file name of the determination target document C is a specific extension (step S33: Yes), whether the determination target document C is further used as a link. Judge whether or not. This determination can be made based on a tag in the case of HTML, for example. The determination target document C being used as a link means that another document can be browsed by selecting (clicking or touching) the document, such as a banner advertisement image.

  For example, when a determination target document C (an image in the example) is used as a link in a document described in HTML, it is often expressed as follows. This illustration is not intended to limit the present invention.

<a href="Document position information of the linked document of the determination target document C"><img src = "Document position information of the determination target document C"></a>
The related non-text content determination unit 104 refers to the document table 111 using the document IDs of the determination target document C and the link source document S, and acquires document position information of both. Then, the related non-text content determination unit 104 stores the link source document S as the site where the determination target document C is stored based on the document position information of the determination target document C and the document position information of the link source document S. It is determined whether or not the site is the same as the site being visited (step S35).

  More specifically, when the document position information is a URL, for example, the related non-text determination unit 104 determines the determination target document C based on the server address or domain of the URL of the determination target document C and the URL of the link source document S. It is determined whether the site storing the URL and the site storing the link source document S are the same.

  When it is determined that the site storing the determination target document C and the site storing the link source document S are the same (step S35: Yes), the determination target document C is related to the contents of the link source document S. Therefore, the process proceeds to step S37 (described later). This is because when the determination target document C is related to the contents of the link source document S, the determination target document C is often stored in the same site as the site where the link source document S is stored. is there.

  On the other hand, when it is determined that the site storing the determination target document C and the site storing the link source document S are different (step S35: No), the related non-text content determination unit 104 further determines the determination target document. Based on the document position information of C and the document position information of the link destination document of the determination target document C, the link source document S is stored in the site where the document that is the link destination of the determination target document C is stored. It is determined whether or not the site is the same (step S36). Note that the document position information of the link destination document of the determination target document C is often described near the tag in which the link is embedded as in the above example.

  When it is determined that the site storing the document that is the link destination of the determination target document C is the same as the site storing the link source document S (step S36: Yes), the process proceeds to step S37. Since it is presumed that the document to which the determination target document C is linked is related to the contents of the link source document S, it can be estimated that the determination target document C is also related to the contents of the link source document S. It is.

  On the other hand, if it is determined that the site storing the document that is the link destination of the determination target document C is different from the site storing the link source document S (step S36: No), the related non-text content determination The unit 104 estimates that the determination target document C is a document that is not related to the contents of the link source document S, such as a banner advertisement, and ends the processing for the determination target document.

  In step S37, the related non-text content determination unit 104 determines whether the determination target document C has been used in the link source document S a predetermined number of times, for example, three times or more. The three times are merely examples and are not intended to limit the present invention. When it is determined that the determination target document C is used three or more times in the link source document S (step S37: Yes), the related non-text content determination unit 104 sets the determination target document C as the content of the link source document S. And the processing for the determination target document C is terminated. Otherwise, the process proceeds to step S38.

  For example, when the determination target document C is in a format such as a list bullet or a document creation material, there is a high possibility that the determination target document C is used multiple times in one document. Since such a document is considered to be unrelated to the contents of the link source document S, it is not treated as related non-text content.

  In the case of No in step S37, the related non-text content determination unit 104 further determines the link destination document of the link source document S from the document table 111 based on the link destination ID included in the link relation information of the link source document S. A file name is acquired, and it is determined whether or not the link source document S has another link destination document having a file name similar to the determination target document C (step S38).

  When it is determined that the link source document S does not have another link destination document having a file name similar to the determination target document C (step S38: No), the process proceeds to step S40, and the related non-text content determination unit 104 In this way, the determination target document C is registered in the non-text content table 115.

  When it is determined that the link source document S has another link destination document having a file name similar to the determination target document C (step S38: Yes), the related non-text content determination unit 104 determines that the determination target document C is determined It is determined whether or not the target document C and the linked document having a similar file name have the youngest file name in the dictionary order (step S39). The dictionary order means, for example, the order of alphabetical order or the order of numbers in ascending order.

  When the related non-text content determination unit 104 determines that the determination target document C has the youngest file name in the dictionary order (step S39: Yes), the process proceeds to step S40, and the determination target document C is determined as the non-text content table 115. To complete the processing for the document. Otherwise (step S39: No), the process for the document is terminated without performing step S40.

  For example, if the link source document S is a document whose contents are displayed as a list such as an album, if all of these are handled as documents related to the contents of the link source document S, the number of related documents increases, which is rather used. It may be complicated when providing a search result to a person. However, in such a case, for example, the remaining parts excluding the numerical part are often the same as each other, such as pict01.jpg, pict02.jpg, pict03.jpg,. Therefore, when there are linked documents having similar file names, it is possible to avoid such complications by registering only the document having the youngest file name in the dictionary order as the related non-text content.

  As described above, after the processing for a certain determination target document C is completed, the related non-text content determination unit 104 refers to the link relation information of the link source document S, and adds the link source document S previously extracted. It is determined whether there is an undecided linked document. When there is an undetermined link destination document, the related non-text content determination unit 104 sets the undetermined link destination document as a new determination target document C, and performs the processing from step S31 on that document.

  When the link source document S does not include another undecided link destination document, the related non-text content determination unit 104 extracts another unprocessed link source document S from the link source document set. The same processing is performed for the link destination document C of the link source document S (not shown). When all the link source documents S have been processed, the related non-text content determination process ends.

  When providing information about each document to the user, information indicating the type of related non-text content linked from the document based on the determination result, together with information indicating the document position information, title, and contents of the document For example, an icon may be provided to the user. As a result, the user can know what related non-text contents exist at the link destination of the document without actually browsing (browsing) the document. Further, when the user selects (clicks or touches) an icon by embedding a link to the related non-content in the icon indicating the type of the related non-text content, the related non-text content is displayed. May be displayed or reproduced on the user's screen (described later).

  Next, a processing procedure for determining the service type of a document will be described with reference to FIG. In a document, various services are often provided to viewers of the document. The service type determination unit 105 determines the type of service provided in the document based on the form tag used in the document. In the following description, three service types of search, shop, and application (registration) are determined.

  Here, the search service refers to a service for searching for something based on a keyword input by a user (or a viewer or the like). Shop service refers to a service for selling products to users. The application (registration) service is a service that accepts a name, an address, and the like from a user and accepts an application or registration of a member or a prize from the user. Note that these three services are examples and are not intended to limit the present invention. By adding more procedures to the process for determining the service type, it becomes possible to determine the service type in more detail.

  First, the service type determination unit 105 extracts a document including text from collected documents (not shown). Whether or not the text is included may be determined based on, for example, the extension of the file name of each document. The following processing is performed for each extracted document.

  Subsequently, the service type determination unit 105 determines whether or not a form tag is included in the document (step S41). When the form tag is not included in the document (step S41: No), it is presumed that the document does not provide a service, and thus the processing for the document is terminated.

  When the document includes a form tag (step S41: Yes), the service type determination unit 105 further determines whether or not the button included in the document includes characters such as “Purchase” or “Buy” (Step S41). S42).

For example, in the case of a document described in HTML, the buttons are often expressed as follows.
<INPUT TYPE = "submit" VALUE = "Text to display on the button">
If the button includes characters such as “purchase”, “purchase”, or “buy” (step S42: Yes), the service type determination unit 105 sets the type of service provided in the document to “shop (sales store)”. (Step S43), the process proceeds to Step S48. The service type determination unit 105 registers the service type of the document as “shop” by storing the service type “shop” determined together with the document ID of the document in the service type table 116 (step S48).

  When there is no character such as “Purchase” or “Buy” on the button (Step S42: No), the service type determination unit 105 further determines whether or not the user input area is included in the document (Step S44). . If the user input area is not included (step S44: No), it is assumed that the service is not provided for the document, and the processing for the document is terminated. When the input area of the user is included in the document (step S44: Yes), the service type determination unit 105 further determines whether or not there is a character such as “search” or “search” in the button included in the document. (Step S45).

  If the button includes characters such as “search” or “search” (step S45: Yes), the service type determination unit 105 determines that the type of service provided by the document is “search” (step S46). The process proceeds to step S48. In step S48, the service type determination unit 105 registers the service provided by the document as described above.

  When there is no character such as “search” or “search” on the button (step S45: No), the service type determination unit 105 determines that the type of service provided by the document is “application” (step S47). The process proceeds to step S48.

  As described above, the service type determination unit 105 can determine the type of service provided in the document based on the form tag without looking at the content of the document.

  Various modifications can be considered for the process of determining the service type. For example, the following processing may be performed between step S45 and step S46. First, after step S45, the service type determination unit 105 further determines whether there is an entry field for ISBN (International Standard Book Number), and if the entry field for ISBN is included, the document The service type provided by is determined as “bookstore” and the process proceeds to step S48. If the ISBN entry field is not included, the process proceeds to step S46. As a result, the service provided by the document can be determined in more detail.

  When providing information about each document to the user, information indicating the type of service provided by the document, such as an icon, based on the determination result, together with information indicating the document position information, title, and contents of the document. It may be provided to the user. Thus, the user can know the type of service provided by the document without actually browsing (browsing) the document. Further, the service type determined in the above determination can be used when classifying each page.

  The page classification unit 106 determines the content of the document based on the words in each document, and classifies each document based on the determination result. For example, “Java (registered trademark)”, “theme park”, and the like can be considered as terms indicating the content of the document. This illustration is not intended to limit the present invention. Since the method of classifying each document by the page classification unit is the same as that of the prior art, a detailed description is omitted. Note that the page classification unit 106 may use, for example, the service type provided by each document determined by the service type determination unit 105 when classifying each document.

  The search service unit 107 searches for a document based on an instruction from the user of the document search device 100, and appropriately sends the search result to the user together with the processing results of the above-described popularity calculation unit 102, popularity transition calculation unit 103, and the like. To provide. More specifically, the search service unit 107 displays the search result together with the processing result on the user terminal. Hereinafter, the processing performed by the search service unit 107 will be described with reference to the screen displayed on the user terminal as appropriate.

  The search service unit 107 provides information regarding documents obtained as a result of the search to the user in various formats. First, a case will be described in which a user inputs a keyword or the like, and a search result based on the keyword or the like is provided to the user.

  First, the search service unit 107 searches for a document based on a keyword or the like input by the user, and acquires the following information from each table for the searched document.

Obtain the latest popularity and popularity ranking from the popularity table 113.
The regression coefficient a (slope) and intercept b based on the latest popularity and popularity ranking are acquired from the popularity change table 114.

Obtain the document ID of the related non-text content from the non-text content table 115.
The service type is acquired from the service type table 116.

  Subsequently, based on the acquired regression coefficient a and intercept b, the search service unit 107 creates a popularity transition icon that illustrates the direction and speed of the popularity change. Specifically, the popularity transition icon is an icon illustrating an arrow, and the direction and speed of the popularity change are indicated by the direction and inclination of the arrow. The search service unit 107 creates, for example, the following six types as popularity degree transition icons. This illustration is not intended to limit the present invention.

Rapid increase icon: Indicates that the popularity is increasing rapidly. The sudden rise icon illustrates a rising arrow with a sharp angle.
Rising icon: Indicates that the popularity is rising. The rising icon shows a rising arrow, and the angle is closer to the horizontal than the sharp rising icon.

Down icon: Indicates that the popularity is decreasing. The descending icon shows a downward-sloping arrow, and the angle is closer to the horizontal than the sudden descending icon.
Rapidly descending icon: Indicates that the popularity is rapidly decreasing. The sudden rise icon illustrates a downward-sloping arrow with a sharp angle.

Stability icon: Illustrates a horizontal arrow pointing to the right. The color may be changed between high value stability and low value stability described later.
Unmarked icon: An icon without an arrow. Other states are shown.

The following two examples are given as examples of the method for creating popularity transition icons.
(Example 1) When the change in popularity is calculated based on the popularity (a natural number up to 10000. The greater the popularity, the higher the popularity). The search service unit 107 calculates each based on the regression coefficient a and the intercept b as follows. Determine the icon to be attached to the document.

Raise icon: When the document's a is 50 or more Raise icon: When the document's a is 30 or more Decline icon: When the document's a is -30 or less Rapid drop icon: The document's a is -50 or less Case High value stable icon: When b of the document is 8000 or more Low value stable icon: When b of the document is 3000 or less Unmarked icon: In other cases (Example 2) Popularity change in popularity ranking (from 1 to total document) When the calculation is performed with a natural number up to a number (the smaller the better the popularity ranking is), the search service unit 107 determines an icon to be attached to each document as follows.

Rapid rise icon: When a / b of the document is -0.1 or less (increased by 10% or more) Lift icon: When a / b of the document is -0.05 or less (increased by 5% or more) Decline icon: That When a / b of document is 0.05 or more (decrease of 5% or more) Rapid drop icon: When a / b of document is 0.1 or more (decrease of 10% or more) High value stable: b of document is 1000 In the following cases: Low value stable: When b of the document is 100,000 or more No mark: Other cases Subsequently, the search service unit 107 illustrates the type of the related non-text content for the document in which the related non-text content is registered. A related media icon is created, and a link to the related non-text content is embedded in the related media icon. Thus, when the user selects the related media icon, the related non-text content can be browsed, reproduced, etc. without browsing the link source document of the related non-text content.

  The related media icon displays, for example, the type of related non-text content. More specifically, when the related non-text content is in the jpg format, the related media icon represents a character string “jpg”. Alternatively, the related media icon may show the camera as an image. When a plurality of related non-text contents are registered in the document, this process is performed for each related non-text content.

  Further, the search service unit 107 creates a service content icon illustrating the type of service type for a document in which the service type has been registered. The service content icon is, for example, an icon that displays the type of service. More specifically, when the service type is a shop, the service content icon describes a character string “shop”. Alternatively, the service content icon may indicate a shop.

  Finally, the search service unit sorts each document obtained as a result of the search based on the popularity ranking, and in the sorted order, each document title, document content information, document document location information, popularity A transition icon, a related media icon, and a service content icon are set on the screen. As a result, a search result display screen as shown in FIG. 14 is created.

  In the search result display screen shown in FIG. 14, the documents are arranged in the order of the latest popularity, that is, in the order of static popularity. The user can know how the degree of popularity of each document has changed and, as a result, the ranking has reached this rank. Further, the user can know what non-text document each document is linked to by the related media icon, and further, by selecting (clicking or touching) the related media icon, It is possible to play back or browse text content. Therefore, the user can know what non-text content is linked from the document without browsing the document.

Furthermore, the user can know what service each document provides by using the service content icon.
In FIG. 14, when the user selects (clicks or touches) the popularity transition icon, the search service unit 107 calculates the document for which the popularity transition icon is selected within a certain past period, for example, within several months. The obtained popularity or popularity ranking is acquired from the popularity table 113, a graph of popularity or popularity ranking for the date for which popularity is calculated is created and set on the screen.

  FIG. 15A shows an example of a popularity transition screen in which a graph of popularity ranking with respect to the date for which popularity is calculated is set. In FIG. 15A, the horizontal axis indicates the date, and the vertical axis indicates the popularity ranking. In the graph, the numbers are described above and below, but the upper number indicates the popularity ranking, and the lower number indicates the date on which the popularity is calculated. This graph shows how the popularity of the document has changed over the past few months. It corresponds to the visualization of the popularity change table. As shown in FIG. 15 (a), URL: www. It can be seen that the popularity ranking of the documents specified by aaa has risen sharply in March and has remained almost stable since May.

  In FIG. 15A, when a part of the graph is selected, the search service unit 107 sets a date within an appropriate period near the selected date as a collection date or an update date, and sets the document ID of the document. The link relation information used as the link destination ID is acquired from the link relation table 112. Then, based on the acquired link relation information, the search service unit 107 creates a list of documents with the document as a link destination within the certain period and sets it on the screen.

  Within a period shown in FIG. 15B, URL: www. A document in which the document specified by aaa is a link destination, that is, URL: www. An example of the screen which shows the list of the link origin documents of the document specified by aaa is shown. FIG. 15B allows the user to know what document the document is linked to at that time. For example, if the user has URL: www. If it is the site master of the document specified by aaa, the user can apply this information to future site maintenance.

  Further, the user registers the document position information and popularity threshold of a document in advance in the search service unit 107, and the search service unit 107 has the popularity of the document equal to or higher than the threshold. In that case, the user may be notified. Also in this case, since the user can automatically know the change in popularity of the document, the user can apply this information to future site maintenance and the like.

  The document search apparatus of the present invention can be used for various purposes other than general search. For example, the document search apparatus 100 can be used as an industry analysis tool. The trend of popularity in a specific industry is displayed using the document search device 100, and the user can use this trend in popularity to help marketing. For this purpose, the user first creates a list (for example, a URL collection) of document position information of company top pages (documents) in the industry to be known.

  Subsequently, the document search apparatus 100 acquires the latest popularity degree of each document included in the list of document position information from the popularity degree table 113, and displays a popularity degree list in which each document is displayed in order of the acquired popularity degree. Set. This popularity list means the current industry ranking.

  FIG. 16A shows an example of the popularity list. Buttons labeled “Past 1 month” and “Past 1 year” are set at the lower end of FIG. When this button is pressed, the document search apparatus further acquires the popularity of each document included in the document position information list calculated for the past month or the past year from the popularity table 111, and the popularity degree A graph showing the transition of popularity with respect to the calculated date is created and set on the screen. Needless to say, popularity ranking may be used instead of popularity.

  FIG. 16B shows an example of a graph showing the transition of the popularity of each document in the past year. FIG. 16B shows the transition of the popularity of each document in the list shown in FIG. 16A in the past year. In FIG. 16A, the button labeled “Past 1 year” is pressed. Displayed on the user's terminal. In FIG. 16B, the horizontal axis indicates the date on which the popularity is calculated, and the vertical axis indicates the popularity. As shown in FIG. 16B, URL: bbb. co. It can be seen that the popularity of documents with jp has soared over the past year.

  Further, for example, the document search device 100 can be used as a regional information search system. For this purpose, first, the page classification unit 106 creates a hierarchical category indicating a region such as a prefecture or a municipality, and classifies each document according to the category. The user can access a service provided by a hierarchical category and a desired document and its popularity, popularity transition, related media, and page.

  FIG. 17 shows an example of a screen of the area information search system. FIG. 17A shows an example of a screen for displaying a list of documents related to the category “Tokyo”. In FIG. 17A, the selected area “Tokyo” is displayed in the upper part of the screen, each ward in Tokyo is displayed in the middle part, and information on each document classified as “Tokyo” is displayed in the lower part. ing. Since the lower part of the screen is the same as the search result display screen shown in FIG. 14, it is omitted in FIG. When the user selects “Minato Ward” in the upper part of the screen of FIG. 17A, the screen transitions to a screen displaying a list of documents related to the category “Minato Ward”.

  FIG. 17B shows an example of a screen that displays a list of documents related to the category “Tokyo-Minato Ward”. In FIG. 17B, the selected area “Minato Ward” is displayed in the upper part of the screen, the town name in Minato Ward is displayed in the middle part of the screen, and “Tokyo-Minato Ward” is classified in the lower part of the screen. Information about each document is displayed. The lower part of the screen is the same as the search result display screen shown in FIG. When the user further selects “Roppongi” in the upper part of the screen of FIG. 17B, the screen transitions to a screen displaying a list of documents relating to the category “Tokyo-Minato-Roppongi”.

  FIG. 17C shows an example of a screen displaying a list of documents related to the category “Tokyo-Minato-Roppongi”. In FIG. 17C, the selected region “Roppongi” is displayed in the upper part of the screen, the other categories are displayed in the middle part of the screen, and “Tokyo-Minato-Roppongi” is classified in the lower part of the screen. Information about the document is displayed.

  The document search apparatus 100 and the user terminal described in the present embodiment can also be configured using a computer (information processing apparatus) as shown in FIG. 18 includes a CPU 201, a memory 202, an input device 203, an output device 204, an external storage device 205, a medium driving device 206, and a network connection device 207, which are connected to each other via a bus 208.

  The memory 202 includes, for example, a ROM (Read Only Memory), a RAM (Random Access Memory), and the like, and stores programs and data used for processing. The CPU 201 performs necessary processing by executing a program using the memory 202.

  When the computer 200 realizes a function corresponding to the document search device 100, the collection unit 101, the popularity calculation unit 102, the popularity transition calculation unit 103, and the related non-text content determination unit that configure the document search device 100 shown in FIG. 104, the service type determination unit 105, the page classification unit 106, and the search service unit 107 are realized as programs indicating processes performed by the respective units, and are stored in specific program code segments in the memory 202. Note that the processing performed by each of the above-described units is described in each flowchart.

  The input device 203 is, for example, a keyboard, a pointing device, a touch panel, and the like, and is used for inputting instructions and information from the user. The output device 204 is, for example, a display or a printer, and is used to output an inquiry to a user of the computer 200, a processing result, and the like.

  The external storage device 205 is, for example, a magnetic disk device, an optical disk device, a magneto-optical disk device, or the like. The above-described program and data can be stored in the external storage device 205, and can be loaded into the memory 202 and used as necessary.

  The medium driving device 206 drives the portable recording medium 209 and accesses the recorded contents. The portable recording medium 209 includes a memory card, a memory stick, a flexible disk, a CD-ROM (Compact Disc Read Only Memory), an optical disk, a magneto-optical disk, a DVD (Digital Versatile Disk), and the like, which can be read by any computer. Is used. The above-described program and data can be stored in the portable recording medium 209, and can be loaded into the memory 202 and used as necessary.

  The network connection device 207 communicates with an external device via an arbitrary network (line) such as a LAN and a WAN, and performs data conversion accompanying the communication. If necessary, the above-described program and data can be received from an external device and loaded into the memory 202 for use.

FIG. 19 is a diagram for explaining computer-readable recording media and transmission signals that can supply programs and data to the computer of FIG.
By supplying the above-described program and data stored in each table to the computer 200 as follows, it is possible to cause the computer 200 to perform a function corresponding to the document search apparatus 100. For this purpose, the above-described program and data are stored in advance in a computer-readable recording medium 29. Then, as shown in FIG. 19, using the medium driving device 206, a program or the like is read from the recording medium 29 by the computer 200 and temporarily stored in the memory 202 or the external storage device 205 of the computer 200. The CPU 201 may be configured to read and execute the stored program.

  Instead of causing the computer to read the program from the recording medium 209, the program may be downloaded from the DB 210 held by the program (data) provider via the communication line (network) 211. In this case, for example, in a computer having the DB 210 and transmitting a program, the program data representing the program is converted into a program data signal, and the converted program data signal is modulated using a modem. Thus, a transmission signal is obtained, and the obtained transmission signal is output to the communication line 211 (transmission medium). The computer that receives the program obtains the program data signal by demodulating the received transmission signal using a modem, and obtains the program data by converting the obtained program data signal.

  If the communication line 211 (transmission medium) connecting the computer on the transmission side and the computer on the reception side is a digital line, it is also possible to communicate program data signals. Further, a computer such as a telephone office may be interposed between a computer having a database (DB) 210 and transmitting a program and a computer downloading the program.

  As mentioned above, although embodiment of this invention was described, this invention is not limited to embodiment mentioned above, Various other changes are possible.

(Supplementary Note 1) A popularity calculation method for calculating a popularity that is a degree of popularity of a document on a network,
Extract link relationships from documents,
Extracting a document updated or collected within a first period as a target for calculating the popularity,
Calculating the popularity of each extracted document;
The popularity calculation method characterized by including thing.

(Supplementary Note 2) The popularity is calculated based on document position information indicating the link relation and the position of the document on the network.
The popularity calculation method according to supplementary note 1, further comprising:

(Additional remark 3) Based on the characteristic of the character string which shows the said document position information, the said popularity is calculated.
The popularity calculation method according to supplementary note 2, further comprising:

(Supplementary Note 4) A popularity change degree indicating a direction and a degree of change in the popularity degree of the document is calculated.
The popularity calculation method according to supplementary note 1, further comprising:

(Supplementary Note 5) Based on the popularity degree calculated within the second period, the popularity change degree is calculated.
The popularity calculation method according to supplementary note 4, further comprising:

(Supplementary Note 6) Calculate a regression equation with respect to time of the popularity calculated within the second period,
Calculating the degree of popularity change based on the regression equation;
The popularity calculation method according to supplementary note 5, further comprising:

(Appendix 7) The popularity change degree is determined based on a regression coefficient of the regression equation.
The popularity calculation method according to appendix 6, further comprising:
(Additional remark 8) Based on the intercept of the said regression equation, the tendency of the transition to the time of the popularity is determined.
The popularity calculation method according to appendix 7, further comprising:

(Supplementary note 9) Based on the popularity calculated within the second period, the rank of each document in the extracted document is determined,
Calculating a regression equation for the time of the rank within the second period;
Calculating the degree of popularity change based on the regression equation;
The popularity calculation method according to supplementary note 5, further comprising:

(Supplementary Note 10) A document relationship determination method for determining a relationship between documents on a network,
Extract link relationships from the first document,
Determining whether a second document linked from the first document is a non-text document related to the content of the first document based on the link relationship;
A document relation determination method characterized by including:

(Supplementary Note 11) Extracting from the first document a character string in the vicinity of a portion linked from the first document to the second document;
Determining whether the second document is a non-text document related to the content of the first document based on the character string;
The document relationship determination method according to supplementary note 10, further comprising:

(Supplementary Note 12) When the character string is a specific character string, the second document is determined to be a non-text document related to the content of the first document.
The document relation determination method according to claim 11, further comprising:

(Supplementary Note 13) Based on the extension of the file name of the second document, it is determined whether or not the second document is a non-text document related to the contents of the first document.
The document relationship determination method according to supplementary note 10, further comprising:

(Supplementary Note 14) If the extension is not a specific extension, the second document is determined not to be a non-text document related to the content of the first document.
The document relationship determination method according to supplementary note 13, further comprising:

(Supplementary Note 15) Based on whether the second document has been used more than a predetermined number of times in the first document, the second document is a non-text document related to the contents of the first document. Determine if there is,
The document relationship determination method according to supplementary note 10, further comprising:

(Supplementary Note 16) If the second document has been used more than a predetermined number of times in the first document, it is determined that the second document is not a non-text document related to the contents of the first document.
The document relationship determination method according to supplementary note 10, further comprising:

(Supplementary Note 17) If the second document has not been used more than a predetermined number of times in the first document, it is determined that the second document is a non-text document related to the contents of the first document. ,
The document relationship determination method according to supplementary note 10, further comprising:

(Supplementary Note 18) When there is a third document having a file name similar to the file name of the second document in the first document, the file name of the second document is the name of the third document. Not registering the second document in the database as a non-text document related to the content of the first document if it is not lexicographically younger than the file name;
The document relationship determination method according to supplementary note 10, further comprising:

(Supplementary note 19) It is determined whether there is a third document linked from the second document.
The document relationship determination method according to supplementary note 10, further comprising:

(Supplementary Note 20) When there is a third document linked from the second document, based on the document position information indicating the position of the first document on the network and the document position information of the second document. Determining whether the second document is a non-text document related to the content of the first document;
The document relationship determination method according to appendix 19, further comprising:

(Supplementary Note 21) Whether the second document is a non-text document related to the contents of the first document based on the document position information of the first document and the document position information of the third document Whether or not
The document relationship determination method according to supplementary note 20, further comprising:

(Supplementary Note 22) If the document position information of the second document and the document position information of the third document do not have the same server address or domain as the document position information of the first document, the second document The document is not a non-text document related to the content of the first document,
The document relation determination method according to supplementary note 21, further comprising:

(Supplementary note 23) A service type determination method for determining a type of service provided by a document on a network,
Extracting a tag specifying user input from the document;
Determining a type of service provided by the document based on a tag designating the user input;
A service type determination method characterized by including:

(Supplementary Note 24) If the document does not include a tag for designating the user input, the document is determined not to provide a service.
The service type determination method according to supplementary note 23, further comprising:

(Supplementary Note 25) Based on the display of buttons included in the document, the type of service provided by the document is determined.
The service type determination method according to supplementary note 23, further comprising:

(Supplementary Note 26) Based on a user input area included in the document, a type of service provided by the document is determined.
The service type determination method according to supplementary note 25, further comprising:

(Supplementary note 27) A program for causing a computer to execute control for calculating the degree of popularity, which is the degree of popularity of documents on a network,
Extract link relationships from documents,
Extracting a document updated or collected within a first period as a target for calculating the popularity,
Calculating the popularity of each extracted document;
A program that causes the computer to execute a process including the above.

(Supplementary Note 28) Calculate a popularity change degree indicating a direction and a degree of change in the popularity degree of the document.
28. The program according to appendix 27, further causing the computer to execute processing including the above.

(Supplementary Note 29) Based on the popularity calculated within the second period, the popularity change is calculated.
29. The program according to appendix 28, further comprising causing the computer to execute a process further including:

(Supplementary Note 30) Calculate a regression equation for the popularity time calculated within the second period,
Calculating the degree of popularity change based on the regression equation;
32. The program according to appendix 29, further comprising causing the computer to execute a process further including:

(Supplementary Note 31) The popularity change degree is determined based on a regression coefficient of the regression equation.
The program according to appendix 30, characterized by causing the computer to execute a process further including the above.

(Supplementary Note 32) Based on the intercept of the regression equation, determine the trend of the popularity with respect to time,
32. The program according to appendix 31, wherein the program further causes the computer to execute processing.

(Supplementary note 33) A program for causing a computer to execute control for determining a relationship between documents on a network,
Extract link relationships from the first document,
Determining whether a second document linked from the first document is non-text content related to the content of the first document based on the link relationship;
A program for causing the computer to execute a process including the above.

(Supplementary Note 34) A program for causing a computer to execute control for determining a type of service provided by a document on a network,
Extracting a tag specifying user input from the document;
Determining a type of service provided by the document based on a tag designating the user input;
A program for causing the computer to execute a process including the above.

(Supplementary Note 35) A document search method for searching a document from a network,
Collect documents from the network,
Extracting link relationships from the document;
Extracting a document updated or collected within a first period as a target for calculating the popularity,
Calculating the popularity of each extracted document;
Search for documents based on search criteria,
Ranking the retrieved documents based on the popularity;
Based on the ranking result, information on the searched document is output.
A document search method characterized by including:

(Supplementary Note 36) Based on the popularity degree calculated within the second period, a popularity change degree indicating a direction and a degree of change of the popularity degree of the document is calculated,
Adding information about the popularity change to information related to the retrieved document;
36. The document search method according to supplementary note 35, further comprising:

(Supplementary Note 37) Based on the link relationship, it is determined whether another document linked from the document is a related non-text document related to the content of the document,
Adding information related to the related non-text document to information related to the retrieved document based on the result of the determination;
36. The document search method according to supplementary note 35, further comprising:

(Supplementary Note 38) Embed a link to the related non-text document in the information about the related non-text document.
38. The document search method according to appendix 37, further comprising:

(Appendix 39) Extracting a tag specifying user input from the document,
Determining the type of service provided by the document based on a tag designating the user input;
Adding information about the type of service to information related to the retrieved document;
36. The document search method according to supplementary note 35, further comprising:

(Appendix 40) Accepting registration of document position information indicating a position of a document on the network and a predetermined value from a user,
When the popularity of the document specified by the document position information reaches the predetermined value, the user is notified that the popularity has reached the predetermined value;
36. The document search method according to supplementary note 35, further comprising:

(Supplementary note 41) A document search device for searching a document from a network,
Collecting means for collecting documents from the network and extracting link relationships from the collected documents;
A popularity degree calculating means for extracting a document updated or collected within a first period as a target for calculating the popularity degree, and calculating a popularity degree of each of the extracted documents;
Search service means for searching for a document based on a search condition, ranking the searched document based on the popularity, and outputting information on the searched document based on the ranking result;
A document search apparatus comprising:

(Supplementary Note 42) A regional information document retrieval device for retrieving a document related to a region from the network,
Collecting means for collecting documents from the network and extracting link relationships from the collected documents;
A popularity degree calculating means for extracting a document updated or collected within a first period as a target for calculating the popularity degree, and calculating a popularity degree of each of the extracted documents;
Popularity degree transition calculating means for calculating a popularity change degree indicating a direction and a degree of change of the popularity degree based on the popularity degree calculated within a second period;
Related non-text content determination means for determining whether a document linked from each document is a related non-text document related to the contents of each document based on the link relationship between the collected documents;
A service type determination unit that extracts a tag specifying user input from the collected document and determines a type of service provided by the document based on the tag specifying the user input;
A classification means for hierarchically classifying the collected documents for each area name;
The document is searched based on a region name specified by the user, the searched document is ranked based on the popularity, and the searched result is searched together with information about the searched document based on the ranking result. Search service means for outputting information on the degree of popularity change of the document, information on the related non-text document, and information on a service type provided by the searched document;
A document search apparatus comprising:

It is a principle diagram of the present invention. It is a block diagram of the document search apparatus concerning this invention. It is a figure which shows an example of the data structure of a document table. It is a figure which shows an example of the data structure of a link relation table. It is a figure which shows an example of the data structure of a popularity table. It is a figure which shows an example of the data structure of a popularity change table. It is a figure which shows an example of the data structure of a non-text content table. It is a figure which shows an example of the data structure of a service classification table. It is a flowchart which shows the procedure of the process which calculates popularity. It is a figure explaining the characteristic of the present invention in calculation of popularity. It is a flowchart which shows the procedure of the process which calculates popularity change. It is a flowchart which shows the procedure of the process which determines related non-text content. It is a flowchart which shows the procedure of the process which determines the service to provide. It is a figure which shows an example of the display screen of a search result. It is a figure which shows an example of a popularity transition screen. It is a figure which shows an example of the screen of the industry analysis tool to which this invention is applied. It is a figure which shows an example of the screen of the area information search system to which this invention is applied. It is a block diagram of a computer. It is a figure explaining the recording medium and transmission signal which can provide a program and data to a computer.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Document rearrangement apparatus 11 Popularity calculation means 12 Popularity transition calculation means 13 Related non-text content determination means 14 Service type determination means 100 Document search device 101 Collection part 102 Popularity calculation part 103 Popularity degree transition calculation part 104 Related non-text content Determination unit 105 Service type determination unit 106 Page classification unit 107 Search service unit 108 Browser 111 Document table 112 Link relation table 113 Popularity table 114 Popularity change table 115 Non-text content table 116 Service type table 200 Computer 201 CPU
202 Memory 203 Input Device 204 Output Device 205 External Storage Device 206 Medium Drive Device 207 Network Connection Device 208 Bus 209 Portable Recording Medium 210 Program (Data) Provider 211 Line

Claims (4)

A program for causing a computer to execute control for determining a type of service provided by a document on a network,
From the text included in the document, extract a tag that matches a character string indicating a tag that specifies user input,
It is determined whether or not a character string indicating a service provided by the document prepared in advance is included in the attribute value included in the extracted tag, and a service provided by the document is specified from the included character string. ,
When outputting the document position information about the document, the information indicating the service provided by the document is also output.
A program for causing the computer to execute a process including the above. A service type determination method for determining a type of service provided by a document on a network,
From the text included in the document, extract a tag that matches a character string indicating a tag that specifies user input,
It is determined whether or not a character string indicating a service provided by the document prepared in advance is included in the attribute value included in the extracted tag, and a service provided by the document is specified from the included character string. ,
When outputting the document position information about the document, the information indicating the service provided by the document is also output.
A service type determination method characterized by including: If the document does not include a tag specifying the user input, the document is determined not to provide a service;
The service type determination method according to claim 2, further comprising: Determining the type of service provided by the document based on the display of buttons included in the document;
The service type determination method according to claim 2, further comprising: