JP2009048380A - Search system, search apparatus and search method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve the problems wherein existing search services will exclude even popular and frequently accessed sites from the scope of search if they are not linked and cannot be reached by crawlers, and the crawlers must frequently visit popular pages, which are expected to be updated frequently, to maintain the latest indexes of search service databases. <P>SOLUTION: Many unspecified persons' access histories are collected by means of tool bar devices, and collection information is created therefrom. Popular new sites can be extracted and indexed quickly from the collection information. The reflection of the collection information in the patrolling schedule of crawlers implements a search service well adapted to users' browsing states. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a technique for efficiently searching for resources such as Web pages in a WWW (World Wide Web) search system.

  When an Internet user tries to search for resources such as Web pages, documents, and image data on the WWW, a Web search system is usually used. The search system generates an indexer related to resources on the WWW as follows in order to execute a search process according to a search keyword from the user. In other words, the search system executes a program called “crawler” that traces hyperlinks between resources and circulates on the WWW, and relates to the URL (Uniform Resource Locator) and content contents of the resources that the crawler circulates on the WWW. Get information. Then, index information related to resources on the WWW is constructed and accumulated from the collected information. When a search keyword or the like is acquired from the user, the search system refers to the index information as a search target and returns the search result to the user terminal.

  By the way, these resources on the WWW are usually updated. One of the features of the Internet is the immediate provision of information (resources). Therefore, in order to obtain the latest updated provision information, the user has a tendency to search mainly Web pages with high update frequency such as current news pages.

  Therefore, in order to meet such user needs, the search system periodically performs resource information collection processing and index information generation / accumulation processing by crawlers according to resource update frequency, etc., and maintains the information freshness. It is desirable to be configured as follows.

  However, in order to maintain the freshness of the index information, for example, if the configuration is such that the crawler's circulation frequency is simply increased to maintain the freshness of the index information, traffic on the WWW will be compressed. Therefore, a technology is also provided that keeps the index information fresh without squeezing too much traffic on the WWW by identifying resources that should be updated frequently and raising the cyclic priority for those resources. .

Specifically, as described above, Internet users tend to search and access Web pages that are frequently updated. In other words, there is a high possibility that frequently accessed resources are frequently updated. Therefore, for example, by counting the number of user accesses, the patrol priority for resources of “crawlers” is determined, and by increasing the number of patrols for resources with high priority, an index can be created without increasing the traffic load on the WWW. It is a technology that can maintain the freshness of information.
JP 2006-277732 A

  However, conventional search systems have the following universal problems. First, in order to maintain the freshness of the index information, as described above, the “crawler” follows the hyperlink between resources and circulates on the WWW. Therefore, the resource to be indexed is hyperlinked to another resource. Need to be. However, a newly opened Web page or the like may not be hyperlinked to any resource. Then, since such a resource cannot be reached by the crawler, it is not indexed. In other words, there is a possibility that a newly opened Web page or the like that should have extremely high information freshness may not be included in the index file that is referred to in the search, and thus may not be reflected in the search result.

  Next, in the conventional search system, the number of accesses of the above users is aggregated to determine the crawler's patrol priority, and the freshness of information is maintained without imposing much network load. It is not limited to active access by the will of the person. For example, there may be included access due to a link being clicked due to an operation error or the like, or access made as a waypoint to reach the target web page by following the link. Therefore, the total number of resource accesses may not indicate an effective access entity (the number of active accesses due to the user's intention to view). For this reason, the crawler's patrol schedule established from the erroneous tabulation results may not have been able to calculate an accurate patrol priority.

  Furthermore, the display ranking (order) of each search result when displaying a list of search results is based on the page rank performed according to the number of links between pages, and there is also a method of ranking the pages clicked by the user at the top. Are known. However, in this case, as described above, since there is a possibility that the effective access entity is not indicated for the user click, the method of displaying the search result based on the erroneous aggregation result is in line with the practical use. There was a possibility that an accurate ranking could not be calculated.

  In order to mainly solve the above problems, the present invention provides the following search system. That is, the browsing information including the URL of the resource browsed by the user is acquired from the browser via the toolbar device at the user terminal or the like, and transmitted to the browsing management server device. The browsing information of each terminal managed by the browsing management server device is collected by a search device having an indexing function for searching resources on the WWW and used by the search device. Provided is a search system configured to be able to In particular, the above configuration is characterized in that browsing information including a URL of a resource browsed by a user in the toolbar device is acquired.

  And in this search system, using browsing information acquired via the toolbar device, for example, first, the contents of resources that are not hyperlinked to other pages are acquired for indexing, or even "crawler" It is further characterized in that it can be designated as a circulation destination. Secondly, it is possible to calculate a traffic rank, which is an effective access index (viewing degree index) of a resource that is tabulated by eliminating such erroneous access as described above, and can use it for a crawler's traveling schedule. It is further characterized in that it is also configured.

  Specifically, a toolbar apparatus having a browsing information acquisition unit that acquires browsing information including at least a browsing URL from a browser, and a browsing information transmission unit that transmits the acquired browsing information to a predetermined browsing management server device, And a search device that collects browsing information transmitted from the browsing information transmitting unit.

  And in order to implement | achieve the said 1st or 2nd further feature point, in addition to the said structure, the search system of this invention is new in a search engine first based on the browsing information collected in the said search apparatus. A first search management server device having a first indexer unit that extracts a new search target URL that is a URL used as a search target and indexes the content of the extracted URL may be included.

  Second, in the search device, a browsing information storage unit that stores the collected browsing information, and a traffic rank that is an audience index for each URL based on the browsing information stored in the browsing information storage unit. A schedule that includes a second browsing management server device having a traffic rank scoring unit to be calculated, and further determines a crawling schedule of the crawler unit based on the crawler unit and the traffic rank calculated by the traffic rank scoring unit And a determination unit.

  According to the first aspect of the present invention configured as described above, browsing information including a URL of a resource browsed by a user transmitted from a browser via a toolbar device is retrieved via a browsing management server device. Can be collected and used in various ways. Specifically, first, the content of a resource that is not hyperlinked to another page can be acquired and index file processing for search can be performed. It can also be designated as a crawler destination. As a result, resources such as a newly opened Web page that seems to have extremely high information freshness can be obtained in a crawler and indexed as a search target.

  Second, it is possible to calculate a traffic rank that is an access index (viewing degree index) of a resource that is tabulated by eliminating erroneous access and the like, and can use it for a traveling schedule of “crawlers”. Therefore, it is possible to establish crawler cyclic scheduling according to the effective access entity of the user's resources.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the present invention is not limited to these embodiments, and can be implemented in various modes without departing from the spirit of the present invention.

  The first embodiment will mainly describe claim 1 and the like. In the second embodiment, claims 2, 7, and 12 will be mainly described. In the third embodiment, claims 3, 8, and 13 will be mainly described. In the fourth embodiment, claims 4, 9, and 14 will be mainly described. In the fifth embodiment, claims 5, 10, 15 and the like will be mainly described. In the sixth embodiment, claims 6, 11 and 16 will be mainly described.

  Example 1

    <Overview>

  FIG. 1 is a conceptual diagram for explaining an example of browsing information collection in the search system of the present embodiment. As shown in this figure, a browser is activated on the client terminal 1. In addition, a toolbar is incorporated in the browser, and the toolbar is displayed in a predetermined area (0101) on the browser as the browser is activated.

  In this embodiment, the search key and URL input by the user to the browser and toolbar are collected by the toolbar program. Also, information indicating the Web page transition time, such as the display time of the Web page in the browser and the destination and source, is acquired by the toolbar. Then, the information acquired at each of the client terminals 1, 2, 3,... On the network in this way is transmitted to the search device (0102) including the browsing management server device.

  FIG. 2 is a diagram illustrating an example of browsing information acquired by the toolbar of the client terminal and collected by the search device via the browsing management server device. As shown in this figure, the tool bar displays the browsing information as the user ID, the URL of the Web page accessed by the browser, the browsing start time (access time) of the Web page, the browsing time (display time of the Web page, etc.), or For example, information indicating a transition path of a Web page indicated by a movement destination or a movement source (not shown) is acquired.

  The search device uses the browsing information collected from each client terminal via the browsing management server device, for example, to analyze the user's Web browsing behavior, or from the analysis result, the site that is popular with the user Etc. can be quickly grasped. For example, when collecting index information for search by a so-called “crawler”, the frequency of frequently crawling the popular site is increased so that the latest information of the popular site is provided by the search. It can be done.

    <Functional configuration>

  FIG. 3 is a diagram illustrating an example of functional blocks in the search system according to the present embodiment. As shown in this figure, the “search system” (0300) of this embodiment includes a “toolbar device” (0310) and a “search device” (0320).

  The “toolbar device” (0310) refers to a device in which an application configured to operate with a browser is incorporated in order to assist, expand, or substitute the browser function on the client terminal. Also, as a general function of the toolbar device, for example, a resource search function on the Web using a keyword input in a search window, a RSS document data collection and display function, a launcher function using a shortcut icon, and the like can be cited. And in the search system of a present Example, it has the further function of acquiring the browsing information containing URL browsed with the browser by the structure mentioned later, and transmitting to a browsing management server apparatus, It is characterized by the above-mentioned. .

  The “search device” (0320) is a method for generating index information by processing of a predetermined search engine using information on resources on the Web collected by a “crawler” as described above, and a search keyword from a user. This refers to a server device that executes index information search processing according to the above. In the search system of the present embodiment, the browsing information acquired by the toolbar device of the client terminal and transmitted to the browsing management server device is collected and used for the search processing, index information generation processing, and the like. To do.

  In this embodiment, as shown in FIG. 3 (a), the browsing management server device will be described as an example of a configuration included in this search device, but of course the configuration of the search system of this embodiment is limited to that. is not. For example, as shown in FIG. 3B, the browsing management server device is provided on the network as a server device separate from the searching device, and the searching device acquires browsing information managed by the browsing management server device. It may be configured as follows.

  Here, the functional configuration of the “toolbar device” (0301) that constitutes the search system of this embodiment together with the search device will be described below. As shown in FIG. 3, the “toolbar apparatus” includes a “browsing information transmission unit” (0303) and a “browsing information acquisition unit” (0304).

  Note that each functional block of the toolbar device and the search device of the system described below can be realized as hardware, software, or both hardware and software. Specifically, if a computer is used, a CPU, a main memory, a bus, or a secondary storage device (a hard disk, a non-volatile memory, a storage medium such as a CD or a DVD, a reading drive for the medium, etc.), a printing device And display devices, hardware components such as other external peripheral devices, I / O ports for the external peripheral devices, driver programs and other application programs for controlling these hardware, and users used for information input Examples include interfaces.

  Then, by CPU processing according to the program developed on the main memory, the data input via the interface and stored in the memory and hard disk are processed and stored, and the above hardware and software are controlled. Instructions are generated. The present invention can be realized not only as a system but also as a method. A part of the invention can be configured as software. Furthermore, a software product used for causing a computer to execute such software and a recording medium in which the product is fixed to a recording medium are naturally included in the technical scope of the present invention (the same applies throughout the present specification). Is).

  The “browsing information acquisition unit” (0311) has a function of acquiring browsing information including at least a browsing URL from a browser. “Browse URL” refers to information indicating the location on the Web of the resource displayed in the browser of the client terminal in which the toolbar device is incorporated. For example, “http: //... /001.bmp And the like.

  In the search system according to the present embodiment, various processes using the browse URL acquired by the toolbar of each client terminal on the network and collected by the search device via the browse management server device, for example, “crawler” It is characterized by performing processing such as determining a patrol target.

  In addition to the browsing URL, the browsing information may include various types of information related to browsing with a browser. For example, user ID, browser ID, toolbar ID, browsing start time information, browsing time information, or transition information such as “movement source resource” and “movement destination resource” when browsing two or more resources on the web continuously Etc.

  Here, the user ID refers to information for identifying the user, and includes, for example, an ID at the time of logging in to the client terminal, an ID assigned in service units, and the like. Similarly, the browser ID and the toolbar ID are information for identifying the browser and the toolbar. For example, a product ID recorded in advance in the browser program or the toolbar program may be used. Since the client terminal can be identified by using these identification information, the browsing information can be managed and analyzed in units of client terminals (users). The ID information can be held by, for example, a method that the toolbar device itself holds, a method that uses the cookie function of the browser device, or the like.

  The browsing start time information refers to a time when a resource on the Web is accessed by the browser, a time when the resource is acquired by the browser by the access, a time when the resource is displayed, or the like. Specifically, the time includes, for example, the time when the loading completion of the resource requested for access by the browser is detected, the time when the Web server responds with an HTTP status code such as “200: OK”, and the like.

  The browsing time information refers to the time when resources on the Web are displayed on the browser. For example, the time when the browser UI window is open, the time when the window is active, and the window For example, the time when the mouse pointer is in the area. In addition, since there is no operation for a certain period of time, when the OS of the client terminal automatically performs logoff processing or when a screen saver is activated, the log off time or the time when the screen saver is activated is excluded. It may be the time after. The browsing time information may be acquired by calculating a difference between the browsing start time and the browsing end time.

  The transition information is information indicating “movement source resource” and “movement destination resource” when two or more resources on the web are browsed continuously. For example, in addition to transition by clicking a hyperlink, Examples include transition information in which two or more URLs continuously input in the URL input field of the toolbar are “movement destination” and “movement source”.

  The toolbar device acquires the browsing information from the browser of the client terminal using, for example, an API (Application Program Interface), a DDE (Dynamic Data Exchange), a file system, or the like.

  The “browsing information transmission unit” (0312) has a function of transmitting the acquired browsing information to a predetermined browsing management server device. Specifically, it has a function of transmitting the browsing information acquired as described above to a predetermined browsing management server apparatus via a communication line or the like using the communication I / F of the client terminal. Note that, for example, a method of specifying a transmission destination address of a predetermined browsing management server device, which is a transmission destination of browsing information, by being described in advance in a program of a toolbar device can be used. In addition, the browsing information is transmitted from the browsing information transmission unit every time the resource displayed in the browser is switched, or is transmitted by batch processing when a certain time or time arrives. Or the timing to send when the tool bar starts or ends.

  And the browsing information acquired in the toolbar apparatus according to browsing with the browser on each client terminal in this way is transmitted to the browsing management server apparatus. Then, as will be described later, the browsing information is collected by the search device via the browsing management server device, so that the search device can easily grasp the browsing (browsing) history such as specifying the browsing resource of the client terminal. , And so on.

  Next, the functional configuration of the “search device” (0320) that constitutes the search system of this embodiment together with the toolbar device will be described below with reference to FIG. As shown in FIG. 3, the “search device” includes a “browsing management server device” (0330).

  The “browsing management server device” (0330) has a function of receiving and managing browsing information transmitted from the browsing information transmitting unit (0312). The browsing management server device can use the browsing information to analyze, for example, the user's Web browsing behavior, or quickly grasp a site popular with the user from the analysis result. When collecting index information for search by a “crawler” in a search device, the frequency of crawling the popular site is increased so that the latest information on the popular site is provided by the search. Processing can be performed.

  In addition, as will be described later in detail in the second embodiment, for example, a URL that is not managed by itself among browsing URLs included in browsing information is set as a URL of a resource that is not hyperlinked to another page. You may perform the process newly designated as a crawler's circulation destination. Alternatively, as will be described later in detail in the third embodiment, the traffic rank is an effective access index (viewing index) of resources that are tabulated by removing erroneous access from information such as browsing time included in browsing information. May be calculated and used for the “crawler” patrol schedule.

  As described above, the browsing management server device may exist as a server device separate from the search device on the network. In that case, as shown in FIG. 3B, the “search device” may collect browsing information managed by the browsing management server device and perform various processes as described above.

    <Hardware configuration>

  FIG. 4 is a schematic diagram illustrating an example of a configuration in the search system when the above functional components are realized as hardware. The operation of each hardware component in the search system of this embodiment will be described with reference to this figure. As shown in this figure, the hardware configuration of the search system according to the present embodiment includes a “toolbar device” (0410) incorporated in the client terminal and a “search device” (0420) arranged on the network as a server device. ). In this example, a configuration in which a “browsing management server program” for realizing the browsing management server device is executed in the search device, that is, a configuration in which the search device includes the browsing management server device will be described as an example.

  The toolbar device and the search device are connected to each other via an electric communication line to transmit and receive information. The telecommunication line includes the Internet.

  The “toolbar device” is a “CPU (central processing unit)” (0411), “RAM” (0412), and a browsing information transmission unit that realizes a browsing information acquisition unit and performs various other arithmetic processes. “Communication I / F” (0413). Also, “UI” (user interface) (0414) which is an input device such as a keyboard and mouse, “VRAM” (0415) for displaying resources processed by the browser program, and “display device” such as a display. (0416) is also provided. They are connected to each other via a data communication path such as a “system bus” to transmit / receive information and process information.

  In addition, a browser program and a toolbar program are stored on the “RAM” of the toolbar device, and various calculation processes of the “CPU” related to acquisition of browsing information, transmission processing, and other processing are executed in accordance with these programs. . The toolbar program secures an area for storing the browsing URL at a predetermined address on the RAM.

  On the other hand, in the “search device”, “CPU” (0421), “RAM” (0422), “Communication I / F” (0423) for performing various arithmetic processes, and a large amount of browsing information are stored. And a “secondary storage device” (0424) such as a hard disk. They are connected to each other via a data communication path such as a “system bus” to transmit / receive information and process information.

  In addition, a browsing management server program is stored in the “RAM” of the search device, and various calculation processes of the “CPU” related to browsing information management processing and other processing are executed in accordance with the program.

  Here, the “toolbar device” receives an operation input from the user via the “UI”, and the following processing is executed according to the browser program and the toolbar program on the “RAM”. That is, when an instruction to access a resource on the Web is input by a browser operation via “UI”, the URL of the resource designated by the access instruction is stored in a predetermined address of “RAM”. Then, an HTTP request is transmitted to the URL specified according to the browser program. A response to the HTTP request is received as “access resource content” by the “communication I / F”. Subsequently, an HTML file, an image file, and the like included in the received content are subjected to rendering processing by arithmetic processing of a “CPU” in accordance with an HTML rendering engine or the like included in the browser program. Then, the processing result is transferred to “VRAM”, and the Web content is displayed on the “display device”. Then, the user browses resources such as the displayed Web page on the WWW.

  At the same time, the toolbar program executes the following process in response to the access instruction input from the browser. That is, the URL stored at a predetermined address of “RAM” is transmitted as browsing information from the “communication I / F” to the “search device” at each predetermined timing as described above.

  In addition, the toolbar program may acquire a user ID, browsing time information, and the like as follows, and transmit it as browsing information in the same manner as the URL. First, when the toolbar program is activated, a user ID is acquired from an Internet cache or the like and stored in a predetermined address on the “RAM”. Further, when acquisition of the HTTP response is completed by the browser program, the current time is obtained from the OS timer function or the like and stored in a predetermined address on the “RAM”. Alternatively, the time during which the window is active is monitored for one URL in the browser, and the time is stored as a browsing time at a predetermined address on the “RAM”. Then, the information stored in the “RAM” is transmitted from the “communication I / F” to the “search device” at the predetermined timings as described above.

  Then, the “search device” receives the browsing information transmitted from each toolbar device through “communication I / F” and stores it in a predetermined address of “RAM”. Then, according to the browsing management server program, for example, a browsing history database or the like having parameters such as URL and user ID included in the browsing information is generated by calculation processing of “CPU” and stored in “secondary storage device” such as HDD. , And so on.

  Then, using the database, for example, the number of accesses for each browsing URL is totaled by the calculation processing of “CPU”. Then, with reference to the “number of accesses—circulation frequency table” or the like, a process of determining the crawler's circulation frequency in descending order of the counting result is performed.

    <Process flow>

  FIG. 5 is a flowchart showing an example of the flow of processing in the search system of the present embodiment. Note that the steps shown below may be processing steps that constitute a program for controlling a computer recorded on a medium.

  As shown in this figure, when the tool bar is activated in the tool bar device, first, browsing information including a URL is acquired (step S0501). Next, it is determined whether or not the browsing URL has been updated (changed) by the user's browser operation (step S0502). If it is determined that the information has been updated, information including the browsing URL is transmitted as browsing information to a search device including the browsing management server device (step S0503).

  In the tool bar device, after the browsing information is transmitted (step S0503), it is determined whether browsing is completed (step S0504). When it is determined that browsing has been completed, such as when the browser is terminated, the process ends. If it is determined that the process has not ended, the process returns to the process of acquiring browsing information including the URL (step S0501).

  In the search device, the browsing information including the URL transmitted by the toolbar device in step S0503 is received (step S0511) and stored in a recording medium or the like (step S0512). A database is generated and stored using browsing information acquired from each toolbar device, and used for various processes.

  Of course, as described above, in step S0503, the browsing information may be transmitted to a browsing management server device separate from the search device. In that case, a step of collecting browsing information received by the browsing management server device by a search measure may be added.

    <Brief description of effect>

  As described above, it is possible to acquire the user's browsing information by using the toolbar device of the client terminal by the search system of the present embodiment, and transmit the browsing information to the searching device via the browsing management server device. In the search device, it is possible to easily collect browsing information transmitted from a plurality of toolbar devices on the network and construct a database or the like.

  Then, using the constructed database, for example, the number of accesses for each browsing URL can be totaled, and processing for determining the crawler's circulation frequency in descending order of the totaling result can be performed.

  << Example 2 >>

    <Overview>

  FIG. 6 is a conceptual diagram for explaining an example of processing in the search system of the present embodiment. As shown in this figure, a user A accesses a Web site A (0602) newly opened by a friend using a client terminal (0601) equipped with a toolbar device. Here, since the new Web site A has just been newly established, there is no hyperlink from other Web resources. Therefore, the crawler is not a circulation target and does not yet exist in the search service index file (0604) of the search device (0603).

  However, in the search system of the present embodiment, browsing information including the URL is transmitted to the search device when the client terminal accesses the Web site A. Therefore, the search device compares the transmitted URL with the URL included in the search service index file. If it is not included in the index file, the contents are acquired and additionally stored in the index file. In addition to this, the URL is set as a direct circulation target URL other than the hyperlink by “crawler”.

    <Functional configuration>

  FIG. 7 is a diagram illustrating an example of functional blocks in the browsing management server device according to the present embodiment. As shown in this figure, the “search system” (0700) of this embodiment has a “toolbar device” (0710) and a “search device” (0720) based on the configuration of the first embodiment. The “toolbar device” includes a “browsing information acquisition unit” (0711) and a “browsing information transmission unit” (0712) as in the first embodiment. The detailed description including the functional configurations of the toolbar device and the search device has already been described in the first embodiment, and will be omitted.

  The feature of the present embodiment is that the “search device” has a “first browsing management server device” (0730), and the first browsing management server device further has a “first indexer unit” (0731). It is.

  The “first indexer unit” (0731) has a function of extracting a new search target URL based on the collected browsing information and indexing the content of the extracted URL.

  “New search target URL” refers to a URL that is newly used as a search target in the search engine. Specifically, the search apparatus generates and holds an index file by collecting information on resources on the WWW collected by the “crawler”. When a search is performed by a search engine, the index file stored therein is referred to perform a search. That is, URLs that are not included in the index file are not searched. Therefore, the URL included in the browsing information, the URL stored in the index file, and the difference information are acquired, and the URL that is the difference is extracted as a new search target URL.

  Then, the URL of the new search target extracted in this way and the content of the resource at the URL are indexed (added to the index file) by the search device, so that the search system of the present embodiment is new. For example, resources on the WWW that were not originally searched, such as an established Web page, can be searched. Here, the “indexing” process is to specifically reconfigure the search engine into a data structure or a file structure that makes it easy to search for a target keyword at high speed. For example, there is a method of newly assigning an index in which the extracted keyword is used as a key and this is converted into a hash table.

  In addition to the above-described indexing process, the search apparatus according to the present embodiment performs a process of adding the extracted new search target URL as a target that the “crawler” directly circulates without following the hyperlink, and acquires update information. It may be configured.

    <Hardware configuration>

  FIG. 8 is a schematic diagram illustrating an example of a configuration in the search system when the above functional components are realized as hardware. The operation of each hardware component in the search system of this embodiment will be described with reference to this figure. As shown in this figure, the hardware configuration of the search system according to the present embodiment includes a toolbar device (0810) and a search device (0820). Both can be connected to each other via a telecommunication line including the Internet.

  In addition, the “toolbar device” (0810) includes “CPU” (0811), “RAM” (0812), “Communication I / F” (0813), “UI” (0814), “VRAM” as in the first embodiment. ”(0815) and“ display device (display, etc.) ”(0816).

  Similarly to the first embodiment, the “search device” (0820) also includes “CPU” (0821), “RAM” (0822), “communication I / F” (0823), and “secondary storage device” (0824). And having.

  As described in the first embodiment, the browsing information including the browsing URL is transmitted to the search device by the processing of the toolbar program of the plurality of toolbar devices on the network, and is stored in the predetermined address of “RAM”.

  In the search device, the following processing is executed in accordance with the first indexer program included in the first browsing management server program. First, an index file for a search service stored in a “secondary storage device” is referred to in advance, and a search process using a browse URL stored in a predetermined address of “RAM” as a key is performed by an operation of “CPU”. Executed. As a result of the search process, if a determination result indicating that the URL is not included in the index file is output, it is determined that the URL is a new search target URL. Then, according to the first indexer program, the “CPU” accesses the URL stored in the “RAM” via the communication I / F (0913), and outputs a command for acquiring the resource content. For the acquired content, the indexing process is executed by the process of the “CPU” and recorded as new index information in the index file held in the “secondary storage device”.

  Further, in accordance with the first indexer program, a process of incorporating the URL stored in the “RAM” into the schedule as a crawler's circulation target for the next and subsequent times may be executed. Still further, assuming that the URL is a resource with high information freshness, processing may be performed so as to generate a schedule with an increased crawler circulation frequency.

    <Process flow>

  FIG. 9 is a flowchart illustrating an example of a processing flow in the search system according to this embodiment. Note that the steps shown below may be processing steps that constitute a program for controlling a computer recorded on a medium.

  Here, when the tool bar is activated in the tool bar device, first, browsing information including the URL is acquired (step S0901). Next, it is determined whether or not the browsing URL has been updated (changed) by the user's browser operation (step S0902). If it is determined that the information has been updated, information including the browsing URL is transmitted as browsing information to the search device including the first browsing management server device (step S0903).

  Subsequently, the search device receives and collects browsing information including n URLs transmitted from each client terminal on the network (step S0911). Next, the browsing information is stored in a recording medium or the like (step S0912). Next, loop 1 is entered, and the index file (database) of the search device is referred to for each of the n URLs stored to determine whether or not the same URL exists (step S0914).

  As a result of the determination process, for the URL determined not to exist in the index file, the resource content is acquired and indexed as a new search target URL (step S0915). On the other hand, the indexing process is skipped for URLs determined to have the same URL. When the determination process is completed for all URLs indicated by the browsing information, the process ends.

    <Brief description of effect>

  As described above, in the search system of this embodiment, since a hyperlink such as a newly opened web page is not provided, resources on the WWW that have not been searched by conventional crawlers are also used as search index files. Can be indexed.

  Example 3

    <Overview>

  In the present embodiment, browsing information is collected from a toolbar device of each client terminal on the network based on the search system of the first embodiment. The search system is characterized by calculating a “traffic rank”, which is an audience degree index with higher accuracy than the related art, by using the browsing information regarding the degree of viewing of resources on the WWW.

  FIG. 10 is a diagram for explaining an example of processing in the search device according to the present embodiment. As shown in this figure, access to the site A via the site B is executed at the client terminal equipped with the toolbar device. Here, the access time to the site B is short because it is simply via a link. On the other hand, the access time to the site A for the purpose of browsing is long.

  When browsing information including such access (browsing) time is transmitted to the search device, the search device calculates the “traffic rank” using the access time and other information included in the browsing information. As will be described later, this traffic rank is an index corresponding to a browsing entity that eliminates access via the above link and erroneous access, and is therefore more accurate than the conventional site (resource on WWW) audience index. That is, a high index can be calculated.

    <Functional configuration>

  FIG. 11 is a diagram illustrating an example of functional blocks in the search device according to the present embodiment. As shown in this figure, the “search system” (1100) of this embodiment has a “toolbar device” (1110) and a “search device” (1120) based on the configuration of the first embodiment. The “toolbar apparatus” (1110) includes a “browsing information transmission unit” (1111) and a “browsing information acquisition unit” (1112) as in the first embodiment. Since each of the above configurations has already been described in the first embodiment, the description thereof is omitted.

  The feature of this embodiment is that the “search device” (1120) has a “second browsing management server device” (1130), and the second browsing management server device further includes a “browsing information storage unit” (1131). And a “traffic rank scoring section” (1132).

  The “browsing information storage unit” (1131) has a function of storing the collected browsing information. Specifically, it has a function of receiving browsing information acquired by the toolbar device and collected by the search device via the browsing management server device as an input and storing it in a storage medium such as an HDD or a non-volatile memory. Note that the browsing information accumulated here is, for example, browsing (access) time for each resource, user ID (client terminal ID, toolbar device ID), resource information, for the purpose of calculating a traffic rank in a traffic rank scoring unit described later. It is desirable to include transition information.

  The “traffic rank scoring unit” (1132) has a function of calculating the traffic rank for each URL based on the browsing information stored in the browsing information storage unit (1131). “Traffic rank” refers to an audience rating index of resources on the WWW, and examples include a method of calculating as follows.

  That is, as an example of the traffic rank calculation method, for example, a method of calculating the percentage of the total display time of the web page A with respect to the total display time of all the web pages in one day as the rank of the web page. By calculating the rank of the web page in this way, it is possible to calculate the access with a short display time such as a click mistake as a low rank. Therefore, it is possible to calculate the traffic rank of a web page as an index indicating an advertisement effect or the like according to the substance by eliminating click mistakes that are difficult to estimate with a simple number of accesses.

  As the web page rank calculation process, calculation processes based on the following three patterns are mainly conceivable. First, for example, a process of calculating a so-called “national rank” for each web page using the display time included in browsing information from all terminals or a part of the terminals, for example, in units of one day. .

  Second, by using the user attribute information specified by referring to the “user ID-attribute information table” from the user ID included in the browsing information, etc., by age, gender, residence, occupation You may comprise so that the "rank according to attribute" of a web page may be calculated for every segment of others.

  The “user ID-attribute information table” can be constructed as follows as an example. For example, when installing a toolbar device in a client terminal, registration of the user attribute information is essential. Then, the registered attribute information is associated with a user ID such as Cookie and managed by the search device, so that the user (attribute information) transmitted from the toolbar device of the client terminal is specified. .

  Thirdly, the browsing information is classified by user using a user ID, etc., and so-called “rank by user” such as “rank of site A in user α” and “rank of site B in user α”. May be configured to calculate.

  Specifically, as a configuration for performing the third calculation process, the toolbar device and the search device of the search system according to the present embodiment may have the following configurations. For example, this toolbar apparatus is configured to include, for example, user identification information that is information for identifying a cookie or other user in the browsing information transmitted by the browsing information transmission unit, and transmits the information to the search apparatus.

  Then, using the user identification information transmitted here, the “rank by user” is calculated by the following configuration of the search device. That is, the search device is characterized in that the traffic rank scoring unit further includes “user-specific traffic rank calculation means”.

  The “user-specific traffic rank calculation means” has a function of calculating a traffic rank for each user. For example, the traffic rank for each user is calculated for each user using the user identification information when calculating the traffic rank of the web page according to the display time described above.

  Further, when calculating the rank of the web page, the traffic rank may be calculated by eliminating duplicate access to the same resource including fraud by the same user. Specifically, it is determined that the same user has accessed the same page using Cookie included in the browsing information. For the display time determined as such, the traffic rank is calculated using not the total value but the average value.

  In addition, for the traffic ranks aggregated as described above, the rate of increase / decrease (burst degree) from the number of accesses the previous day is calculated, and a calculation formula that reflects the rate of increase / decrease in the rank of the web page is used. Also good. By reflecting such an increase / decrease rate, it is possible to reflect the daily excitement of the web page.

  FIG. 12 is a diagram illustrating an example of the traffic rank for each Web page (resource) calculated as described above. As shown in this figure, the traffic rank is “140”, “120”, “60” for each resource according to a predetermined function using, for example, the number of viewing times and the burst degree (the rate of increase / decrease from the number of accesses on the previous day). For example, “89” is calculated. Then, with respect to the traffic rank calculated in this way, the search device of this embodiment associates it with the URL of the resource and holds it as table data as shown in the figure. Then, using the traffic rank, various processes such as the schedule adjustment of the crawler circulation frequency and the sorting of the search result rank can be executed as will be described later in the fourth and fifth embodiments.

  For example, “top page”, “page 1”, “page 2”, etc. of a certain web site, usually, a single web content is often composed of a plurality of resources. Therefore, the traffic rank calculated by the traffic rank scoring unit does not have to be for each URL. For example, if it is a Web page that constitutes the same Web site, it is judged by comparing the IP address, the domain of the URL, etc., and a single traffic rank is calculated even if the URLs are different. Also good.

    <Hardware configuration>

  FIG. 13 is a schematic diagram illustrating an example of a configuration in the search system when the above functional components are realized as hardware. The operation of each hardware component in the search system of this embodiment will be described with reference to this figure.

  As shown in this figure, the hardware configuration of the search system of the present embodiment is composed of a toolbar device (1310) and a search device (1320), which are connected to each other via a telecommunication line including the Internet. It is possible.

  In addition, the “toolbar device” (1310) includes a “CPU” (1311), a “RAM” (1312), a “communication I / F” (1313), a “UI” (1314), and a “VRAM” as in the first embodiment. ”(1315) and“ display device (display, etc.) ”(1316).

  Similarly to the first embodiment, the “search device” (1320) also includes “CPU” (1321), “RAM” (1322), “communication I / F” (1323), and “secondary storage device” (1324). And having.

  As described in the first embodiment, the browsing URL is stored at a predetermined address in the “RAM” of the toolbar device by the processing of the toolbar program of the toolbar device. Further, in the present embodiment, information such as access time and browsing (access) time for each browsing URL acquired as described below is similarly stored in a predetermined address of “RAM”.

  Specifically, for example, in response to an operation input of “UI”, the browser program obtains the time when the content of the resource (in this case, the Web page) of the URL is received by “Communication I / F” with the built-in clock, and accesses Stored in “RAM” as time. In addition, by monitoring the processing by the browser program, control is performed so that the web page indicated by the URL is controlled to be displayed, for example, in the foreground on the display, or the pointing device is on the web page window. For example, it is measured by a counter or a built-in clock (not shown) and stored in “RAM” as browsing time.

  As shown in FIG. 2, the toolbar apparatus adds a user ID indicated by, for example, a cookie to the browsing URL acquired in this way, the access time to the resource indicated by the URL, or the browsing time in accordance with the toolbar program. Create a table. Then, the information is transmitted as browsing information from the “communication I / F” to the search device.

  In the search device, the browsing information transmitted by the processing according to the toolbar program of the toolbar device is received by the “communication I / F” and stored in a predetermined address of the “RAM”. The browsing information may be recorded and accumulated in the “secondary storage device” as necessary.

When the browsing information is stored in the search device, the following processing is executed according to the traffic rank calculation program. That is, referring to the browsing information stored on the “RAM” or the like, for example, a calculation process using the following function is executed in the “CPU” to calculate the traffic rank.

Here, for example, traffic rank R _a web page a is calculated using a function such as the number 1. t _a is a total value of viewing time of page a by all user IDs. T is the total viewing time of all URLs by all user IDs. B _a is a burst value represented by the following equation (2). The burst value will be described later. N _I is the total viewing ID number. In addition, n _a is the sum of the Views of the page a by all user ID. N is the total number of browsing times of all web pages by all users. R _x is the traffic rank of the related page x of page a.

In this way the number 1, to calculate the percentage of pages a viewing time of the viewing time of all the pages _{(T) (t a) (} t a / T). With this numerical value, an access with a short display time such as a click mistake can be calculated with a low traffic rank. Therefore, as described above, it is possible to eliminate the click mistake that is difficult to estimate with a simple number of accesses and the like, and to calculate the audience rating index of the resource as an index in accordance with the substance.

Furthermore, the burst of B _a URL for a Web page a, for example when determined by a function expression shown in Equation 2. r _at is the number of browsing web pages a today. Moreover, _ray is the number of browsing of the web page a yesterday. That is, the burst degree calculated by Equation 1 is the rate of increase / decrease in the number of today's views based on the number of views yesterday. Therefore, this value is higher when a rapid increase is recognized compared to the normal number of accesses. That is, the increase in the attention level of the page can be expressed by the burst degree and reflected in the traffic rank of the Web page (resource).

  In addition, the following effects can be expected by using the total browsing ID number (NI) as a function variable. That is, for example, in order to increase the traffic rank of a certain site illegally, a user performs multiple accesses to a certain site. However, even in such a case, by using the variable NI, it is possible to calculate a higher traffic rank for multiple accesses by multiple users than for multiple accesses by one user.

Further, the traffic rank (r _x ) of the related pages x1, x2,..., Such as the link destination of the web page a that is the traffic rank calculation target, may be used as the variable of the function.

Further, the traffic rank r _x of the related page used here may be a traffic rank calculated by a weighted average or the like only for a page provided with a hyperlink from a URL to be calculated, for example. Alternatively, a URL that has a probability of transition from a certain URL that is greater than or equal to a certain value may be extracted, and the traffic rank of each URL calculated by a weighted average may be used as the traffic rank of the related URL.

  In addition to the so-called “national rank”, the calculated rank may be “attribute rank” or “individual rank” as described above. Specifically, a cookie or the like included in the received browsing information is stored in the “RAM”. Then, using the cookie as a key, a process for specifying a segment or an individual with respect to the display time indicated by the browsing information from previously stored user registration information or the like is executed by a calculation process of the “CPU”. Then, for example, in the summation processing of the display time of URLa and all URLs, the sum value is calculated for each specified individual or segment, and the ratio of the display time is calculated by the arithmetic processing of “CPU”. .

  Then, the traffic rank calculated for each browsing URL in this way is recorded and held in the “secondary storage device” as a traffic rank database. Then, as will be described later in the fourth and fifth embodiments, the traffic rank database is referred to when adjusting the schedule of the crawler's circulation frequency or sorting the search result rank.

    <Process flow>

  FIG. 14 is a flowchart illustrating an example of a processing flow in the search system according to this embodiment. Note that the steps shown below may be processing steps that constitute a program for controlling a computer recorded on a medium.

  As shown in this figure, when the tool bar is activated in the tool bar device, a browsing URL or the like is first acquired (step S1401). Next, it is determined whether or not the browsing URL has been updated (changed) by the user's browser operation (step S1402). If it is determined that the URL has been updated, the access time of the URL is acquired (step S1403). Then, the information including the acquired browsing URL and access time is transmitted as browsing information to a search device including the browsing management server device (step S1404).

  Next, in the search device, as shown in this figure, browsing information including n URLs and information such as user ID and browsing time associated with the URLs is received from the toolbar device (step S1411). Next, the received browsing information is stored in the main memory or the like (step S1412). Thereafter, the following processing starting from loop 1 (step S1413) is executed. First, the n browsing information accumulated is used as a function variable to calculate a traffic rank (step S1414). Then, the traffic rank calculated for each of the n URLs is recorded as a database in, for example, an HDD. Then, for example, referring to the database, the schedule adjustment of the crawling frequency of the “crawler” is executed, or the ranking of the search results is executed.

    <Brief description of effect>

  In this manner, the search system of the present embodiment can calculate a traffic rank that is an effective resource access index (viewing degree index) that is tabulated while eliminating erroneous access and the like. In addition, if functions such as those described above are used, the Web page is evaluated not only by the number of accesses for each URL of the Web page and the number of unique users, but also by a comprehensive evaluation index including an increase in the degree of attention. The traffic rank that can be calculated can be calculated.

  Example 4

    <Overview>

  The present embodiment is a search system based on the third embodiment, wherein the calculated traffic rank is used as an index of resource popularity and is used to determine the crawler's circulation priority.

  FIG. 15 is a conceptual diagram for explaining an example of processing in the search system of the present embodiment. As shown in this figure, the search device calculates the traffic rank by the configuration and processing described in the third embodiment and holds it as a database. Further, in the search device of the present embodiment, for example, “tour priority” is 12 for the site C of “TR (traffic rank): 60”, while “tour” is the site D of “TR: 20”. The priority is calculated according to the traffic rank, such as “priority: 4”.

  Then, using the priority according to the traffic rank, the search device of this embodiment schedules the crawler's patrol schedule as “3 times / week” for the site C, for example. The site D having a low rank is scheduled as “once / week”.

  That is, as described above, Internet users tend to search and access Web pages that are frequently updated. In other words, there is a high possibility that resources with high access frequency are frequently updated. As described in the third embodiment, the “traffic rank” is an index indicating the number of effective accesses by the user. Therefore, the crawler's cyclic priority can be determined according to the effective number of accesses by the traffic rank.

    <Functional configuration>

  FIG. 16 is a diagram illustrating an example of functional blocks in the search system according to the present embodiment. As shown in this figure, the “search system” (1600) of this embodiment has a “toolbar device” (1610) and a “search device” (1620) based on the third embodiment. The “toolbar device” (1610) includes a “browsing information transmission unit” (1611) and a “browsing information acquisition unit” (1612), as in the third embodiment.

  The “search device” (1620) also has a “second browsing management server device” (1630) based on the third embodiment, and the second browsing management server device is a “browsing information storage unit” (1631). “Traffic rank scoring unit” (1632). Note that each configuration of the toolbar device and the search device has already been described in the first and third embodiments, and thus description thereof is omitted.

  The characteristic point of this embodiment is that the search device further includes a “crawler unit” (1621) and a “schedule determination unit” (1622).

  The “crawler unit” (1621) can be realized by a so-called “crawler” program that accesses each resource and collects content by following hyperlinks between resources on the Internet and circulating on the WWW.

  In addition, this crawler program has a function of collecting contents according to a schedule in which the URL of the resource to be normally visited, the tour start time, and the like are determined in advance. In this embodiment, the following configuration is used to plan the circulation frequency in the scheduling according to the traffic rank.

  The “schedule determining unit” (1622) has a function of determining the crawling schedule of the crawler unit (1621) based on the traffic rank calculated by the traffic rank scoring unit (1632).

  Specifically, for example, the cyclic priority y for executing the crawler program for the URL is calculated by a CPU calculation process using a proportional function y = f (TR) with the traffic rank as a variable. Then, based on the calculated tour priority, for example, “3 times / week (patrol priority 12)”, “1 time / week (patrol priority 4)”, etc., and a schedule including the crawler program tour frequency is determined. , And so on.

  In this way, the crawler's cyclic priority is determined based on the traffic rank indicating the number of effective accesses of the user, and information on resources on the WWW that is considered to have high update frequency and high information freshness is suitable for the search device. Updates can be acquired at the timing.

    <Hardware configuration>

  FIG. 17 is a schematic diagram illustrating an example of a configuration in the search system when the above functional components are realized as hardware. The operation of each hardware component in the search system of this embodiment will be described with reference to this figure.

  As shown in this figure, the hardware configuration of the search system of the present embodiment is composed of a toolbar device (1710) and a search device (1720), which can be connected to each other via a telecommunication line including the Internet. It has become.

  The “toolbar apparatus” (1710) performs the same configuration and processing as the toolbar apparatus of the third embodiment, and thus the description thereof is omitted here. The configuration of the “search device” (1720) is the same as that of the above embodiment, “CPU” (1721), “RAM” (1722), “communication I / F” (1723), and “secondary storage device”. (1724) and execute the following processing.

  In other words, the search device searches the browsing information including the browsing URL and the access time and browsing (access) time for each browsing URL transmitted from the toolbar device by the processing described in the third embodiment. Received by “communication I / F” and stored in a predetermined address of “RAM”.

  Then, the same processing as described in the third embodiment is executed, and the traffic rank for each URL is calculated using the browsing information stored in the “RAM”. Recorded and stored in “storage device”.

  Subsequently, the following processing is executed according to the schedule determination program. In other words, the traffic rank database held in the “secondary storage device” is referred to using the URL for which the crawler's patrol schedule should be set as a key, and the traffic rank value TR of the URL is acquired. Then, for example, calculation processing using a function such as y = f (TR) is executed in the “CPU”, and the cyclic priority y of the crawler program for the URL related to the traffic rank is calculated. Then, with reference to the “cyclic priority-cyclic frequency table” stored in the “secondary storage device”, the cyclic frequency such as “3 times / week” associated with the calculated cyclic priority y is determined. decide.

  The effective date and time of the crawler program is reserved by a task scheduler or the like according to the determined tour frequency such as “3 times / week”, and the resource tour and content acquisition processing by the crawler program is executed at the determined frequency. Condition.

    <Process flow>

  FIG. 18 is a flowchart illustrating an example of a processing flow in the search system according to this embodiment. Note that the steps shown below may be processing steps that constitute a program for controlling a computer recorded on a medium.

  First, the process flow in the toolbar apparatus is the same as the process (S1401 to S1404) described in the third embodiment with reference to FIG.

  Next, in the search device, as shown in this figure, browsing information including n URLs and information such as a user ID and browsing time associated with the URLs is received from the toolbar device (step S1811). Next, the received browsing information is stored in the main memory or the like (step S1812). Thereafter, the following processing starting from loop 1 (step S1813) is executed. First, the n browsing information stored is used as a variable of the above-described function, for example, to calculate a traffic rank (step S1814). Then, the crawler unit crawling schedule for the URL is determined based on the calculated traffic rank (step S1815). Then, the above process is repeated for n URLs (step S1816).

    <Brief description of effect>

  As described above, the crawler's cyclic priority can be determined by the traffic rank indicating the effective number of accesses of the user by the search system of this embodiment. Therefore, it is possible to update and acquire information on resources on the WWW that is considered to have a high update frequency and high information freshness at a suitable timing by the search device.

  Example 5

    <Overview>

  The present embodiment is a search system characterized in that the search results for the search requests from the toolbar device are sorted using the calculated traffic rank based on the third and fourth embodiments.

  FIG. 19 is a conceptual diagram for explaining an example of search processing in the search system of the present embodiment. As shown in this figure, a client terminal accesses a search Web page in the search system, and inputs and transmits a search query. Then, the search device extracts the resource including the search query from the search index file as in the conventional search system. Then, the extracted resources are listed, and a search result screen is generated.

  Here, the search device according to the present embodiment further executes the following processing different from the conventional one. That is, according to the magnitude of the traffic rank value related to the URL (list) indicated by the search result, for example, the site B is sorted to the top of the search result (list), and the site A is sorted second. For example, the search results are rearranged.

  In this manner, the search system according to the present embodiment can return a search result corresponding to the traffic rank indicating the effective number of accesses of the user to the client terminal.

    <Functional configuration>

  FIG. 20 is a diagram illustrating an example of functional blocks in the search system according to the present embodiment. As shown in this figure, the “search system” (2000) of this embodiment has a “toolbar device” (2010) and a “search device” (2020) based on the third embodiment. The “toolbar device” (2010) includes a “browsing information transmission unit” (2011) and a “browsing information acquisition unit” (2012), as in the third embodiment.

  The “search device” (2020) also includes a “second browsing management server device” (2030) based on the third embodiment, and the second browsing management server device includes a “browsing information storage unit” (2031), A “traffic rank scoring unit” (2032). In addition, based on the fourth embodiment, the search device may include a “crawler unit” and a “schedule determination unit” (not shown). Note that the configurations of the toolbar device and the search device have already been described in the above-described embodiments, and thus description thereof will be omitted.

  The feature point of this embodiment is that the search device further includes a “rank sort output unit” (2021).

  The “rank sort output unit” (2021) has a function of sorting search results based on the traffic rank calculated by the traffic rank scoring unit (2032) and outputting the result to the client.

  Specifically, before the sorting by the “rank sort output unit”, the search system of the present embodiment executes the same search process as the conventional search system. That is, the search index file is searched using the search query transmitted from the client terminal as a key. Then, the resource including the search query is extracted from the index file.

  Subsequently, the extracted resources are rearranged and returned to the client as a search result in, for example, an item format (list format). Here, in a normal search system, the search results are rearranged by, for example, the oldest / newest order of data generation or acquisition, the number of accesses, the order of access, or the recursive rating of resources assigned using links, Execute according to etc. However, the search system of the present embodiment is characterized in that a traffic rank indicating the actual number of accesses of the user calculated in the third embodiment is used. That is, the traffic rank of the resource extracted by the search is acquired from the traffic rank database described in the third embodiment. Then, the magnitude comparison of the traffic ranks is executed by the arithmetic processing of the CPU, and for example, the resources are rearranged (sorted) in descending order of the traffic ranks to obtain search results.

  In this way, a search result according to the traffic rank indicating the effective number of accesses of the user, that is, the substantial access popularity can be returned to the client terminal.

    <Hardware configuration>

  FIG. 21 is a schematic diagram illustrating an example of a configuration in the search system when the above functional components are realized as hardware. The operation of each hardware component in the search system of this embodiment will be described with reference to this figure.

  As shown in this figure, the hardware configuration of the search system of this embodiment is composed of a toolbar device (2110) and a search device (2120), both of which can be connected to each other via a telecommunication line including the Internet. It has become.

  The “toolbar device” (2110) performs the same configuration and processing as the toolbar device of the third and fourth embodiments, and thus the description thereof is omitted here. The configuration of the “search device” (2120) is the same as in the above embodiment, “CPU” (2121), “RAM” (2122), “communication I / F” (2123), and “secondary storage device”. (2124) and execute the following processing.

  That is, using the browsing information transmitted from the toolbar device and browsing information including information such as the access time and browsing (access) time for each browsing URL, the traffic rank is calculated by the search device, and the “secondary storage device” Is recorded and retained.

  After that, when the toolbar device on the network accesses the search Web page and inputs and transmits the search query, the search device receives an HTTP request including the search query through “Communication I / F”, Is stored at a predetermined address. Then, the search device executes the following processing according to the search server program. That is, the search query stored in the “secondary storage device” is referenced using the search query stored in the “RAM” as a key, and resources including the search query are extracted.

  Subsequently, referring to the traffic rank database similarly held in the “secondary storage device”, the traffic rank of the extracted resource is acquired. Then, the comparison of the traffic ranks is executed by the comparison calculation processing of “CPU”, and the order of the search results is determined in the order of, for example, the traffic ranks in descending order (ranking is high). Then, a search result in which the extracted resources are arranged in accordance with the determined arrangement order to form an item form is generated by the calculation processing of “CPU”, and is returned from the “communication I / F” to the toolbar device that is the transmission source of the search query. That's it.

    <Process flow>

  FIG. 22 is a flowchart illustrating an example of a processing flow in the search system according to this embodiment. Note that the steps shown below may be processing steps that constitute a program for controlling a computer recorded on a medium.

  First, the processing flow in the client terminal in which the toolbar device is incorporated is the same as the processing (S1401 to S1404) described in the third embodiment with reference to FIG. The processing up to the calculation of the traffic rank (step S2211) in the search device is the same as the processing (S1411 to S1414) described in the third embodiment, and the description thereof is omitted.

  As shown in this figure, when the search device receives a search query transmitted from a client terminal on the network (the toolbar device may not be incorporated) (step S2212), the search device searches using the search query as a key. The index file search process is executed (step S2213). Then, a resource including the search query in the content is extracted and acquired as a search result (step S2214).

  Subsequently, the traffic rank of the extracted resource is acquired (step S2215), and the search result resources are rearranged according to, for example, the order of the traffic rank (step S2216). Then, the sorted search results are returned to the client terminal that has transmitted the search query (step S2217).

    <Brief description of effect>

  As described above, the search system according to the present embodiment can return a search result corresponding to the traffic rank indicating the effective access number of the user, that is, the substantial access popularity, to the client terminal.

  Example 6

    <Overview>

  The present embodiment has a function of extracting a new search target in the same manner as the second embodiment and additionally updating a search index file using the extracted new search target URL. The difference from the second embodiment is that the traffic rank described in the third embodiment is used in extracting the new search target URL.

  Specifically, in the search device, browsing information is acquired by the toolbar device of the client terminal, and the traffic rank is calculated for each URL as described in the above embodiment. Here, as in the second embodiment, it is determined whether the URL is included in the search index file. If the traffic rank is not included in the index file and the traffic rank is equal to or higher than a predetermined value, for example, it is determined that the resource is suitable for newly adding to the index file, and the crawler is newly searched as in the second embodiment. For example, the indexing process is executed.

    <Functional configuration>

  FIG. 23 is a diagram illustrating an example of functional blocks in the search system according to the present embodiment. As shown in this figure, the “search system” (2300) of this embodiment has a “toolbar device” (2310) and a “search device” (2320) based on the third embodiment. The “toolbar device” (2310) includes a “browsing information transmission unit” (2311) and a “browsing information acquisition unit” (2312), as in the third embodiment.

  The “search device” (2320) also has a “second browsing management server device” (2330) based on the third embodiment, and the second browsing management server device is a “browsing information storage unit” (2331). And “traffic rank scoring section” (2332). Further, based on the fourth and fifth embodiments, the search device may include a “crawler unit”, a “schedule determination unit”, and a “rank sort output unit” (not shown). Note that the configurations of the toolbar device and the search device have already been described in the above-described embodiments, and thus description thereof will be omitted.

  A feature point of the present embodiment is that the search device further includes a “second indexer unit” (2321).

  The “second indexer unit” (2321) extracts and extracts a new search target URL that is a URL that is newly used as a search target in the search engine based on the traffic rank calculated by the traffic rank scoring unit. It has a function of indexing URL content.

  Note that the processing for setting the URL related to the calculated traffic rank as the target of a new crawler search is the same as that described in the second embodiment, and thus the description thereof is omitted. Also, in this embodiment, not only the difference between the URL in the index file and the URL related to the traffic rank, but also the traffic rank that is the audience rating index of the resource indicated by the URL is described in the second embodiment. In addition to the processing, the following processing may be executed.

  That is, a comparison / determination process for determining whether or not the traffic rank is equal to or higher than a predetermined value is executed. On the other hand, when it is determined that the traffic rank is equal to or lower than the predetermined value, the traffic rank is not added to the index file as a new crawler search target.

    <Hardware configuration>

  FIG. 24 is a schematic diagram illustrating an example of a configuration in the search system when the above functional components are realized as hardware. The operation of each hardware component in the search system of this embodiment will be described with reference to this figure.

  As shown in this figure, the hardware configuration of the search system of this embodiment is composed of a toolbar device (2410) and a search device (2420), which can be connected to each other via a telecommunication line including the Internet. It has become.

  The “toolbar device” (2410) performs the same configuration and processing as those of the toolbar devices of the third, fourth, and fifth embodiments, and thus description thereof is omitted here. The configuration of the “search device” (2420) is similar to that of the above embodiment, “CPU” (2421), “RAM” (2422), “communication I / F” (2423), and “secondary storage device”. (2424) and execute the following processing.

  First, by using the browsing information including the browsing URL transmitted from the toolbar device and information such as the access time and browsing (access) time for each browsing URL by the same processing as in the above embodiment, the search device uses the browsing information. The traffic rank is calculated and recorded and stored in the “secondary storage device”.

  Subsequently, the index file for the search service stored in the “secondary storage device” is referred to in advance, and a search process using the URL related to the traffic rank held in the “secondary storage device” as a key is executed. . As a result of the search process, if a determination result indicating that the URL is not included in the index file is output, it is determined that the URL is a URL candidate for a new search target.

  Subsequently, according to the second indexer program, the “CPU” stores the traffic rank related to the candidate URL in the “RAM”, and executes a size comparison process with a predetermined value held in the “secondary storage device” in advance. . If it is determined that the traffic rank is greater than or equal to a predetermined value as a result of the size comparison process, the URL is indexed as a new search target URL (additional update process to the index file held in the secondary storage device). Is executed.

    <Process flow>

  FIG. 25 is a flowchart illustrating an example of a processing flow in the search system according to this embodiment. Note that the steps shown below may be processing steps that constitute a program for controlling a computer recorded on a medium.

  First, the process flow in the toolbar apparatus is the same as the process (S1401 to S1404) described in the third embodiment with reference to FIG.

  Next, in the search device, as shown in this figure, browsing information including n URLs and information such as user ID and browsing time associated with the URLs is received from the toolbar device (step S2511). Next, the received browsing information is stored in the main memory or the like (step S2512). Thereafter, the following processing starting from loop 1 (step S2513) is executed. First, the stored n browsing information is used as a function variable to calculate a traffic rank (step S2514).

  Subsequently, a first determination process is performed to determine whether or not the URL exists in the index file of the search device (step S2515). If a determination result indicating that the URL does not exist in the index file is output as a result of the first determination process, a determination process is executed to determine whether or not the traffic rank related to the URL is equal to or greater than a predetermined value (step S2516).

  If the determination result that the traffic rank is equal to or higher than the predetermined value is output as a result of the second determination process, the URL extracted as a resource suitable as a crawler's new patrol search target is used as the crawler's new search target. An indexing process is executed as the URL (step S2517).

    <Brief description of effect>

  In this way, the search system of the present embodiment determines whether the resource is suitable for addition to the index file (indexer), and a hyperlink such as a newly opened Web page is not provided. The indexer can also be executed for resources on the WWW that have not been searched.

Conceptual diagram for explaining an example of browsing information collection in the search system of Embodiment 1 The figure showing an example of the browsing information collected by the search device of the search system of Example 1. The figure showing an example of the functional block in the search system of Example 1. 1 is a diagram illustrating an example of a hardware configuration in a search system according to a first embodiment. 7 is a flowchart illustrating an example of a processing flow in the search system according to the first embodiment. Conceptual diagram for explaining an example of processing in the search system according to the second embodiment. The figure showing an example of the functional block in the search system of Example 2. The figure showing an example of the hardware constitutions in the search system of Example 2. A flowchart showing an example of the flow of processing in the search system of Example 2. Conceptual diagram for explaining an example of processing in the search system according to the third embodiment. The figure showing an example of the functional block in the search system of Example 3. The figure showing an example of the traffic rank for every web page calculated in the traffic rank scoring part of the search system of Example 3. The figure showing an example of the hardware constitutions in the search system of Example 3. A flowchart showing an example of the flow of processing in the search system of Example 3. Conceptual diagram for explaining an example of processing in the search system according to the fourth embodiment. The figure showing an example of the functional block in the search system of Example 4. The figure showing an example of the hardware constitutions in the search system of Example 4. A flowchart showing an example of the flow of processing in the search system of Example 4. Conceptual diagram for explaining an example of processing in the search system according to the fifth embodiment. The figure showing an example of the functional block in the search system of Example 5. The figure showing an example of the hardware constitutions in the search system of Example 5. 10 is a flowchart illustrating an example of a processing flow in the search system according to the fifth embodiment. The figure showing an example of the functional block in the search system of Example 6. The figure showing an example of the hardware constitutions in the search system of Example 6. Flowchart showing an example of a processing flow in the search system according to the sixth embodiment.

Explanation of symbols

0300 Search system 0310 Toolbar device 0311 Browse information acquisition unit 0312 Browse information transmission unit 0320 Search device 0730 First browse management server device 0731 First indexer unit 1130 Second browse management server device 1131 Browse information storage unit 1132 Traffic rank scoring unit

Claims (16)

A browsing information acquisition unit that acquires browsing information including at least a browsing URL from a browser;
A browsing information transmitter that transmits the acquired browsing information to a predetermined browsing management server device;
A toolbar device having
A search device for collecting browsing information transmitted from the browsing information transmitting unit;
Search system consisting of The search device includes:
A first browsing management server apparatus having a first indexer unit that extracts a new search target URL, which is a URL that is newly used as a search target in a search engine, based on the collected browsing information, and indexes the content of the extracted URL The search system according to claim 1, comprising: The search device includes:
A browsing information storage unit for storing the collected browsing information;
The search system according to claim 1, further comprising a second browsing management server device having a traffic rank scoring unit that calculates a traffic rank that is an audience rating index for each URL based on browsing information stored in the browsing information storage unit. . The search device includes:
Crawler part,
A schedule determination unit that determines a crawling schedule of the crawler unit based on the traffic rank calculated by the traffic rank scoring unit;
The search system according to claim 3.   The search system according to claim 3 or 4, wherein the search device includes a rank sort output unit that sorts search results based on the traffic rank calculated by the traffic rank scoring unit and outputs the result to the client. The search device includes:
A second indexer unit that extracts a new search target URL, which is a URL to be newly used as a search target in the search engine, based on the traffic rank calculated by the traffic rank scoring unit and indexes the content of the extracted URL The search system according to any one of claims 3 to 5. A search device that collects browsing information from a plurality of toolbar devices according to claim 1,
A first browsing management server apparatus having a first indexer unit that extracts a new search target URL, which is a URL that is newly used as a search target in a search engine, based on the collected browsing information, and indexes the content of the extracted URL Search device including A search device that collects browsing information from a plurality of toolbar devices according to claim 1,
A browsing information storage unit for storing the collected browsing information;
A search device including a second browsing management server device having a traffic rank scoring unit that calculates a traffic rank that is an audience rating index for each URL based on browsing information stored in a browsing information storage unit.   The search device according to claim 8, further comprising a rank sort output unit that sorts search results based on the traffic rank calculated by the traffic rank scoring unit and outputs the result to the client. Crawler part,
A schedule determination unit that determines a crawling schedule of the crawler unit based on the traffic rank calculated by the traffic rank scoring unit;
The search device according to claim 8 or 9, comprising: A second indexer unit that extracts a new search target URL that is a URL to be newly used as a search target in the search engine based on the traffic rank calculated by the traffic rank scoring unit, and indexes the content of the extracted URL. The search device according to any one of claims 8 to 10. A browsing information collection step for collecting browsing information including a browsing URL;
Extracting a new search target URL that is a URL to be newly used as a search target in the search engine based on the collected browsing information;
A first indexing step for indexing the content of the extracted URL;
A search method that causes a computer to execute. A browsing information collection step for collecting browsing information including a browsing URL;
A browsing information storage step for storing collected browsing information;
A traffic rank scoring step for calculating a traffic rank that is an audience rating index for each URL based on browsing information;
A search method that causes a computer to execute. The search method according to claim 13, further causing the computer to execute a rank sort output step of sorting search results based on the traffic rank calculated in the traffic rank scoring step and outputting the result to the client. Crawler step,
A schedule determination step for determining a crawling schedule for the crawler step based on the traffic rank calculated in the traffic rank scoring step;
The search method according to claim 13 or 14, which causes a computer to execute. Extracting a new search target URL that is a URL to be newly used as a search target in the search engine based on the traffic rank calculated in the traffic rank scoring step;
A second indexing step for indexing the content of the extracted URL;
The search method according to any one of claims 13 to 15, wherein the computer is executed.

Images