JP2000242661A - Relating information retrieval device and storage medium recording program for executing relating information retrieval processing - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a relating information retrieval device capable of dynamically retrieving information relating to provided information at the time of passively receiving the provided information. SOLUTION: This relating information retrieval device which is provided in information provision terminal equipment provided with an information reproduction means 101 for successively reproducing the provided information and is for retrieving relating information relating to the provided information is provided with an information extraction means 111 for receiving reproduction position information for indicating a position where a reproduction processing for the provision information is completed from the information reproduction means 101 and extracting at least a part of the provided information as evaluation object information based on the reproduction position information, a feature evaluation means 112 for receiving the evaluation object information from the information extraction means 111 and evaluating a feature amount for indicating the feature, a retrieval means 113 for retrieving the relating information from an accessible retrieval server based on the feature amount and a relating information output means 114 for outputting a retrieved result obtained by the retrieval means 113 and presenting it to a user.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a related information retrieval apparatus for automatically retrieving information related to information provided by a digital video distribution service or an electronic library service, and a program for executing a related information retrieval process. It relates to a storage medium that has been used.

[0002]

2. Description of the Related Art In a service for providing information via a network, there are the following methods for providing a user with information related to information to be provided. First, in a hypertext system represented by the WWW (World Wide Web), there is a method of setting a link between information to be provided and related information.

FIG. 8 shows a configuration example of an information providing terminal to which a hypertext system is applied. In this information providing terminal device, the information providing processing unit 401 receives, via the network control unit 403, information to be provided (hereinafter referred to as provided information) transmitted to the network by the server 402 on the information provider side, A configuration is provided to the user via an output device such as a display device 405 or a speaker 406 via an input / output control unit 404.

[0004] In the information provision processing unit 401, the reproduction control unit 411 is provided for the processing of the display processing unit 412 and the audio processing unit 413 in accordance with the type of the provision information received via the network control unit 403. ing. In some cases, the reproduction control unit 411 temporarily stores the provided information in the information storage unit 407, reads the provided information from the information storage unit 407, and provides the read information to the display processing unit 412 and the audio processing unit 413.

[0005] The display processing unit 412 shown in FIG. 8 receives text information, still image information, and moving image information included in the provided information, creates display information to be displayed on the display device 405, and controls input / output. The configuration is provided for the processing of the unit 404. On the other hand, the audio information included in the provided information is transmitted to the input / output control unit 40 via the audio processing unit 413.
4 and a sound reproduction operation is performed by the speaker 406.

A link analysis unit 414 analyzes a designated link in response to an instruction from the input reception unit 415, and sends a link destination to the server 402 on the information provider side via the network control unit 403. Is required. In this case, the user enters the keyboard 408
By operating an input device such as a mouse or a mouse 409 to specify an information element to which a link is set, the input receiving unit 4
15 and the operation of the link analysis unit 414 are started to follow the set link, and the server 4 on the information provider side is started.
02 can be provided with related information.

In the digital video distribution service, an information provider prepares a script in advance, and describes a process of reproducing moving image information and a process of providing related information as a time-series procedure. Related information can be provided to the user in synchronization with the reproduction of the video. FIG. 9 shows a configuration example of a conventional information providing terminal device for a digital video distribution service.

[0008] This information providing terminal device is provided with a script analysis unit 416 in place of the link analysis unit 414 shown in FIG.
The script included in the provided information received via the reproduction control unit 411 is analyzed, and the reproduction control unit 411 operates the display processing unit 412 and the audio processing unit 413 in accordance with the interpretation result.
Is controlled.

As described above, when the link between the video information to be provided and the related information is indicated by the sequence of procedures described in the script, the user simply synchronizes with the reproduction of the video information to be provided. And receive relevant information without having to take any special action. In each of the above two methods, related information prepared in advance is provided to the user by a link set by the information provider, and the method uses a static link.

On the other hand, as a technique for dynamically retrieving and providing information related to an arbitrary information element included in information to be provided, Japanese Patent Application Laid-Open No. 9-91307, “Related Information Display Device and Related information display method "has been proposed. FIG. 10 shows a configuration example of an information providing terminal device to which the technique of JP-A-9-91307 is applied.

The information providing terminal device shown in FIG.
Information element extracting unit 4 in place of the link analyzing unit 414 shown in FIG.
21 and a search control unit 422. The information element extraction unit 421 receives the display position specified by the user via the input reception unit 415, and receives information from the display information generated by the display processing unit 412. The information element displayed at this display position is extracted and provided to the processing of the search control unit 422.

[0012] Further, the search control unit 422 performs a search based on the information element extracted by the information element extraction unit 421.
The search server 423 is requested to search for related information, and the obtained related information is displayed by the display processing unit 412 and the voice processing unit 4.
13 is provided.

According to this technique, the user can use the mouse 409
When the gazing point is moved by operating the keyboard or the keyboard 408, related information related to the information element displayed at the designated gazing point is searched from a database or the like provided in the search server 423, and the display device 405 is searched. And the user via the speaker 406.

[0014]

The conventional method of providing related information using a hypertext system and the method of providing related information by using a script are all based on a static link set by an information provider. By
The information provider provides related information prepared in advance.

Therefore, the range of the related information is limited to the link set by the information provider, and as long as the setting of the link is not changed, regardless of the change of the information registered in a general-purpose database or the like, the related information is newly set. Cannot provide additional information. On the other hand, JP-A-9-91
According to the technique described in Japanese Patent Publication No. 307, it is possible to dynamically search for information related to an arbitrary information element included in the information to be provided. It is necessary to specify the information element.

Therefore, automatically supplied information is displayed on the screen, such as when receiving moving picture information by a digital video distribution service or when receiving an information providing service using a push technology. If the information is passively received, the user cannot specify a gazing point and cannot search for new related information.

The present invention relates to a related information search apparatus and a related information search program capable of dynamically searching for information related to information to be provided, even when passively receiving automatically supplied information. It is an object of the present invention to provide a storage medium on which is recorded.

[0018]

FIG. 1 is a block diagram showing the principle of a related information retrieval apparatus according to the present invention. The invention according to claim 1 is provided in an information providing terminal device having information reproducing means 101 for sequentially reproducing provided information provided by an information provider side, and a related information search device for searching related information related to the provided information. , Receiving reproduction position information indicating a position where reproduction of the provided information is completed from the information reproducing means 101, and extracting at least a part of the provided information as evaluation target information based on the reproduction position information Means 111, feature evaluation means 112 which receives evaluation target information from the information extraction means 111, and evaluates a feature amount representing the feature, and a search means 113 which searches related information from an accessible search server based on the feature amount.
And a related information output unit 114 that outputs a search result obtained by the search unit 113 and presents it to the user.

According to the first aspect of the present invention, the information extracting means
Automatically analyzes the features of the provided information by extracting at least a part of the provided information as evaluation target information based on the playback position information received from the information playback unit 101 and providing the extracted information to the processing of the feature evaluation unit 112. And search means 1
13 to search for related information, and to output related information
4 can be provided to the user.

According to a second aspect of the present invention, in the related information retrieval apparatus according to the first aspect, the information extracting means 111 displays the reproduced information spatially continuously and maintains the information for a predetermined period. Range determining means 121 for determining a range having a predetermined relationship with the reproduction position indicated by the reproduction position information as a range of the evaluation target information based on the reproduction position information of the provision information in the information form, Range extracting means 122 for extracting information corresponding to the range of the target information
And a configuration comprising:

According to a second aspect of the present invention, the first range determining means 12
1 and the operation of the range extracting unit 122, the feature evaluation target information can be extracted by paying attention to the spatial continuity when the provided information is reproduced by the information reproducing unit 101. For example, text information reproduced in a predetermined range spatially adjacent to the provided information reproduced at the present time can be provided to the processing of the characteristic evaluation unit 112. , It is possible to search for related information.

According to a third aspect of the present invention, in the related information retrieving apparatus according to the first aspect, the information extracting means 111 is provided based on the reproduction position information of the provided information in the information form which is reproduced continuously in time. A second range determination unit 125 for determining, as the range of the evaluation target information, a range having a predetermined relationship with the point in time at which the provision information of the reproduction position indicated by the reproduction position information is reproduced; It is characterized by comprising a range extracting means 122 for extracting information corresponding to the range.

According to a third aspect of the present invention, the second range determining means 12
5 and the operation of the range extracting means 122, paying attention to the temporal continuity when the provided information is reproduced by the information reproducing means 101, for example, reproducing the information in a predetermined range which is temporally close to the present time. Can be provided to the processing of the feature evaluation unit 112 as feature evaluation target information.
Based on the characteristics of the information currently being provided to users,
It is possible to search for related information.

According to a fourth aspect of the present invention, in the related information retrieval apparatus according to the first aspect, the information extracting unit 111 includes a hint information extracting unit 126 for extracting hint information included as sub-information in the provided information. The search means 113 includes a search server indicated by the hint information in a set of accessible search servers, and sends a search instruction for designating a search operation by designating these search servers. Instructing means 131 and search executing means 13 for executing a search process for a specified search server in response to a search instruction
2 is provided.

The invention according to claim 4 is a method for extracting hint information 1
In accordance with the hint information extracted by 26, the search server indicated by the hint information can be added to the set of accessible search servers by the first search instruction unit 131. Therefore, if the information provider embeds hint information indicating a recommended search server which is considered to provide valid related information in the provided information, the search execution means of the search means 113 on the user side according to the hint information 132, the search processing for the recommended search server is performed, so that valid related information can be reliably searched.

According to a fifth aspect of the present invention, there is provided the related information search device according to the first aspect, wherein a designated information holding unit for holding designated information indicating a search server frequently used by a user.
The search means 113 includes a search server indicated by the designated information in a set of accessible search servers, and sends a search instruction for designating a search operation by designating these search servers. It is characterized by comprising a means 133 and a search executing means 132 for executing a search process for a specified search server in response to a search instruction.

According to a fifth aspect of the present invention, the second search instruction means 13
By the operation of 3, the search server indicated by the designated information held in the designated information holding unit 115 is added to the set of accessible search servers, and the search execution unit 132 searches for related information from these search servers. As a result, it is possible to obtain a search result of related information reflecting the user's preference.

According to a sixth aspect of the present invention, there is provided an information reproducing means for sequentially reproducing information provided by an information provider.
In a storage medium provided with the information providing terminal device having No. 1 and recording a related information search program for searching for related information related to the provided information, the position at which the reproduction process of the provided information from the information reproducing means 101 has been completed is determined. Receiving the reproduction position information shown in the table, and extracting, based on the reproduction position information, at least a part of the provided information as the evaluation target information, and receiving the evaluation target information obtained in the information extraction procedure, and expressing the characteristic. A feature evaluation procedure for evaluating the feature quantity, a search procedure for searching for related information from an accessible search server based on the feature quantity, and a search result obtained in the search procedure being output and presented to a user. And causing the computer to execute the information output procedure.

According to a sixth aspect of the present invention, in the information extracting step, at least a part of the provided information is extracted as information to be evaluated based on the reproduction position information received from the information reproducing step. , The characteristics of the provided information are automatically analyzed, the related information is searched in the search procedure, and the related information can be provided to the user by the related information output procedure.

[0030]

Embodiments of the present invention will be described below in detail with reference to the drawings. FIG. 2 shows an embodiment of an information providing terminal device to which the related information search device of claim 1 is applied. The information providing terminal device shown in FIG. 2 has a feature extraction processing unit 21 instead of the link analysis unit 414 shown in FIG.
0, and a search control unit 220 corresponding to the search unit 112 described in claim 1. The feature extraction processing unit 210 performs feature extraction and evaluation processing described later, and is obtained as an evaluation result. The feature amount is provided to the processing of the search control unit 220.

The search control unit 220 is connected to the network via the network control unit 403, and is connected to the local search server 202 or the network provided in the information providing terminal device based on the received feature amount. It is configured to perform a search process on the connected external document search server 203 and image search server 204.

The related information display control unit 2 shown in FIG.
Numeral 30 corresponds to the related information output means 114 described in claim 1, receives the related information obtained by the above-described search control unit 220, and displays the related information via the display processing unit 412. Is performed.

FIG. 3 shows a detailed configuration of the feature extraction processing unit. In the feature evaluation processing unit 210 illustrated in FIG. 3, the text evaluation unit 211, the still image evaluation unit 212, the moving image evaluation unit 213, and the audio evaluation unit 214 include a reproduction control unit 411.
The reproduction position information indicating the reproduction completion position at which the reproduction processing for the corresponding information of the corresponding form is completed is received from the corresponding form, and the feature evaluation processing described later is performed on at least a part of the provision information of the corresponding form. ing.

In the text evaluation section 211 shown in FIG. 3, the text extraction section 221 fulfills a part of the function of the information extraction means 111 described in claim 1, and the text information received from the reproduction control section 411. For example, text information near the reproduction completion position is extracted from the provided information held as a text information file in the information storage unit 407 based on the reproduction position information related to
22.

The text analysis unit 222 fulfills a part of the function of the feature evaluation unit 112 described in claim 1, performs a morphological analysis of the received text information, and performs a statistical analysis based on the analysis result. The configuration is such that a feature amount representing a property is obtained and used for processing of the search control unit 220. At this time, for example, the text analysis unit 222 may focus on a morpheme that can be an information element among morphemes included in the text information, and obtain the appearance frequency distribution of the morpheme.

On the other hand, in the still image evaluation unit 212, the still image extraction unit 223 performs a part of the function of the information extraction unit 111 described in claim 1, and the reproduction control unit 4
For example, based on the reproduction position information on the still image information received from 11, the still image information in the vicinity of the reproduction completion position is extracted from the provision information held as the still image information file in the information storage unit 407, and the still image analysis unit 224.

The image analysis unit 224 fulfills a part of the function of the feature evaluation unit 112 described in claim 1. For example, the image analysis unit 224 determines the distribution state of each color in the still image information received from the still image extraction unit 223. The search control unit 220 examines the search result as a feature amount representing the feature of the still image and supplies the feature amount to the search control unit 220. Also, the moving image evaluation unit 213
In, the moving image extracting unit 225 performs a part of the function of the information extracting unit 111 described in claim 1, and, for example, based on the reproduction position information on the moving image information received from the reproduction control unit 411, The moving image information in the vicinity of the reproduction completion position is extracted from the provided information held as the moving image information file in the information storage unit 407, and the moving image analysis unit 226
Is provided.

The moving image analysis unit 226 fulfills a part of the function of the feature evaluation unit 112 described in claim 1. For example, by performing a character recognition process on the received moving image information, If a configuration is adopted in which characters represented there are recognized from a pattern representing a caption or a signboard in a video, and text information composed of the obtained characters is provided to the processing of the text analysis unit 222 via the transfer processing unit 216. Good.

The text information obtained in this way is
Since the information elements included in the moving image information to be evaluated are appropriately reflected, the text information is represented by the moving image information by performing the above-described feature evaluation processing by the text analysis unit 222. It is possible to evaluate the feature by paying attention to the meaning conveyed to the user by the video.

Since the feature amount obtained in this manner is suitable for a search process in a normal search server or the like, by providing the obtained feature amount to the process of the search control unit 220, an effective feature amount can be obtained. Related information can be easily searched. On the other hand, in the voice evaluation unit 214, the voice extraction unit 227 is the information extraction unit 111 described in claim 1.
Based on the reproduction position information on the audio information received from the reproduction control unit 411, for example, the vicinity of the reproduction completion position is determined from the provided information held as the audio information file in the information storage unit 407. Is extracted and provided to the processing of the voice analysis unit 228.

The voice analysis unit 228 fulfills a part of the function of the feature evaluation means 112 described in claim 1 and performs voice recognition processing on the received voice information.
The configuration may be such that the text information is converted into text information corresponding to words expressed by voice, and the obtained text information is provided to the processing of the text analysis unit 222 via the transfer processing unit 216.

The text information obtained in this way is
Since the information elements included in the audio information to be evaluated are appropriately reflected, the text information is subjected to the above-described feature evaluation processing by the text analysis unit 222, so that the text represented by the audio information It is possible to evaluate the characteristics by paying attention to the conveyed meaning.

Since the feature amount obtained in this way is suitable for a search process in a normal search server or the like, by providing the obtained feature amount to the process of the search control unit 220, an effective feature amount can be obtained. Related information can be easily searched. Here, the information to be provided may have a configuration in which various types of information are combined.

For example, the hypertext information includes still image information and moving image information as well as text information. When such hypertext is received as provided information, the reproduction control unit 411 causes the reproduction control unit 411 to perform processing on the text information. Along with the reproduction position information, reproduction position information regarding the still image information and the moving image information is sent to the text evaluation unit 211, the still image evaluation unit 212, and the moving image evaluation unit 213, respectively, and these units perform a characteristic evaluation process.

Similarly, when video information such as a movie or news is the information to be provided, the reproduction control unit 411 sends reproduction position information relating to moving image information and audio information, and the moving image evaluation unit 213. Then, a feature evaluation process is performed by the voice evaluation unit 214. Further, in the case of video information to which text information called closed caption is attached, in addition to the feature evaluation processing on moving image information and audio information, the reproduction control unit 411 sends reproduction position information on the text information and outputs the text information. The characteristic evaluation processing is also performed by the evaluation unit 211.

As described above, when the feature amount obtained by the text analysis unit 222 is received corresponding to one of the partial information constituting the provided information, the search control unit 220 A search process is performed on the server 202 or an external document search server 203,
For example, the related information display control unit 2 searches for text information indicating detailed information on a morpheme having a similar appearance frequency distribution or a morpheme having a high appearance frequency as related information.
30.

In response, the related information display controller 230
Instructs the display processing unit 412 to open, for example, a window different from the window in which the provided information is displayed, and displays the relevant information in the new window via the display processing unit 412. I just need. On the other hand, when the feature amount obtained by the still image analysis unit 224 is received corresponding to any of the partial information constituting the provided information, the search control unit 220 transmits the external image via the network control unit 403. Perform search processing for the search server 204,
What is necessary is just to search for an image similar to the received feature amount as related information and to provide the related information display control unit 230 with the processing.

In response, the related information display control unit 230
As in the case of the text information described above, a new window may be opened via the display processing unit 412, and the image information obtained as the related information may be displayed.

As described above, when a user is passively receiving information through a movie or news video distribution service or an information providing service to which the push technology is applied, the information is currently provided to the user. It is possible to automatically and dynamically search for information related to the existing information, and to provide the obtained related information to the user together with information provided by the service being used.

In this case, since the related information relating to the information currently provided to the user is searched from the databases available at each search server, the latest information registered in the database can be obtained. . Also, the functions of the above-described feature extraction processing unit 210, search control unit 220, and related information display control unit 230 can be realized by software, and a program that performs the functions of these units is stored in a floppy (registered trademark) disk or the like. CD-
It is also possible to record it on a storage medium such as a ROM and distribute it.

The program that performs the function of the feature extraction processing unit 210 corresponds to the program that executes the information extraction procedure and the feature evaluation procedure described in claim 6.
The search control unit 220 and the related information display control unit 23
The programs that perform the respective functions correspond to the programs that respectively execute the search procedure and the related information output procedure described in claim 6.

As described above, the present invention is based on the premise that the user is paying attention to the information currently being reproduced by the output device such as the display device or the speaker, and the display processing unit 412 and the audio processing The information in the vicinity of the reproduction position by the unit 413 is provided to the feature extraction process as information currently provided to the user. At this time, if information that is semantically collected can be extracted as information near the reproduction position, the feature amount obtained by the feature evaluation process is an index that faithfully indicates the feature of the information that the user is paying attention to. Therefore, effective related information can be searched efficiently.

For example, in text information, information on at least one topic is connected to each other with a certain extent, and portions describing related topics are often connected continuously. Therefore, the text extracting unit 221 shown in FIG. 3 obtains, for example, a range of text information corresponding to the page being displayed on the display device based on the reproduction position information received from the reproduction control unit 411, and Information storage unit 40
7 from the text information file stored in
What is necessary is just to provide for the process of the text analysis part 222.

As described above, the text extracting unit 211 operates in accordance with the reproduction position information from the display processing unit 412, so that the functions of the first range determining unit 121 and the range extracting unit 122 described in claim 2 are achieved. Focusing on the spatial continuity of the display position of the provided information, the text analysis unit 222 extracts text information near the reproduction completion position as a feature evaluation target, and performs a function of the feature evaluation unit 112 on the text information. Can be provided.

The text information extracted in this way is
Since it can be expected to be a set of text information describing one topic or a plurality of related topics, the text information is subjected to the processing of the text analysis unit 222 as a feature evaluation processing target.
It is possible to obtain a feature amount that faithfully indicates the feature of the information that the user is currently paying attention to.

In this case, the reproduction control unit 411 indicates the end of the reproduced text information, for example, every time the reproduction processing of the text information of n lines which can be displayed on one screen of the display device is completed. The playback position information may be sent to the text extraction unit 221. In response to this, the text extracting unit 221 reads the above-described n lines of text information from the information storage unit 407, going back from the text information position indicated by the reproduction position information, and sends the text analysis unit 222 What is necessary is just to provide to the process of the search control unit 220 via the.

In this manner, valid related information related to the text information being displayed can be searched at any time according to the progress of the reproduction of the text information. On the other hand, text information is composed of at least one “paragraph”, and some text information has a structure separated by “chapter” or “section”. Next, a description will be given of a method of extracting text information in an appropriate range as a target of feature evaluation processing using such a structure of text information.

In this case, as shown in FIG. 3, a text extracting section 221 is provided with a structural information retrieving section 241 and a structural text reading section 242, and the structural information extracting means 123 described later in the additional disclosure item (1). Is searched for structural information from the text information held in the information storage unit 407, and according to the search result,
The structure text reading unit 242 reads the range of the text information delimited by the structure information from the above-described text information, and supplies the range to the processing of the text analysis unit 222.

At this time, the structure information search unit 241 searches for structure information before and after the reproduction completion position indicated by the reproduction position information received from the reproduction control unit 411, for example. An information storage unit 407 based on the structure information obtained by the information search unit 241
Reads out text information including the above-mentioned reproduction completion position and divided by the searched structural information from the text information held in, and supplies the read text information to the text analysis unit 222 as text information to be subjected to feature evaluation processing. It has become.

As described above, the structure information retrieval unit 241 and the structure text reading unit 242 operate based on the reproduction position information, and thereby the functions of the structure analysis unit 124 and the range extraction unit 122 described in the additional disclosure item (1) later. Is realized, and sections of text information having a predetermined structure, such as paragraphs, chapters, and sections, are extracted, and semantically organized text information can be reliably targeted for feature evaluation processing.

As a result, it is possible to provide a feature amount faithfully indicating a feature of information which is presumed to be of interest to the user at the present time to the processing of the search control unit 220, thereby improving the search accuracy of the related information. Thus, more effective related information can be provided to the user. On the other hand, information that is continuously reproduced, such as a movie or a news video, has a configuration in which information for reproducing scenes each having a certain length is linked, and further, continuous scenes are related to each other. It is thought that there are many cases.

Therefore, as for the information to be continuously reproduced, information that is temporally close to the currently provided information indicated by the reproduction position information may be extracted. FIG. 4 shows a configuration of a main part of a related information search apparatus to which the invention of claim 3 is applied. In the display processing unit 412 shown in FIG. 4, the moving image decoder 243 receives the encoded moving image information received as a stream by the network control unit 403 via the reproduction control unit 411, and decodes the information. The displayed image information is used for display processing by the display device via the input / output control unit 404.

The moving image evaluation unit 213 shown in FIG.
Is provided with a buffer memory 244 that fulfills the function of the moving image extracting unit 225, the buffer memory 244 holds the decoding processing result by the moving image decoder 243 described above, and provides the processing result to the moving image analyzing unit 226. Has become. In this case, if the buffer memory 244 is configured using a FIFO having a capacity corresponding to the moving image information reproduced in the predetermined period T1, the second range determination described in claim 3 is performed by the buffer memory 244. The functions of the unit 125 and the range extracting unit 122 are realized for the moving image information, and the moving image reproduced from the present time to the above-mentioned predetermined period T1 can be extracted as the information of the feature evaluation target.

As described above, a series of moving image information extracted in this manner constitutes the same scene or related scene as the moving image information currently displayed on the display device and provided to the user. It is highly probable that the image of each frame displayed by this series of moving image information can be expected to include the same or related character pattern.

For example, since a news video or a subtitle of a movie is displayed over a series of frames, if moving image information corresponding to these frames is extracted and provided to the processing of the moving image analyzing unit 226, From the sequence of character patterns common to these frames, text information indicating the content described in the caption can be obtained. Since this text information is closely related to the moving image currently provided to the user, the text information is provided to the processing of the search control unit 220 via the text analysis unit 222, so that the effective related information can be obtained. It is possible to search for information.

On the other hand, when the network control unit 403 receives encoded audio information (hereinafter referred to as “encoded audio information”) as a stream, the encoded audio information is transmitted to the audio through the reproduction control unit 411. The decoding result sent to the processing unit 413 and obtained by the decoding processing by the audio processing unit 413 is supplied to an output process by a speaker via the input / output control unit 404.

In this case, as shown in FIG. 4, the buffer memory 24 is assumed to perform the function of the sound extracting unit 227.
6, the buffer evaluation unit 214 is provided, and the buffer memory 246 receives the decoding result of the above-described audio processing unit 413.
Forty-six pieces of voice information can be provided to the processing of the voice analysis unit 228 as evaluation target information.

In this case, if the buffer memory 246 is formed by using a FIFO having a capacity corresponding to the audio information reproduced during the predetermined period T2, the buffer memory 246 allows the second buffer described in claim 3 to be used. Range determining means 1
25 and the functions of the range extracting means 122 are realized for the audio information, and the audio information reproduced during the period from the present time to the above-mentioned predetermined period T2 can be extracted as the information to be evaluated.

As described above, the series of audio information extracted as described above can constitute the same scene or the related scene as the audio currently output from the speaker and provided to the user. It is highly probable that words expressed by this series of audio information can be expected to represent semantically coherent text.

For example, an announcement of a news video or a narration of a movie describes one matter over a certain period of time, so that audio information reproduced during an appropriate period is extracted and processed by the audio analysis unit 228. , It is possible to obtain text information indicating the contents described in the announcement or narration. Since this text information indicates the meaning of the voice information currently provided to the user, by providing this text information to the processing of the search control unit 220 via the text analysis unit 222, an effective related information can be obtained. It is possible to search for information.

In this case, since the decoding result obtained at the time of reproducing the provided information is used as it is, the efficiency of resource utilization is high, and it is possible to cope with the case where the information is provided as a stream via a network. . Of course, the configuration shown in FIG. 4 can also be applied to a case where the provided information is temporarily stored in the information storage unit 407 as a file.

On the other hand, when the provided information is stored as a file in the information storage unit 407, the provided information in the form of continuous playback is temporally near the playback position at the present time. It is also possible to adopt a range that spans the future. FIG. 5 shows another configuration example of the related information search device to which the invention of claim 3 is applied.

The moving image evaluation unit 213 shown in FIG.
7 and a moving image reading unit 248. The moving image reading unit 248, based on the reproduction position information received from the reproduction control unit 411, outputs a code corresponding to a predetermined range described later from the information storage unit 407. Read out the converted moving image information, and read the moving image analyzing unit 2 via the moving image decoder 247.
26.

The moving image reading unit 248 stores, for example, the past n times around the reproduction completion position indicated by the reproduction position information.
The coded information corresponding to the moving picture information reproduced in the second and the coded information corresponding to the moving picture information to be reproduced in the next m seconds are read out and provided to the processing of the moving picture decoder 247. . In this case, the moving image reading unit 248 and the moving image decoder 247 operate according to the reproduction position information, so that the second range determining unit 12 described in claim 3 is operated.
5 and the function of the range extracting means 122 are realized for the moving image information, and the moving image information including the reproduction completion position and extending over the past and the future can be extracted as the evaluation target information.

Further, the voice evaluation unit 214 shown in FIG.
The audio reading unit 249 includes an audio reading unit 249 and an audio decoder 250. The audio reading unit 249 is configured to store information based on the reproduction position information received from the reproduction control unit 411.
7, audio information corresponding to a predetermined range described later is read and provided to the audio decoder 250 for processing.

The audio reading section 249 stores, for example, the encoded audio information corresponding to the audio reproduced for the past n seconds and the next m with the reproduction completion position indicated by the reproduction position information as the center.
What is necessary is just to read out the coded audio information corresponding to the audio to be reproduced in the second, and to provide it to the processing of the audio decoder 250. Further, the audio decoder 250 shown in FIG.
In the same manner as the audio processing unit 413 shown in FIG.
The coded audio information received from 49 is decoded, and the obtained audio information is provided to the processing of the audio analysis unit 228.

In this case, the function of the second range determining means 125 and the range extracting means 122 described in claim 3 is performed by operating the audio reading section 249 and the audio decoder 250 in accordance with the reproduction position information. And audio information in a range including the reproduction completion position and extending between the past and the future can be extracted as the evaluation target information. As described above, when the moving image information and the sound information before and after the reproduction completion position are set as the evaluation target information, the feature amount that more faithfully represents the feature of the moving image or the sound currently being provided to the user is determined. It is possible to ask.

By the way, the feature evaluation process for the provided information and the related information search process based on the obtained feature amount include:
A considerable processing time is required for each, and the reproduction processing of the provided information is continued even during the time required for these processings. For this reason, a time lag occurs between the reproduction of the provided information and the display of the related information, and when the related information obtained as the search result is displayed by the related information display control unit 230, the search processing is performed. There is a possibility that the user is paying attention to information completely different from the information that has become.

Next, a method of providing relevant information to a user at an appropriate timing in consideration of the above-described time lag will be described. FIG. 6 shows another configuration of the main part of the related information search device to which the invention of claim 3 is applied. In the moving image evaluation unit 213 shown in FIG. 6, the moving image reproduction position correction unit 251 estimates an expected reproduction completion position at the provision timing of the related information based on the reproduction position information received from the reproduction control unit 411. The playback position information indicating the expected playback completion position is provided to the processing of the moving image reading unit 248.

The moving image reproduction position correcting section 251 calculates the average processing time T2 based on, for example, the average value of the time required for the characteristic evaluation processing of the moving image information and the average value of the time required for the search processing of the related information. In advance, during the average processing time T2, an expected reproduction completion position may be obtained in consideration of the amount of moving image information reproduced by the moving image decoder 243.

The average value of the time required for the characteristic evaluation processing of the moving image information and the average value of the time required for the search processing of the related information may be obtained in advance by experiments or the like.
As described above, the moving image reproduction position correcting unit 251 and the moving image reading unit 248 operate according to the reproduction position information from the reproduction control unit 411, so that the second range determination unit described later in the additional disclosure item (2). 125 and range extracting means 122
Is realized, and the moving image information corresponding to the temporal vicinity of the expected reproduction position can be extracted and provided to the processing of the moving image analysis unit 226.

In the sound evaluation section 214 shown in FIG. 6, the sound reproduction position correction section 252 includes a reproduction control section 411.
Of the predicted reproduction completion position at the provision timing of the related information, based on the reproduction position information received from the
9 is provided. The audio reproduction position correcting unit 252 obtains an average processing time T3 based on, for example, an average value of a time required for a characteristic evaluation process of audio information and an average value of a time required for a search process of related information. During the processing time T3, the expected reproduction completion position may be obtained in consideration of the amount of audio information to be reproduced by the audio processing unit 413.

The average value of the time required for the process of evaluating the characteristics of audio information and the average value of the time required for the process of searching for related information may be determined in advance by experiments or the like. Thus, according to the reproduction position information from the reproduction control unit 411, the audio reproduction position correction unit 252 and the audio read unit 249
Operates to realize the functions of the second range determining means 125 and the range extracting means 122 described later in the additional disclosure item (2), and extract audio information corresponding to a temporal vicinity of the predicted reproduction position, and perform audio analysis. The processing of the unit 228 can be provided.

In this case, the moving picture decoder 247 includes the moving picture coded information and the voice coded information in a predetermined range including the expected reproduction completion position as the information to be evaluated.
And a video analysis unit 226 via the audio decoder 250
And the processing of the voice analysis unit 228. When the feature evaluation process and the related information search process are completed for the moving image information and audio information thus extracted,
It can be expected that the display processing unit 412 and the audio processing unit 413 are reproducing the moving image and the audio corresponding to the respective predicted reproduction completion positions.

Therefore, if the obtained related information is subjected to the processing of the related information display control unit 230 (FIG. 2), the related information is provided to the user in synchronization with the moving image and the sound being reproduced at that time. It can be provided with good timing. By the way, the search control unit 22 provided in the related information search device
0 (FIG. 2) can access a wide variety of search servers via the network.

However, it is not realistic to search for related information from all search servers, and it is not always possible to reliably search for valid related information from any search server. On the other hand, if the user prefers a specific search server, the related information obtained from the specific search server is the relevant information that the user considers to be valid, and the information provider side determines the information to be provided. If the information related to the information has information on a specific search server in which the information is registered, the search information obtained from the specific search server is the relevant information that the information provider considers effective.

Next, a description will be given of a method of searching for and providing valid related information using such user's preference and knowledge of the information provider. Here, the information to be provided is
HTML (Hypertext Markup Language) or XML (Ext
When the information is described by a tagged document such as an ensible markup language, hint information indicating a search server recommended by the information provider can be embedded as tag information.

If hint information indicating a search server recommended by the information provider is embedded as sub-information included in the MPEG-2 format, the hint information described above can be synchronized with moving images and audio. Can be passed to the information providing terminal device on the user side. FIG. 7 shows an embodiment of a related information search apparatus to which the inventions of claims 4 and 5 are applied.

The text evaluation unit 211 shown in FIG.
In addition to the structure information retrieval unit 241 and the structure text reading unit 242 shown in FIG. 5, a tag information analysis unit 253 corresponding to the hint information extraction unit 126 described in claim 4 is provided. The hint information is extracted from the tag information included in the text information read from the information storage unit 407 (FIG. 6) by the reading unit 242, and is provided to the processing of the search control unit 220.

The moving image evaluation unit 213 includes the hint information extracting unit 126 described in claim 4 in addition to the moving image reproduction position correcting unit 251, the moving image reading unit 248, and the moving image decoder 247 shown in FIG. Information analysis unit 25 corresponding to
The sub-information analysis unit 254 receives the sub-information restored by the video decoder 247, extracts the hint information from the sub-information, and supplies the hint information to the processing of the search control unit 220.

On the other hand, in the search control section 220 shown in FIG. 7, the server selecting section 255 includes the hint information received from the tag information analyzing section 253 or the sub information analyzing section 254 and the designation information described in claim 5. A search server that executes a search for related information is selected based on the designated information held in the user information holding unit 256 corresponding to the holding unit 115, and a search corresponding to the search execution unit 132 described in claim 4 is performed. It is configured to instruct the execution unit 257.

As described above, the server selection unit 255 selects the search server based on the hint information and the designation information. Second search instruction means 13 described
The third function is realized, and related information can be searched from the search server specified by the hint information and the search server specified by the specified information.

In this case, if the user sets designation information indicating a favorite search server in the user information holding unit 256, the server selection unit 255 described above together with the search server designated by the hint information. The user can select a search server designated by the user and instruct the search execution unit 257 to perform a search process on these search servers. In this way, by preferentially searching for the related information from the search server that the user prefers to use or the search server recommended by the information provider, the related information or the information provider that the user considers to be effective is effective. It is possible to reliably search for related information considered to be.

In the search control unit 220 shown in FIG. 7, the weight setting unit 258 corresponds to the weight setting unit 134 described later in the additional disclosure item (3). Based on the search execution unit 25
7 for each search server that performed the search process,
An appropriate weight indicating the importance is set and provided to the processing of the importance adjustment unit 259 corresponding to the importance calculation means 135.

The importance adjusting section 259 corresponds to the importance calculating means 135 described in the additional disclosure item (3), and the similarity added to the search result obtained by the search processing by the search execution section 257. Based on the degrees and the weights described above,
The degree of importance of each of the obtained related information is obtained, and the importance is displayed together with the related information on the related information display control unit 230.
Is provided.

As described above, by using the weight given by the weight setting unit 258 to obtain the importance of the related information obtained from each search server by the importance adjustment unit 259, the recommended search server and the user The importance of the related information obtained from the favorite search server can be adjusted.

The related information display control unit 2 shown in FIG.
At 30, the sort processing unit 261
The display data creation unit 262 sorts the relevant information received from 0 on the basis of the importance described above, and displays the display data for arranging and displaying the relevant information of the search result in the order according to the processing result by the sort processing unit 261. Is created and provided to the processing of the display processing unit 412.

Here, the similarity added to the related information obtained from each search server is an index indicating the importance of the related information obtained in each search server. Is given by
It shows the importance of each search server. Therefore, the importance obtained by the importance adjustment unit 259 described above is an index reflecting both the importance of the search result obtained by each search server and the importance of each search server. Accordingly, by displaying the related information obtained from each search server, an index appropriately indicating the validity of the related information can be presented to the user.

[0099]

As described above, according to the first and sixth aspects of the present invention, the reproduction is completed based on the reproduction position information which is updated as needed in accordance with the progress of the reproduction processing of the provided information. The information provided near the position is automatically extracted as the information to be evaluated, and the related information can be searched based on the characteristics of the information to be evaluated. Also, it is possible to dynamically search and provide related information according to the characteristics of the provided information reproduced at each time.

According to a second aspect of the present invention, by utilizing the fact that the characteristics of information to be reproduced have spatial continuity, a spatial neighborhood of a reproduction completion position is extracted as information to be evaluated. It is possible to search for related information that faithfully reflects the characteristics of the information currently provided to the user, and provide the related information to the user.

According to a third aspect of the present invention, by utilizing the fact that the characteristics of information to be reproduced have temporal continuity, a temporal neighborhood of a reproduction completion position is extracted as information to be evaluated. It is possible to search for related information that faithfully reflects the characteristics of the information currently provided to the user, and provide the related information to the user. According to the fourth and fifth aspects of the present invention, it is possible to select a search server that performs search processing of related information in accordance with hint information given by an information provider and designated information set by a user. Therefore, it is possible to obtain related information that reflects the knowledge of the information provider provided by the information provider and the preference of the user.

With respect to the above description, the following items are further disclosed. 3. The related information search device according to claim 2, wherein the first range determination unit 121 extracts structure information indicating a range having a semantic unit added to the provided information. Based on the structure information obtained by the structure information extracting means 123 and the structure information extracting means 123, a range having a meaningful unit including the reproduction position indicated by the reproduction position information is obtained, and the structure analyzing means 124 is provided to the processing of the range extracting means 122 A related information search device characterized by having a configuration including:

The related information search apparatus operates the structure information extracting means 123 and the structure analyzing means 124 to pay attention to the structure information included in the provided information, and to provide the provided information in a semantically integrated range as a feature evaluation target. Since the information can be provided to the processing of the characteristic evaluation unit 112 as information, it is possible to accurately evaluate the characteristics of the information currently being provided to the user.

[0104] In the related information search device according to the third aspect of the present invention, the second range determining means 125 reproduces the information after a predetermined time has elapsed from the present time based on the received reproduction position information. A related information search apparatus characterized in that a predetermined range is provided as a range of the evaluation target information to be processed by the range extracting means 122.

In the related information search device, the second range determining means 125 designates the range of the provided information to be reproduced in the future after a predetermined time has elapsed from the present time as the range of the feature evaluation target information. Before the provided information in this range is reproduced, the characteristic evaluation process and the related information search process can be performed. Therefore, by appropriately setting the above-mentioned predetermined time, the provided information extracted as the information for feature evaluation can be provided to the information reproducing unit 10.
1, the related information corresponding to the provided information can be presented to the user in synchronization with the timing of the reproduction by the user.

(3) In the related information search device according to claim 4 or claim 5, the search means 113 comprises:
Weight setting means 134 for setting an appropriate weight for each search server specified by the search instruction; similarity information added to the search result from each search server obtained by the search execution means 132; Calculates the importance of search results based on the weight set for,
The related information output unit 114 is configured to discard the search results to be output in accordance with the importance corresponding to each search result. A related information search device characterized in that the selection and output order are determined.

The related information search device is provided with a weight setting means 1
34 and the operation of the importance calculation means 135, the importance of each piece of related information obtained can be adjusted according to the weight set for each search server, and based on the adjusted importance, By operating the related information output unit 114, it is possible to provide an index for the user to select valid related information together with the related information obtained as a search result. Can be provided to users.

[Brief description of the drawings]

FIG. 1 is a principle block diagram of a related information search device of the present invention.

FIG. 2 is a diagram showing a configuration of an information providing terminal device to which the related information search device of claim 1 is applied.

FIG. 3 is a diagram illustrating a detailed configuration of a feature extraction processing unit.

FIG. 4 is a diagram showing a configuration of a main part of a related information search device to which the invention of claim 3 is applied.

FIG. 5 is a diagram showing another configuration example of a main part of a related information search device to which the invention of claim 3 is applied.

FIG. 6 is a diagram showing another configuration example of a main part of a related information search device to which the invention of claim 3 is applied.

FIG. 7 is a diagram showing a configuration of a main part of a related information search device to which the inventions of claims 4 and 5 are applied.

FIG. 8 is a diagram illustrating a configuration example of a conventional information providing terminal device using a hypertext system.

FIG. 9 is a diagram illustrating a configuration example of a conventional digital video distribution service information providing terminal device.

FIG. 10 is a diagram showing a configuration example of an information providing terminal device to which the technique described in Japanese Patent Application Laid-Open No. 9-91307 is applied.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 101 Information reproduction means 111 Information extraction means 112 Feature evaluation means 113 Search means 114 Related information output means 115 Designated information holding means 121 First range determination means 122 Range extraction means 123 Structural information extraction means 124 Structural analysis means 125 Second range determination means 126 hint information extraction means 131 first search instruction means 132 search execution means 133 second search instruction means 134 weight setting means 135 importance calculation means 202 local search server 203 document search server 204 image search server 210 feature extraction processing unit 211 text evaluation Unit 212 still image evaluation unit 213 moving image evaluation unit 214 audio evaluation unit 216 transfer processing unit 220 search control unit 221 text extraction unit 222 text analysis unit 223 still image extraction unit 224 still image analysis unit 225 moving image extraction 226 Video analysis unit 227 Voice extraction unit 228 Voice analysis unit 230 Related information display control unit 241 Structure information search unit 242 Structure text read unit 243, 247 Video decoder 244, 246 Buffer memory 248 Video read unit 249 Voice read unit 250 Audio decoder 251 Moving image reproduction position correction unit 252 Audio reproduction position correction unit 253 Tag information analysis unit 254 Sub-information analysis unit 255 Server selection unit 256 User information holding unit 257 Search execution unit 258 Weight setting unit 259 Importance adjustment unit 401 Information Provision processing unit 402 Information provider server 403 Network control unit 404 Input / output control unit 405 Display device (CRT) 406 Speaker 407 Information storage unit 408 Keyboard 409 Mouse 411 Reproduction control unit 412 Display processing unit 413 Audio Processing section 414 link analysis unit 415 input receiving unit 416 the script analysis unit 421 information element extractor 422 local search server

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) G06F 15/419 320 F term (Reference) 5B075 KK07 KK34 KK40 ND02 ND16 NK06 NK08 NK37 NK39 PP10 PP28 PQ02 PQ32 PQ42 PR08 QM05 UU40

Claims (6)

[Claims] 1. A related information search device provided in an information providing terminal device having information reproducing means for sequentially reproducing provided information provided by an information provider side, wherein the related information searching device searches for related information related to the provided information. Information extraction for receiving, from the information reproducing means, reproduction position information indicating a position at which reproduction of the provided information has been completed, and extracting at least a part of the provided information as evaluation target information based on the reproduced position information. Means, receiving evaluation target information from the information extracting means, a characteristic evaluation means for evaluating a characteristic amount representing the characteristic, a search means for searching related information from an accessible search server based on the characteristic amount, Related information output means for outputting a search result obtained by the search means and presenting the search result to a user. Search equipment. 2. The related information search device according to claim 1, wherein the information extracting means displays the reproduced information spatially continuously and provides the information in an information form maintained for a predetermined period. First range determining means for determining, as the range of the evaluation target information, a range having a predetermined relationship with the reproduction position indicated by the reproduction position information, based on the reproduction position information of And a range extracting means for extracting information corresponding to the related information. 3. The related information search device according to claim 1, wherein the information extracting means determines the reproduction position information based on the reproduction position information of the provided information in the information form which is reproduced continuously in time. Second range determining means for determining, as the range of the evaluation target information, a range having a predetermined relationship in time with the point in time at which the provided information of the indicated reproduction position is reproduced; A related information search apparatus characterized by comprising a range extracting means for extracting information. 4. The related information search device according to claim 1, wherein the information extraction means includes hint information extraction means for extracting hint information included as sub-information in the provided information. A first search instructing unit that includes a search server indicated by the hint information in a set of accessible search servers, and sends a search instruction instructing a search operation by designating these search servers; And a search execution means for executing a search process for a specified search server in accordance with the related information search apparatus. 5. The related information search device according to claim 1, further comprising: a designated information holding unit that holds designated information indicating a search server frequently used by a user, wherein the search unit is configured to store the designated information indicated by the designated information. A second search instructing unit that includes the searched search server in a set of accessible search servers, sends a search instruction designating these search servers and instructing a search operation; A related information search device, comprising: a search execution unit that executes a search process for a search server. 6. An information providing terminal device having information reproducing means for sequentially reproducing provided information provided by an information provider side, and records a related information search program for searching for related information related to the provided information. Receiving, from the information reproducing unit, reproduction position information indicating a position at which reproduction of the provided information has been completed, and, based on the reproduced position information, converting at least a part of the provided information into information to be evaluated. Information extraction procedure to extract as, receiving the evaluation target information obtained in the information extraction procedure,
A feature evaluation procedure for evaluating a feature quantity representing the feature, a search procedure for searching for relevant information from an accessible search server based on the feature quantity, and a search result obtained in the search procedure, A storage medium storing a related information search program for causing a computer to execute a related information output procedure presented to a user.