JP2006227219A - Information generating device, information output device, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an information generating device, an information output device, and a program in which a contents unit that is a portion of the contents can be accumulated from raw contents related to the explanation or the discussion of an exhibited object and the contents unit can appropriately be output. <P>SOLUTION: The information generating device accumulates the contents unit that is a portion of the contents from raw contents related to the explanation or the discussion of the exhibited object. The information generating device has a voice information receiving section which receives voice information that is the information having voice, a contents unit extracting section which extracts the contents unit having voice information equivalent to the time, that matches with a prescribed condition, based on the voice information received by the voice information receiving section and a contents unit accumulating section which accumulates the contents unit information that is related to the contents unit extracted by the contents unit extracting section. Thus, the information generating device can extract the contents unit which can be reused. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an information generation apparatus that records audio information when an object such as an exhibit or an exhibition panel is discussed or discusses, and performs or outputs a summary process that extracts a part of the audio information. The present invention relates to an information output device and the like.

In a conventional moving image summarizing method for summarizing moving images and the like, there is a moving image summarizing method capable of automatically extracting semantically important events in a moving image (for example, see Patent Document 1). The video summarization method includes a feature extractor 40 that extracts video features, a probability model 42 that uses a model such as a hidden Markov model to integrate features and determine boundaries, and commercial and non-commercial slow motion playback segments. A commercial / non-commercial filter 44 and a summary generator 46 that generates a summary based on the detected slow motion playback segment. The feature extractor 40 in this moving image summarization method extracts features from the color histogram in block 50 and extracts three features from the pixel-based differences in block 52. The features extracted at block 52 characterize each component of the playback segment's slow motion, still field, and / or normal speed playback. The feature extracted in block 50 in the moving image summarizing method characterizes the editing effect component.
Further, Non-Patent Document 1 discloses a raw content acquisition method described in the present embodiment.
JP 2002-232840 A (first page, FIG. 1 etc.) Yasuyuki Kaku et al., “Recording and Sharing Experiences in Ubiquitous Environment”, System / Control / Information (Journal of System Control Information Society), November 2004, Vol. 48, no. 1, pp. 458-463

  However, the conventional video summarization method cannot extract reusable content units from raw content recorded when explaining or discussing objects such as exhibits and display panels. There was a problem. In addition, there is a problem that it is not possible to realize a clone agent that is an agent that appropriately outputs such a content unit and performs explanation in an exhibition or the like instead.

The information generation apparatus according to the first aspect of the present invention is an information generation apparatus that accumulates content units, which are part of content, from raw content related to an explanation or discussion of an object on display. An audio information reception unit that receives audio information, a content unit extraction unit that extracts content units having audio information corresponding to a predetermined time based on the audio information received by the audio information reception unit, and the content unit extraction An information generation apparatus including a content unit storage unit that stores content unit information that is information about content units extracted by the unit. Here, the object is an exhibition panel or an exhibit. An exhibition panel is display information which is information displayed on a display, for example. The exhibits include, for example, art museum paintings and museum dinosaurs.
With this configuration, it is possible to automatically extract a meaningful and reusable content unit that is content related to an explanation or discussion of an object on display.

In addition, the information generating apparatus according to the second aspect of the present invention provides a display information storage unit that stores display information that is information to be displayed, a display unit that displays the display information, The content unit extraction further includes: an instruction receiving unit that receives an instruction for display information displayed by the display unit; and a position information acquisition unit that acquires position information that is information related to a position of the instruction received by the instruction receiving unit. The unit includes voice information corresponding to a predetermined condition based on the instruction received by the instruction receiving unit and the voice information received by the voice information receiving unit, and the position information acquisition unit during the time. Is an information generation device that extracts a content unit having position information acquired.
With this configuration, it is possible to automatically extract content units corresponding to explanations and discussion points.

In the information generating apparatus according to the third aspect of the invention, in contrast to the first and second aspects of the invention, the content unit extractor is a voice that is determined to be uttered by a single speaker for a predetermined time or more. The explanation voice detecting means for detecting the explanation voice, the typical explanation information storing means for storing the typical explanation information having one or more position information indicating the position of the instruction with respect to the display information, and the position information acquisition unit A description corresponding to the one or more pieces of position information acquired by the position information acquisition unit based on the one or more pieces of position information and one or more pieces of position information stored in the typical explanation information storage means, and the typical explanation information Similarity detection means for detecting the similarity to the explanation corresponding to, and the similarity detected by the explanation voice detection means and the similarity detected by the similarity detection means is equal to or greater than a predetermined similarity Degree If an information generating apparatus having a lecture content unit extracting means for extracting a lecture content unit having a time duration of the audio information of the description speech.
With this configuration, a lecture content unit, which is a typical content unit for explanation, can be automatically extracted.

Further, in the information generating apparatus according to the fourth invention, in contrast to the first and second inventions, the content unit extraction unit is configured such that the speech paired with one speaker identifier and another speaker who is substantially continuous. Dialogue voice detecting means for detecting a dialogue voice having a voice paired with an identifier, and an interaction content unit having voice information corresponding to the time of the dialogue voice when the dialogue voice detection means detects the dialogue voice An information generation apparatus including content unit extraction means.
With this configuration, it is possible to automatically extract an interaction content unit that is a content related to a discussion on the displayed object.

In addition, the information generating apparatus according to the fifth aspect of the present invention further includes a content unit output unit that outputs at least audio information included in the content unit stored in the content unit storage unit. An information generation device.
With this configuration, the accumulated content unit can be output, and the content unit can be reused. In the first to fourth aspects of the invention, the content unit may be output by another device.

The information generation apparatus of the sixth aspect of the invention detects a content unit storage unit that stores one or more content units that are contents related to the explanation or discussion of the displayed object, and a trigger for output of the content unit. When the trigger detection unit detects the trigger, the content unit acquisition unit that acquires one or more content units from the content unit storage unit, and the content unit acquired by the content unit acquisition unit are output. An information output apparatus including a content unit output unit.
Such a configuration is preferable because the content unit is automatically output upon detection of the trigger.
In addition, the information generating apparatus of the seventh invention further includes a display information storage unit that stores display information that is information to be displayed, and a display unit that displays the display information, relative to the sixth invention. Information output device.
With this configuration, a content unit such as an explanation of an exhibition panel can be automatically output.
The information generation apparatus according to the eighth aspect of the invention is the information output apparatus according to the seventh aspect, wherein the trigger detected by the trigger detection unit is an instruction for the display information displayed by the display unit. .
With such a configuration, the content unit is automatically output in accordance with an instruction for the display information, which is preferable.

The information generating apparatus according to the ninth aspect of the present invention is the information generating apparatus according to the eighth aspect, wherein the content unit has audio information and position information, and is information related to the position of the instruction detected by the trigger detection unit. The information output device further includes a position information acquisition unit that acquires information, and the content unit acquisition unit acquires a content unit corresponding to the position information acquired by the position information acquisition unit from the content unit storage unit. .
With this configuration, a content unit for explanation or discussion corresponding to a location designated by the user is output, which is preferable.

Further, in the information generating device according to the tenth aspect of the present invention, in contrast to the sixth to ninth aspects, the content unit acquisition unit is a distance that is information relating to a distance between the display on which the display information is displayed and the user. An information output device comprising: distance information acquisition means for acquiring information; and content unit acquisition means for acquiring one or more content units from the content unit storage unit based on distance information acquired by the distance information acquisition means. .
With this configuration, the user's interest in the object can be estimated based on the distance between the object on display and the user, and a content unit corresponding to the interest can be output.

In addition, the information generation apparatus according to the eleventh aspect of the invention provides an object identifier acquisition unit that acquires an object identifier that is information for identifying a user, and the content unit output unit is content based on the sixth to tenth aspects of the invention. And a description information storage unit that stores the object identifier acquired by the object identifier acquisition unit in association with the content unit while outputting the unit, and the content unit acquisition unit includes the object identifier. The content unit associated with the object identifier acquired by the acquisition unit is an information processing apparatus that does not acquire the content unit.
With this configuration, it is possible to avoid duplication of explanations for one user.

  Further, the information generation apparatus of the twelfth aspect of the invention further comprises an agent storage unit storing one or more agents with respect to the sixth to eleventh aspects of the invention, wherein the content unit output unit includes: When outputting a content unit, the information output device also outputs the agent.

  With such a configuration, by outputting the content unit in a manner described by the substitute agent agent, even if there is no explanation staff around the exhibit, a sense of reality that the explanation staff actually explains, Can be provided to users (visitors).

  According to the information generating apparatus of the present invention, it is possible to accumulate content units, which are part of content, from raw content related to explanations or discussions on objects on display. Moreover, according to the information output device of the present invention, the content unit can be output appropriately.

Hereinafter, embodiments of an information generation device and the like will be described with reference to the drawings. In addition, since the component which attached | subjected the same code | symbol in embodiment performs the same operation | movement, description may be abbreviate | omitted again.
(Embodiment 1)

In Embodiment 1, a video or audio when an object such as an exhibit or an exhibition panel is explained or discussed is recorded, and the object is explained or discussed from the recorded video or audio. An information generation apparatus that extracts and stores content that is considered to be important in the process (such content is appropriately referred to as “content unit”) will be described. Note that although the information generation apparatus also has a case of having an information output function, the main information output function will be described in the second embodiment. In addition, content that has video and audio when an object such as an exhibit or display panel is explained or discussed, and content that has not been processed is referred to as raw content.
FIG. 1 is a block diagram of an information generation apparatus according to the present embodiment.

  The information generation apparatus includes a target identifier storage unit 101, a display information storage unit 102, a display unit 103, an instruction reception unit 104, a position information acquisition unit 105, a video information reception unit 106, an audio information reception unit 107, an object identifier acquisition unit 108, A content unit extraction unit 109, a content unit storage unit 110, and a content unit output unit 111 are provided.

  The content unit extraction unit 109 includes explanation voice detection means 1091, typical explanation information storage means 1092, similarity detection means 1093, lecture content unit extraction means 1094, dialogue voice detection means 1095, and interaction content unit extraction means 1096.

  The target identifier storage unit 101 stores a target identifier for identifying a target of explanation or discussion. The object of explanation or discussion is, for example, an exhibition panel. Here, the object is mainly described as an exhibition panel displayed on a display (such as a touch panel), but also includes exhibits such as museums and art galleries. The object identifier may be any data structure such as an ID or an object name. The target identifier is necessary when handling raw content and content units for a plurality of targets at the same time. However, in the specific example of this specification, in order to simplify the description, the object to be handled is usually described as one. The target identifier storage unit 101 is preferably a nonvolatile recording medium, but can also be realized by a volatile recording medium. It may be a non-volatile recording medium or a volatile recording medium.

  The display information storage unit 102 stores display information that is information to be displayed. The display information is, for example, information that constitutes an exhibition panel, and its structure is not limited. The display information may be anything such as a bitmap file, a text file, a file described in a document description language such as TeX or PostScript. The display information storage unit 102 is preferably a non-volatile recording medium, but can also be realized by a volatile recording medium. It may be a non-volatile recording medium or a volatile recording medium.

  The display unit 103 displays the display information stored in the display information storage unit 102. The display unit 103 may be considered as including or not including an output device such as a display. The display unit 103 can be implemented by output device driver software, or output device driver software and an output device. Here, the display includes, for example, a touch panel.

  The instruction receiving unit 104 receives various instructions from the user. For example, the instruction receiving unit 104 receives an instruction for display information displayed on the display unit 103. This instruction is an instruction input by an instructor explaining display information, a person discussing display information, or the like. The instruction receiving unit 104 receives a content unit extraction instruction that is an instruction to extract a content unit. The content unit extraction instruction is an instruction for acquiring a lecture content unit, an interaction content unit, etc., which will be described later, from the accumulated audio information or the like. The instruction receiving unit 104 receives a content unit output instruction that is an instruction to output a content unit. The instruction input means is usually a press on the touch panel, but may be anything such as a mouse, a numeric keypad, a keyboard, or a menu screen. The instruction receiving unit 104 can be realized by a device driver of an input unit such as a touch panel or a mouse.

  The position information acquisition unit 105 acquires position information that is information regarding the position of the instruction received by the instruction reception unit 104. The position information is usually coordinate information (x, y), but it may be a region around the designated area or a paragraph around the designated area (a paragraph of the document displayed on the panel). The position information acquisition unit 105 is realized by, for example, touch panel driver software. Note that the position information acquisition unit 105 may be realized by an MPU, a memory, or the like. The position information acquisition unit 105 is configured by means for acquiring position information in which the exhibit is instructed when the object to be instructed is a physical exhibit (for example, a dinosaur in a museum). In this case, for example, a button that can be instructed to the exhibit is installed, and the position information acquisition unit 105 recognizes the pressing of the button and the pressed button, and acquires position information corresponding to the button.

  The video information receiving unit 106 receives video information having a video. Such a video usually shows a person explaining the object, a person discussing it, a display object of the object, and the like. The video information includes, for example, a video and a camera identifier that identifies the camera that acquired the video. However, the video information may be only the video, and may include other information (time information, location information (for example, location information is acquired by a GPS receiver not shown), etc.) The data structure does not matter. The video information receiving unit 106 may or may not include a camera that acquires video. The video information receiving unit 106 can be realized by a communication unit that receives video information from a camera. The video information receiving unit 106 may be realized by means for reading video information from a recording medium such as a DVD.

  The voice information receiving unit 107 receives voice information that is information having voice. The voice is, for example, a voice uttered by a speaker such as an explainer explaining the target or a person discussing the target. The voice information includes, for example, voice and a speaker identifier that identifies the speaker. The voice information receiving unit 107 may be considered to include or not include a microphone. The voice information receiving unit 107 can be realized by a means for receiving voice information transmitted from a microphone, for example. In such a case, the microphone transmits a pair of voice and voice information having information for identifying the microphone (here, a speaker identifier) to the voice information reception unit 107. The microphone may be a throat microphone.

  The object identifier acquisition unit 108 identifies one or more objects, and acquires one or more object identifiers that are information for identifying the objects. Here, the object means an object such as an exhibit or an exhibition panel, and a person (speaker) who explains or discusses the object.

  An example of a method for outputting such an object identifier will be described below. First, a tag for transmitting an infrared ID (hereinafter, referred to as “LED tag” as appropriate) is attached to an object or a speaker. A large number of video cameras are installed at the venue where the display panels are installed. The video camera is provided with a sensor for recognizing the infrared ID (hereinafter referred to as “IR tracker”). The object identifier acquisition unit 108 can be realized by such an IR tracker. Details of examples of means for realizing the object identifier acquisition unit 108 are described in Non-Patent Document 1, and thus detailed description thereof is omitted here. The object identifier acquisition unit 108 may be configured to recognize an image received by the video reception unit 106, identify a speaker shown in the video, and acquire a speaker ID (object identifier).

  Based on the audio information received by the audio information receiving unit 107, the content unit extracting unit 109 extracts audio information for a time that matches a predetermined condition and a content unit having the target identifier. “Matching a predetermined condition” means that, for example, the lecture described in detail below can be recognized, or the interaction described in detail below can be recognized. “Matching a predetermined condition” means that, for example, when a time when the volume of the sound is greater than or equal to a predetermined time continues for a predetermined time or more, the sound information for the time matches the predetermined condition. Become. In addition, there are various predetermined conditions. The content unit has at least audio information. In addition, the content unit may include one or more types of information among display related information, which is information regarding video, one or more object identifiers, position information, and display information corresponding to the position information. The display related information is information indicating a paragraph of the display information, partial information of the display information, and the like. In addition, the content unit includes types such as a lecture content unit and an interaction content unit, which will be described in detail below. Further, “extracting a content unit” may be extracting an output content unit itself or extracting information (pointer information) for extracting an output content unit. The content unit extraction unit 109 can be usually realized by an MPU, a memory, or the like. The processing procedure of the content unit extraction unit 109 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The content unit accumulating unit 110 accumulates content unit information that is information regarding the content unit extracted by the content unit extracting unit 109. The content unit information may be the output content unit itself, or may be information (pointer information) for extracting the output content unit. The recording medium in which the content unit storage unit 110 stores the content unit may be a recording medium of an external device of the information generation apparatus or a recording medium included in the information generation apparatus. The content unit storage unit 110 can usually be realized by an MPU, a memory, or the like. The processing procedure of the content unit storage unit 110 is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The content unit output unit 111 outputs the content unit stored by the content unit storage unit 110. The content unit output unit 111 need not output all types of information of content units. The content unit output unit 111 outputs at least audio information included in the content unit. The content unit output unit 111 may be considered as including or not including an output device such as a display or a speaker. The content unit output unit 111 can be implemented by output device driver software, or output device driver software and an output device. Here, output is a concept including display on a display, printing on a printer, sound output, transmission to an external device, and the like.

  The explanation voice detecting means 1091 detects explanation voice, which is a voice that is determined to be spoken by a single speaker for a predetermined time or more. The explanation voice detecting means 1091 can be usually realized by an MPU, a memory or the like. The processing procedure of the explanation voice detecting means is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The typical explanation information storage unit 1092 stores typical explanation information having one or more pieces of position information indicating the position of the instruction with respect to the display information. The typical explanation information usually has one or more pieces of information on the position where the explanation staff indicates the target when the explanation staff gives a typical explanation about the target such as an exhibition panel (display information here). . However, the significance indicated by the typical explanation information does not matter. That is, the typical explanation information may have one or more pieces of information on the position where the explanation staff indicates the target in the explanation pattern in the case of peculiarity. The typical explanation information may be configured to include one or more instruction information included in a pair of position information and time information. The typical explanation information storage unit 1092 is preferably a non-volatile recording medium, but can also be realized by a volatile recording medium. It may be a non-volatile recording medium or a volatile recording medium.

  The similarity detection means 1093 is one or more acquired by the position information acquisition unit based on the one or more position information acquired by the position information acquisition unit 105 and the one or more position information stored in the typical explanation information storage unit 1092. The degree of similarity between the explanation corresponding to the position information and the explanation corresponding to the typical explanation information is detected. The similarity detection unit 1093 includes one or more pieces of instruction information (this is referred to as acquisition instruction information) having a pair of the position information acquired by the position information acquisition unit 105 and the time information at which the position information is acquired. The degree of similarity may be detected by comparing the typical explanation information (including time information) of the explanation information storage unit 1092. There are various methods for detecting the degree of similarity, and any method can be used. The similarity detection means 1093 can be usually realized by an MPU, a memory, or the like. The processing procedure of the similarity detection means 1093 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The lecture content unit extraction means 1094 detects the explanation voice when the explanation voice detection means 1091 detects the explanation voice and the similarity detected by the similarity detection means 1093 is equal to or higher than a predetermined similarity. The lecture content unit having the audio information and the target identifier for the time is extracted. However, when there is only one target, the lecture content unit may have only audio information. Lecture content unit extraction means 1094 can usually be realized by an MPU, a memory, or the like. The processing procedure of the lecture content unit extraction means 1094 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The dialog voice detecting means 1095 detects a dialog voice having a voice paired with one speaker identifier and a voice paired with another substantially continuous speaker identifier. “Substantially continuous” means that there may be temporal overlap within a predetermined interval or there may be a temporal interval within a predetermined interval. Also, the “predetermined interval” when there is a temporal overlap and when there is a time interval may be different. The dialogue voice detecting means 1095 can be usually realized by an MPU, a memory or the like. The processing procedure of the dialog voice detecting means 1095 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

The interaction content unit extraction unit 1096 extracts the interaction content unit having the voice information and the target identifier for the time of the dialogue voice when the dialogue voice detection unit 1095 detects the dialogue voice. Note that the target identifier is not essential for the interaction content unit. The interaction content unit may be only audio information. However, since the interaction content unit includes two or more voices, it is preferable to have information (for example, the above-described object identifier) for identifying the voice. Further, the interaction content unit may have only information that can recognize the voice of a different person even if there is no information that recognizes who the voice is. Further, the interaction content unit extraction means 1096 detects not only the audio information but also the interaction content unit on condition that two or more persons (objects) that are interacting with each other face each other, as will be described later. It may be extracted. Here, “facing” does not need to face from the front, but may be in a positional relationship that allows dialogue. An algorithm example for recognizing “facing” will be described later, but the method is not limited. The interaction content unit extraction unit 1096 can be usually realized by an MPU, a memory, or the like. The processing procedure of the interaction content unit extraction means 1096 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).
Next, the operation of the information generating apparatus will be described using the flowcharts of FIGS.
(Step S <b> 201) The display unit 103 reads display information from the display information storage unit 102.
(Step S202) The display unit 103 displays the display information acquired in step S201.

  (Step S203) The instruction receiving unit 104 determines whether or not a content unit extraction instruction has been received. If a content unit extraction instruction is accepted, the process proceeds to step S204, and if a content unit extraction instruction is not accepted, the process jumps to step S206.

(Step S204) Based on the audio information received by the audio information receiving unit 107, the content unit extracting unit 109 extracts audio information for a time that matches a predetermined condition and a content unit having the target identifier. Details of the process of extracting the content unit will be described later.
(Step S205) The content unit storage unit 110 stores the content unit information extracted in step S204.

  (Step S206) The instruction receiving unit 104 determines whether or not a content unit output instruction has been received. If the content unit output instruction is accepted, the process goes to step S207, and if the content unit output instruction is not accepted, the process jumps to step S209.

  (Step S207) The content unit output unit 111 determines whether there is a content unit stored by the content unit storage unit 110. Alternatively, the content unit output unit 111 determines whether there is a content unit corresponding to the instruction received in step S206. As a result of the determination, if it is determined that there is a content unit that matches the condition, the process goes to step S208. If it is determined that there is no content unit, the process returns to step S203.

(Step S208) The content unit output unit 111 outputs the content units stored by the content unit storage unit 110. Here, it is preferable that the content unit output unit 111 selects and outputs a content unit corresponding to the instruction received in step S206. The “content unit corresponding to the received instruction” is, for example, a content unit corresponding to position information indicated by the instruction (paired with position information close to the position information). In addition, it is preferable that the content unit output unit 111 selects and outputs a content unit according to other conditions. Details of an example of processing for outputting a content unit will be described in Embodiment 2.
(Step S209) The video information receiving unit 106 receives video information having a video. The video information receiving unit 106 does not matter the data amount (shooting time) of the video information that is received at one time.
(Step S210) The voice information receiving unit 107 receives voice information that is information having voice.

  (Step S211) The object identifier acquisition unit 108 acquires one or more object identifiers which are information for identifying an object. The object identifier acquisition unit 108 acquires one or more object identifiers from the IR tracker, for example.

  (Step S212) The content unit extraction unit 109 acquires the target identifier from the target identifier storage unit 101. The target identifier is, for example, an identifier for identifying an exhibition panel, a booth where the exhibition panel is displayed, or an exhibit (for example, a picture of a museum). Note that the process of acquiring the target identifier is not essential.

(Step S213) The instruction receiving unit 104 determines whether an instruction for display information has been received. If the instruction | indication with respect to display information is received, it will go to step S214, and if the said instruction | indication is not received, it will fly to step S215. Note that the instruction receiving unit 104 may receive an instruction for the exhibit instead of the display information.
(Step S214) The position information acquisition unit 105 acquires position information that is information regarding the position of the instruction received in step S213.

  (Step S215) The content unit extraction unit 109 configures information to be accumulated by pairing the video information, audio information, object identifier, target identifier, and position information acquired in steps S209 to S214. Note that the information to be stored does not necessarily include all information of video information, audio information, object identifier, target identifier, and position information. For example, only the audio information, only the audio information and the position information, and the audio information Only the target identifier may be used. Further, the information to be stored is the above-described raw content.

(Step S216) The content unit extraction unit 109 temporarily stores the raw content configured in step S215. Until at least the content unit extraction instruction is received, at least the content unit extraction unit 109 holds the raw content. The recording medium holding such information may be a non-volatile recording medium such as a hard disk or a ROM, or a volatile recording medium such as a RAM. Further, it is preferable that the content unit extraction unit 109 stores the raw content in a nonvolatile recording medium. In particular, when the content unit information is information (pointer information) for extracting an output content unit, the raw content is normally stored in a nonvolatile recording medium.
In the flowchart of FIG. 2, the processing of step S215 and step S216 may be performed by other means other than the content unit extraction unit 109 or other means (not shown).
In the flowchart of FIG. 2, the process is ended by power-off or a process end interrupt.
Next, a first example of processing for extracting a content unit will be described with reference to the flowcharts of FIGS. A process of extracting a lecture content unit will be described with reference to FIGS.
(Step S301) The similarity detection unit 1093 acquires the typical explanation information stored in the typical explanation information storage unit 1092.
(Step S302) The similarity detection unit 1093 substitutes 1 for a counter i. It does not matter how the counter i is incremented (increment data size width).

(Step S303) The similarity detection unit 1093 determines whether or not the i-th voice is greater than or equal to a predetermined volume (that is, whether or not an utterance is being made). If it is equal to or larger than the predetermined size, the process goes to step S304, and if it is less than the predetermined size, the process goes to step S309. When i is determined, it is determined which position (address) of the audio data is the audio data.
(Step S304) The similarity detection unit 1093 acquires continuous voices uttered by the same speaker as the i-th voice. Here, it is assumed that a speaker identifier exists in a pair with the voice.

  (Step S305) The similarity detection unit 1093 acquires the typical explanation information acquired in Step S301, the positional information for the time of the continuous sound acquired in Step S304 (hereinafter referred to as “voice group” as appropriate), and the like. The similarity between the position information and the typical explanation information is calculated. Details of the similarity calculation processing example will be described with reference to the flowchart of FIG. Here, the position information or the like may be only position information, or may include position information and time information corresponding to the position information. Note that the degree of similarity acquired in step S305 may be a degree of dissimilarity. If the degree of similarity acquired in step S305 is a degree that is not similar, the similarity is a case where the degree of similarity is equal to or less than a predetermined value.

  (Step S306) The lecture content unit extraction unit 1094 determines whether the similarity calculated in step S306 is equal to or greater than a predetermined value (similar). If the similarity is greater than or equal to a predetermined value (if similar), go to step S307, and if the similarity is less than a predetermined value (if not similar), go to step S308.

  (Step S307) The lecture content unit extraction unit 1094 extracts a lecture content unit including audio information and the like. Here, extraction may be to separately store a lecture content unit in a recording medium (not shown), or in the content stored in step S216, a flag indicating that it is a lecture content unit (usually a start flag (described below)). In this example, a start point) and an end flag (in the following example, there is an end point) may be recorded. When the flag is recorded, the lecture content unit is obtained by using the flag as a mark at the time of output. The flag may be a pointer indicating the start point and the end point of the content constituting the lecture content unit.

(Step S308) The lecture content unit extraction unit 1094 increments the counter i by one. Note that incrementing i by 1 means that the pointer is advanced to the next audio data for which the similarity is to be calculated, and the width of i to be incremented (the number of bytes in the audio data) is not constant. If it is good, that is, if the similarity of the continuous sounds acquired in step S304 is low (if they are not similar to the typical explanation information), the lecture content unit extraction unit 1094 advances the pointer to the next sound. Also, if the i-th audio is less than a predetermined level, for example, the lecture content unit extraction unit 1094 advances the inter to the next audio. Go to step S303.
(Step S309) The lecture content unit extraction means 1094 determines whether or not the i-th audio exists. If the i-th voice exists, the process goes to step S308, and if not, the process returns to the upper function.
Next, details of an example of similarity calculation processing will be described using the flowchart of FIG. In the description of the similarity calculation process, the similarity of the position information corresponding to the typical explanation information and the acquired continuous sound is calculated.

  (Step S401) The similarity detection unit 1093 initializes. That is, the similarity detection unit 1093 substitutes 1 for the counter j. The similarity detection unit 1093 substitutes 0 for the variable “distance” and the variable “time difference”. The variable “distance” is a variable into which information obtained by accumulating the difference between two pieces of position information is substituted. The two pieces of position information are position information such as position information of typical explanation information and position information corresponding to continuous sound. The variable “time difference” is a variable for substituting the cumulative difference between the time interval in which the two pieces of position information included in the typical explanation information are recorded and the time interval in which the two pieces of position information included in the position information are recorded.

(Step S402) The similarity detection unit 1093 determines whether or not the j-th position information exists in the typical explanation information. If the j-th position information exists, the process proceeds to step S403, and if the j-th position information does not exist, the process proceeds to step S412.
(Step S403) The similarity detection unit 1093 acquires the j-th position information from the typical explanation information.
(Step S404) The similarity detection unit 1093 acquires the j-th position information from the position information corresponding to the acquired continuous sound.
(Step S405) The similarity detection unit 1093 calculates the distance between the position information acquired in step S403 and the position information acquired in step S404.
(Step S406) The similarity detection unit 1093 adds the distance calculated in step S405 to the variable “distance”.
(Step S407) The similarity detection unit 1093 acquires the time between the (j−1) th to jth position information from the typical explanation information. When j is “1”, the time between the position information is “0”.

(Step S408) The similarity detection unit 1093 acquires the time between the (j−1) th to jth position information from the position information corresponding to the acquired continuous sound. When j is “1”, the time between the position information is “0”.
(Step S409) The similarity detection unit 1093 calculates a time difference which is a difference between the time acquired in Step S407 and the time acquired in Step S408.
(Step S410) The similarity detection unit 1093 adds the time difference calculated in Step S409 to the variable “time difference”.
(Step S411) The similarity detection unit 1093 increments the counter j by 1. The process returns to step S402.

  (Step S412) The similarity detection unit 1093 calculates the similarity using the value of the variable “distance” and the value of the variable “time difference” as parameters. The calculation function is not limited. For example, the similarity detection unit 1093 calculates the similarity by f = “0.7 × (1 / variable“ distance ”value) + 0.3 × (1 / variable“ time difference ”value)”. In such a case, the difference in distance is more important than the time difference. Return to upper function.

Needless to say, the algorithm for calculating the similarity is not limited to the above. For example, the similarity detection unit 1093 may calculate the similarity by comparing only the position information. Further, for example, the similarity detection unit 1093 includes the area indicated by the position information of the typical explanation information in the case where all the areas indicated by the position information corresponding to the acquired continuous voice are included (the continuous voice is described in the typical explanation). An algorithm may be used in which the similarity is set to a high value in the case where it is considered that the voice is when all the portions are described. The similarity may be simple information such as “similar (value is“ 1 ”)” or “not similar (value is“ 0 ”)”.
Next, a first example of processing for extracting a content unit will be described with reference to the flowcharts of FIGS. The process of extracting the interaction content unit will be described with reference to FIGS.

(Step S501) The dialogue voice detecting means 1095 determines a set (i, j). i and j correspond to speaker identifiers. That is, it is assumed that there are two or more speakers, and in such a case, a set of two speakers is determined in this step. Usually, “i = 1” and “j = 2” are initially set.
(Step S502) The dialogue voice detecting means 1095 acquires the voice of the speaker identifier corresponding to i.
(Step S503) The dialogue voice detecting means 1095 acquires the voice of the speaker identifier corresponding to j.

  (Step S504) The dialogue voice detecting means 1095 performs processing for detecting a dialogue based on the voice obtained in Step S502 and the voice obtained in Step S503. Details of this dialogue detection processing will be described with reference to the flowchart of FIG.

  (Step S505) The interaction content unit extraction unit 1096 extracts an interaction content unit including audio information and the like constituting the dialogue based on the dialogue voice group detected in step S504. The extraction of the interaction content unit may be a process of acquiring all the contents (such as audio and video) constituting the interaction content unit, or acquiring pointer information (for example, “start point” and “end point”) for the raw content. Processing is also acceptable.

(Step S506) The dialogue voice detecting means 1095 determines whether or not the next (i, j) exists. If the next (i, j) exists, the process returns to step S501. If the next (i, j) does not exist, the process returns to the upper function. Note that (1, j) in (1, 2) is, for example, (1, 3). For example, it is assumed that the speaker identifier is given as “1, 2, 3,...”.
Next, an example of the conversation detection process will be described using the flowchart of FIG.
(Step S601) The dialog voice detecting means 1095 is initialized. Here, the dialog voice detecting means 1095 substitutes 1 for the counters k and l.

(Step S602) The dialogue voice detecting means 1095 determines whether or not the k-th continuous voice group of the speaker of i exists. If the voice group exists, the process proceeds to step S603, and if the voice group does not exist, the process returns to the upper function.
(Step S603) The dialogue voice detecting means 1095 acquires the k-th continuous voice group of the speaker of i.

(Step S604) The dialog voice detecting means 1095 determines whether or not the l-th continuous voice group of the j speaker is present. If the voice group exists, the process goes to step S605, and if the voice group does not exist, the process returns to the upper function.
(Step S605) The dialogue voice detecting means 1095 acquires the l-th continuous voice group of the j speaker.

  (Step S606) The dialogue voice detecting means 1095 determines whether or not the k-th continuous voice group of the i speaker and the l-th continuous voice group of the j speaker are substantially continuous. If almost continuous, the process goes to Step S607, and if not substantially, the process goes to Step S608. Note that “substantially continuous” refers to the situation of (a), (b), and (c) in FIG. That is, (a) indicates that two voice groups (voice 1, voice 2) are continuously output with only a slight overlap (x). In this case, the time interval of x is preferably “5 seconds or less”, for example. Further, (b) is a case where two voice groups (voice 1, voice 2) are continuous without overlapping or gaps. Furthermore, (c) is a case where there is a short time interval (y) between the two voice groups (voice 1, voice 2). For example, “y” is preferably “3 seconds or less”. Further, the modes shown in FIGS. 7D and 7E cannot be determined to be “substantially continuous”. This is because the time during which the two speakers speak twice is a predetermined time or more. In FIG. 7, the line indicates that the user is speaking, and the horizontal axis is time (t). Voice 1 and voice 2 are voices uttered by different persons (voices paired with different speaker identifiers).

  (Step S607) The dialogue voice detecting means 1095 is a voice that constitutes one dialogue between the k-th continuous voice group of the speaker of i and the l-th continuous voice group of the speaker of j. A mark (predetermined value) is given. If it is determined in step S606 that the predetermined value constituting the mark is not “substantially continuous”, a different mark is given by incrementing the predetermined value, for example.

(Step S608) The dialogue voice detecting means 1095 determines whether the counter k has been incremented immediately before or the counter l has been incremented immediately before. When the counter k is incremented immediately before, the process goes to step S609, and when the counter l is incremented immediately before, the process goes to step S610.
(Step S609) The dialogue voice detecting means 1095 increments the counter l by 1. The process returns to step S604.
(Step S610) The dialogue voice detecting means 1095 increments the counter k by 1. The process returns to step S602.

  In the flowchart of FIG. 6, the dialogue voice is detected based on only the voice information, but may be based on the object identifier and the voice. For example, the object identifier acquired by “IR tracker” held by user A is the object identifier of user B, and the object identifier acquired by “IR tracker” held by user B is the object identifier of user A. In some cases, it can be determined that user A and user B are facing each other. That is, the object identifier transmitted from the information acquisition apparatus having the “IR tracker” of the user A is the object identifier of the user B, and the object identifier transmitted from the information acquisition apparatus having the “IR tracker” of the user B Is the object identifier of user A, the information generating apparatus determines that user A and user B are facing each other.

An information acquisition apparatus having an “IR tracker” includes, for example, a CCD camera and a microphone. An example of such an information acquisition apparatus is shown in FIG. In FIG. 8, a "CCD camera", an "infrared ID tag", and an "infrared sensor" are provided above the user's ear. The “CCD camera” acquires an image. The “infrared ID tag” transmits an infrared signal on which a signal indicating the object identifier of the user is superimposed. The “infrared sensor” receives an infrared signal from the outside. That is, the “infrared sensor” is the “IR tracker” described above. It has a “microphone” at the mouth and a “throat microphone” at the throat. In addition, an HMD (head mounted display) is provided in front of the eyes. The signal from the “CCD camera” is acquired by the PC on the back of the user and transmitted from the PC to the information generation apparatus. Further, the HMD is used to output information on the position of the user, the displayed exhibits and the people (objects) facing the user. Needless to say, the information acquisition apparatus in FIG. 8 is an example of an information acquisition method.
Hereinafter, a specific operation of the information generation apparatus according to the present embodiment will be described.

  Suppose that it is a meeting place of an academic society with many exhibition panels now. In this exhibition panel, a panel to be announced (information on technical explanation) is displayed on a display having a touch panel. There are many visitors to the exhibition panel at the venue. Visitors are user A and user B, for example. In addition, there is an explanation panel for each exhibition panel. The exhibitor of the display panel is user C, for example. Each user wears the information acquisition device shown in FIG. 8 and sees the video he / she sees, the object identifier for identifying the object (person, display panel, etc.) facing the user, the voice uttered (a set of such information is the raw content Is). Then, the raw content is sent to the information generation device.

  Now, user A and user B are discussing in front of the display panel X in front of the display panel X. Here, it is assumed that the object identifier of the user A is “35”, the object identifier of the user B is “38”, and the object identifier of the exhibition panel X is “1”. Such an object identifier is held in advance by each information acquisition device. The “infrared ID tag” of the information acquisition device transmits an infrared signal on which an object identifier is superimposed. In addition, the “IR tracker” acquires an object identifier from a signal transmitted from an “infrared ID tag” attached to an object (a person, an exhibition panel, etc.) facing each other.

  In this case, FIG. 9 shows a conceptual diagram of the raw content (raw content having video, audio, and object identifier) acquired by each information acquisition device. This information generation device accepts raw content acquired by each information acquisition device of user A, user B, and user C. Such raw content is received by, for example, wireless or wired communication means.

In FIG. 9, the raw content sent from the information acquisition device of each user is a set of video, audio, and object identifier. In addition, video, audio, and object identifier information are sent synchronously. In FIG. 9, the information sent from the information acquisition device of user A includes the video that user A is viewing, the voice that user A is uttering, the identifier of the object that user A is facing (here, user B, etc.) Identifier).
Next, it is assumed that the user C who is an explanation member of the exhibition panel X comes near the exhibition panel X and starts explaining the exhibition panel X to the user A and the user B.

  In such a case, an example of raw content sent to the information generation device from the information acquisition device of the user C and the information acquisition device installed in or around the display panel X is shown in FIG. FIG. 10 is a conceptual diagram of raw content accumulated from when user C started explaining the display panel X. In addition, since the user C performs an explanation while indicating the explanation location of the display panel X, position information (here, coordinate information) indicating the instructed position is also acquired. Note that the instructed target (here, the target identifier “1” of the display panel X) is described in the header of the position information. Here, the target identifier “1” is the same as the object identifier of the display panel X.

The information generating apparatus temporarily accumulates the position information indicated by the raw content received from one or more information acquisition apparatuses and the instruction for the target received by the instruction receiving unit 104. Examples of the temporarily stored information are shown in FIGS. The raw content may be considered to include the position information.
Next, it is assumed that the user inputs a content unit extraction instruction. Then, the information generation apparatus starts extracting the content unit.
Here, a specific example of content unit extraction processing will be described.

  First, processing for extracting a lecture content unit from the raw content in FIG. 10 will be described. FIG. 11 is an example of typical explanation information. The typical explanation information here includes time information and position information. This typical explanation information first instructs (explains) around (20, 40), instructs (explains) around (20, 80) after 100 seconds, and instructs around (120, 40) after 80 seconds. (Explanation) and indicates (description) around (120, 80) after 30 seconds. FIG. 12 is a schematic diagram showing a state of such a typical explanation instruction.

  First, the explanation voice detecting unit 1091 detects a section in which the voice of the user C in FIG. 10 is uttered for a predetermined time or more. That is, the explanation voice detecting unit 1091 detects a section from “t = 350” to “t = 2850” in the voice of the user C in FIG.

  Next, with respect to such a voice segment, the similarity detection means 1093 causes the position information “(20, 38), t = 611”, “(20, 88), t = 720” “(125, 40)” in FIG. “t = 800” “(131, 80), t = 841” is acquired. Each “t” value indicates the time when each piece of position information is acquired (see FIG. 10).

  The similarity detection means 1093 then calculates the distance between (20, 40) and (20, 38), the distance between (20, 80) and (20, 88), and the distance between (120, 40) and (125, 40). , (120, 80) and (131, 80) are calculated. Then, the similarity detection means 1093 obtains the sum “26” of the four distances.

Also, the similarity degree detection means 1093 is the difference between the time intervals of the four points “difference between (100-0) and (720-611)” “difference between (180-100) and (800-720)” “(210 The difference “−180) and (841−800)” is obtained as “20”.
Then, the similarity detection unit 1093 calculates the similarity as “1 / (26 + 20)” based on “26” and “20”, for example.

The lecture content unit extraction means 1094 is assumed to be adopted as a typical explanation because the similarity is “1/46” which is equal to or higher than the predetermined similarity “1/100”. Information (user ID = 52, start time = 350, end time = 2850) from the time “350” of the voice of C to “2850 (not shown)” where the utterance ends is extracted. Note that the lecture content unit extraction unit 1094 may acquire information including audio from the time “350” to “2850” by cutting it from the information of the user C in FIG.
Next, processing for extracting an interaction content unit from various information in FIG. 9 will be described.

First, the dialogue voice detecting means 1095 determines two target speaker identifiers. Here, two speaker identifiers are determined as (35, 38). The determination of the speaker identifier is performed by sequentially determining a combination of two speaker identifiers from all the speaker identifiers received by the content unit extraction unit 109, and the determination algorithm is not limited. Also, the interaction content unit extraction process described below is usually performed for all combinations.
Then, the dialogue voice detecting means 1095 acquires voices corresponding to the speaker identifiers “35” and “38”.

  Next, the dialogue voice detecting means 1095 acquires the first continuous voice group among the voices of the speaker identifier “35”. As a result, the dialogue voice detecting means 1095 acquires the voice (referred to as the first voice) from “t = 0” to “t = 260” (see FIG. 9) in the voice of the speaker identifier “35”. This voice is a voice uttered when the speaker identified by the speaker identifier “35” speaks “This exhibition panel is interesting.

  Next, the dialogue voice detecting means 1095 acquires the first continuous voice group among the voices of the speaker identifier “38”. As a result, the dialogue voice detecting means 1095 acquires (trays as the second voice) voices from “t = 258” to “t = 360” (see FIG. 9) in the voice of the speaker identifier “38”. This voice is a voice uttered when the speaker identified by the speaker identifier “38” speaks “I wonder? Where?”.

  Then, the dialogue voice detecting means 1095 determines whether or not the first voice and the second voice are almost continuous. Here, the difference between the end point “260” of the utterance time of the first voice and the start point “258” of the utterance time of the second voice is “2”, which is equal to or less than a predetermined time interval (for example, “5”). Therefore, it is judged that it is almost continuous.

  Next, the dialog voice detecting means 1095 tries to acquire the second continuous voice group in the voice of the speaker identifier “35”. However, in FIG. 9, since the second continuous voice group does not exist, the second continuous voice group cannot be acquired, and the dialogue detection process is completed.

  Next, the interaction content unit extraction unit 1096 acquires the start point “0” of the first sound and the end point “360” of the second sound. Then, the content unit storage unit 110 stores content unit information (in this case, information having at least “0” and “360”). The content unit information may be only information indicating the start point and end point for extracting the output content unit, or may be information on the content unit itself. The information of the content unit itself includes voice, video, etc. for the time interval from “t = 0” to “t = 260” of the speaker identifier “35”, and “t = 258” of the speaker identifier “38”. The content is configured by cutting out audio, video, and the like for a time interval of “t = 360”. The content unit information usually has information (for example, “35” and “38”) that identifies the start and end points for extracting the output content unit, and all the content of the speaker identifiers “35” and “38”. Information). An example of content unit information is shown in FIG. The content unit information in FIG. 13 includes the first object identifier (speaker identifier “35”), the start point, the end point, and the second object for extracting the voice transmitted from the object identified by the first object identifier. This is a data structure having an identifier (speaker identifier “38”), a starting point and a ending point for extracting the voice transmitted from the object identified by the second object identifier.

Further, in the above, as described above, a process for determining that the object having the speaker identifier “35” and the object having the speaker identifier “38” are facing each other is further added, and only when both are facing each other, the interaction is performed. It is more preferable to extract the content unit. In FIG. 9, the information of the speaker identifier “35” has the object identifier “38” from the time “t = 0” to “t = 260”, and the information of the speaker identifier “38”. Has the object identifier “35” in the time from “t = 258” to “t = 360”, it is determined that the two are facing each other.
Next, in the same manner as described above, the dialogue voice detecting means 1095 acquires voices corresponding to the speaker identifiers “35” and “52”.

  Next, the dialogue voice detecting means 1095 acquires the first continuous voice group among the voices of the speaker identifier “35”. As a result, the dialogue voice detecting means 1095 acquires the voice (referred to as the first voice) from “t = 0” to “t = 260” (see FIG. 9) in the voice of the speaker identifier “35”. This voice is a voice uttered when the speaker identified by the speaker identifier “35” speaks “This exhibition panel is interesting.

  Next, the dialogue voice detecting means 1095 acquires the first continuous voice group among the voices of the speaker identifier “52”. As a result, the dialog voice detecting means 1095 acquires the voice from “t = 400” to “t = 500” (see FIG. 9) in the voice of the speaker identifier “52” (referred to as the second voice). This voice is a voice uttered when the speaker identified by the speaker identifier “52” utters “I will explain ...”.

  Then, the dialogue voice detecting means 1095 determines whether or not the first voice and the second voice are almost continuous. Since the above-described algorithm does not satisfy the condition that the first voice and the second voice are substantially continuous, the interaction content unit extraction unit 1096 uses the interaction content unit in the set of the speaker identifiers “35” and “52”. Do not extract.

  Further, similar to the above, the dialogue voice detecting means 1095 acquires voices corresponding to the speaker identifiers “38” and “52”. Similarly to the above, since the condition that the first voice of the speaker identifier “38” and the second voice of “52” are almost continuous is not satisfied, in the set of the speaker identifiers “38” and “52” The interaction content unit extraction unit 1096 does not extract the interaction content unit.

  Through the above process, the lecture unit and the interaction content unit are extracted. The lecture unit and the interaction content unit are stored in a form that can be identified. For example, the storage media (including tables and the like) to be stored are different, or flags identifying the unit type are stored in pairs in the stored content unit information.

  Next, it is assumed that the user inputs a predetermined content unit output instruction. The instruction receiving unit 104 receives a content unit output instruction. Specifically, for example, the display panel X is displayed on a display having a touch panel as shown in FIG. In the lower left of the display, a “Lecture” button and an “Interaction” button are displayed. When the user presses the “Lecture” button, the instruction receiving unit 104 recognizes the pressing of the button, and the content unit output unit 111 acquires and outputs the lecture content unit stored in the content unit storage unit 110.

  In addition, when the user presses a questionable part in the display panel, the position information acquisition unit 105 acquires position information (coordinate information), and acquires and outputs a lecture content unit having an explanation outlined in the position information. Is also suitable.

When the user presses the “interaction” button, the instruction receiving unit 104 recognizes that the button is pressed, and the content unit output unit 111 acquires and outputs the interaction content unit stored by the content unit storage unit 110. .
As described above, according to the present embodiment, a content unit that can be reused can be automatically extracted from raw content recorded when an object such as an exhibit or a display panel is described or discussed.

  Furthermore, the processing in the present embodiment may be realized by software. Then, this software may be distributed by software download or the like. Further, this software may be recorded and distributed on a recording medium such as a CD-ROM. This also applies to other embodiments in this specification. Note that the software that implements the information generation apparatus according to the present embodiment is the following program. In other words, this program is a program for accumulating a content unit, which is a part of content, from raw content related to an explanation or discussion of an object on display. A receiving audio information receiving step, a content unit extracting step for extracting a content unit having audio information for a time matching a predetermined condition based on the audio information received in the audio information receiving step, and the content unit extracting step A program for executing a content unit accumulation step for accumulating content unit information, which is information relating to the content unit extracted in (1).

  Further, a display step for displaying display information on a computer, an instruction receiving step for receiving an instruction for the display information displayed in the display step, and a position of the instruction received in the instruction receiving step for the program A position information acquisition step of acquiring position information as information is further executed, and in the content unit extraction step, based on the instruction received in the instruction reception step and the audio information received in the audio information reception step, A program that extracts audio information for a time that matches the condition and a content unit that has the position information acquired in the position information acquisition step during the time may be used.

  For the above program, the content unit extraction step includes an explanation voice detection substep for detecting explanation voice, which is a voice determined to be spoken by a single speaker for a predetermined time or more, and the position A description corresponding to the one or more position information acquired in the position information acquisition step based on the one or more position information acquired in the information acquisition step and one or more position information included in the stored typical description information; The similarity detection substep for detecting the similarity with the explanation corresponding to the typical explanation information, the explanation voice is detected at the explanation voice detection substep, and the similarity detected at the similarity detection substep is determined in advance. Lecture content unit having audio information for the time of the explanation audio when the similarity is equal to or higher than the similarity It may be a program having a content unit extraction sub-step.

  In addition, for the above program, the content unit extraction step detects a dialog voice having a voice paired with one speaker identifier and a voice paired with another substantially continuous speaker identifier. The program may include an interaction content unit extraction substep for extracting an interaction content unit having audio information corresponding to the time of the dialogue voice when the dialogue voice is detected in the substep and the dialogue voice detection substep.

Moreover, the program may further cause the computer to further execute a content unit output step for outputting at least audio information of the content unit stored in the content unit storage step.
(Embodiment 2)
FIG. 15 is a block diagram of the information output apparatus in the present embodiment.

The information output device includes an instruction receiving unit 104, a content unit storage unit 1501, a trigger detection unit 1502, a position information acquisition unit 1503, an object identifier acquisition unit 1504, an already described information storage unit 1505, a content unit acquisition unit 1506, and an agent storage unit 1507. , A content unit output unit 1508, a display information storage unit 1509, and a display unit 1510.
The content unit acquisition unit 1506 includes distance information acquisition means 15061 and content unit acquisition means 15062.

  The content unit storage unit 1501 stores one or more content units. The content unit may be the content unit itself output as it is, or the raw content and information (pointer information) for extracting the output content unit. The raw content is information including the video and audio as they are acquired by the video information receiving unit 106 and the audio information receiving unit 107 described above. The content unit has, for example, audio information and position information. The content unit is, for example, the above-described lecture content unit or interaction content unit. Such content units are preferably content units extracted by the information output apparatus described in the first embodiment. The content unit storage unit 1501 is preferably a non-volatile recording medium, but can also be realized by a volatile recording medium. It may be a non-volatile recording medium or a volatile recording medium.

  The trigger detection unit 1502 detects an output trigger of the content unit. The trigger detected by the trigger detection unit 1502 is, for example, an instruction for display information displayed on the display unit 1510. The trigger detected by the trigger detection unit 1502 may be, for example, recognizing that the user is nearby. Such recognition is performed, for example, by the object identifier acquisition unit 1504 receiving a signal output from the user's IR tracker and acquiring the object identifier from the signal. The trigger detection unit 1502 can usually be realized by an MPU, a memory, or the like. The processing procedure of the trigger detection unit 1502 is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The position information acquisition unit 1503 acquires position information that is information regarding the position of the instruction detected by the trigger detection unit 1502. The position information is usually coordinate information (x, y), but it may be a region around the designated area or a paragraph around the designated area (a paragraph of the document displayed on the panel). The position information acquisition unit 1503 is realized by, for example, touch panel driver software. Note that the position information acquisition unit 1503 may be realized by an MPU, a memory, or the like. The position information acquisition unit 1503 is configured by means for acquiring position information in which the exhibit is instructed when the object to be instructed is a physical exhibit (for example, a dinosaur in a museum). In this case, for example, a button that can be instructed to the exhibit is installed, and the position information acquisition unit 1503 recognizes the pressing of the button and the pressed button, and acquires position information corresponding to the button.

  The object identifier acquisition unit 1504 acquires an object identifier that is information for identifying a user. Here, the user is a person and is a kind of object. Here, the user is a person who observes the exhibits and the display panels. The object identifier acquisition unit 1504 can be realized by the above-described IR tracker, for example. The object identifier acquisition unit 1504 may be software that receives an object identifier from the above-described IR tracker, for example.

  The already-explained information storage unit 1505 stores the object identifier acquired by the object identifier acquisition unit 1504 in association with the content unit while the content unit output unit 1508 outputs the content unit. The already-explained information storage unit 1505 may configure and store already-explained information having a content unit identifier for identifying a content unit and one or more object identifiers, for example. The data structure of the stored information in the already described information storage unit 1505 does not matter. The already-explained information storage unit 1505 may associate the object identifier of the already described (output) object with the output content unit. The already-explained information storage unit 1505 can usually be realized by an MPU, a memory, or the like. The processing procedure of the already-explained information storage unit 1505 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The content unit acquisition unit 1506 acquires one or more content units from the content unit storage unit 1501 when the trigger detection unit 1502 detects a trigger. The content unit acquisition unit 1506 may select and acquire content units at random each time an instruction is given. Further, the content unit acquisition unit 1506 may acquire a content unit corresponding to the position information acquired by the position information acquisition unit 1503 from the content unit storage unit 1501. The content unit corresponding to the location information may be a content unit that is paired with the location information acquired by the location information acquisition unit 1503, or the location information acquired by the location information acquisition unit 1503 and the location information within a predetermined distance. Paired content units may be used. The content unit acquisition unit 1506 may detect the distance to the user and select and acquire a content unit according to the distance. Such processing is performed by the distance information acquisition unit 15061 and the content unit acquisition unit 15062. Moreover, it is preferable that the content unit acquisition unit 1506 does not acquire the content unit associated with the object identifier acquired by the object identifier acquisition unit 1504. That is, the content unit once output is not output again to the same user (object). The content unit acquisition unit 1506 can usually be realized by an MPU, a memory, or the like. The processing procedure of the content unit acquisition unit 1506 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The agent storage unit 1507 stores one or more agents. An agent is a symbol or mark that reminds an exhibitor of an exhibit or a visitor, and the data structure is not limited. The agent is, for example, bitmap data. The agent storage unit 1507 is preferably a non-volatile recording medium, but can also be realized by a volatile recording medium. It may be a non-volatile recording medium or a volatile recording medium.

  The content unit output unit 1508 outputs the content unit acquired by the content unit acquisition unit 1506. The content unit output unit 1508 normally outputs an agent when outputting a content unit. However, agent output is not mandatory. Here, output is a concept including display on a display, printing on a printer, sound output, transmission to an external device, and the like. The content unit output unit 1508 may or may not include an output device such as a display or a speaker. The content unit output unit 1508 can be implemented by output device driver software, or output device driver software and an output device.

  The display information storage unit 1509 stores display information that is information to be displayed. Display information is information that constitutes an exhibition panel. If the object to be displayed is an exhibit (for example, a dinosaur in a museum), the display information storage unit 1509 and the display unit 1510 may be unnecessary. The display information storage unit 1509 is preferably a nonvolatile recording medium, but can also be realized by a volatile recording medium. It may be a non-volatile recording medium or a volatile recording medium.

  A display unit 1510 displays display information in the display information storage unit 1509. The display unit 1510 may be considered as including or not including a device such as a display or a touch panel. The display unit 1510 can be realized by device driver software, or device driver software and an output device.

  The distance information acquisition unit 15061 acquires distance information that is information regarding the distance between an exhibit (for example, a display on which display information is displayed) and the user. The distance information acquisition unit 15061 can be realized by, for example, an ultrasonic receiver and software for calculating the distance. The distance information acquisition unit 15061 is software that detects an approximate distance depending on whether an infrared signal transmitted from two or more IR tags (IR tags held by a user) having different communication distances is received. It may be realized. There are various means for calculating the distance between two objects (here, the displayed object and the user), and any means may be used.

The content unit acquisition unit 15062 acquires one or more content units from the content unit storage unit 1501 based on the distance information acquired by the distance information acquisition unit 15061. The content unit acquisition unit 15062 can be usually realized by an MPU, a memory, or the like. The processing procedure of the content unit acquisition unit 15062 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).
Next, the operation of the information output apparatus will be described with reference to the flowcharts of FIGS.

(Step S1601) The trigger detection unit 1502 determines whether or not a trigger for outputting the content unit has been detected. If a trigger is detected, the process goes to step S1602, and if no trigger is detected, the process returns to step S1601. Such a trigger is, for example, pressing on the touch panel or a user (visitor) approaching the exhibit within a predetermined distance.
(Step S1602) The content unit acquisition unit 1506 acquires a content unit. Details of the content unit acquisition processing will be described with reference to the flowchart of FIG.

  (Step S1603) The content unit output unit 1508 acquires an agent corresponding to the content unit acquired in step S1602 from the agent storage unit 1507. The agent corresponding to the content unit is, for example, the following agent. The agent storage unit 1507 stores, for example, an explanation agent agent (having, for example, “0” as an attribute value) and a visitor agent (having, for example, “1” as an attribute value). If the content unit acquired in step S1602 is a lecture content unit, an agent for an explanatory staff with attribute value “0” is acquired. If the content unit acquired in step S1602 is an interaction content unit, an agent for an explanatory member with attribute value “0” and an agent for a visitor with attribute value “1” are acquired. At the time of agent output, two types of agents are arranged and output corresponding to the respective voices and the like.

  (Step S1604) The content unit output unit 1508 configures content to be output based on the content unit acquired in step S1602 and the agent acquired in step S1603. For example, the content unit output unit 1508 configures display information to be output by overlaying an agent on the display information, and configures content having the display information to be output and sound to be output.

  (Step S1605) The content unit output unit 1508 starts outputting the content unit configured in step S1604. Here, the content unit output unit 1508 displays, for example, display information on which the agent is superimposed on a display, and outputs sound to a speaker. Here, it is assumed that the output of the content unit is performed in parallel with the processing in the following steps.

  (Step S1606) The object identifier acquisition unit 1504 acquires an object identifier which is information for identifying a user. This user is a visitor of a display object (for example, a display on which display information is displayed), and is a person who is near the display object.

  (Step S1607) The already-explained information storage unit 1505 determines whether or not the object identifier has been acquired in step S1606. If the object identifier can be acquired, the process proceeds to step S1608. If the object identifier cannot be acquired, the process jumps to step S1609.

  (Step S1608) The already-explained information storage unit 1505 temporarily stores the object identifier acquired in Step S1606. The temporarily storing recording medium is a recording medium such as a main memory or a hard disk (not shown).

  (Step S1609) The already-explained information storage unit 1505 determines whether or not the output of the content unit started in step S1605 has ended. If the output of the content unit is completed, the process goes to step S1610, and if the content unit is being output, the process returns to step S1606.

  (Step S1610) The already-explained information storage unit 1505 has already explained (has listened to the output content unit) or has already been explained with respect to the output content unit based on the one or more object identifiers temporarily stored in step S1608. And the object identifier of the object that is seen). Such a determination is performed by the following algorithm, for example. The already-explained information accumulation unit 1505 is an object identifier accumulated within a predetermined time (for example, 5 seconds) after the start of content output, and accumulated within a predetermined time (for example, 8 seconds) before the end. The object identifier is determined as the object identifier of the object that has already been described. The already-explained information storage unit 1505 may determine all the object identifiers temporarily stored in step S1608 as the object identifiers of the objects that have already been described. Furthermore, the already-explained information storage unit 1505 may determine the object identifiers detected at all predetermined intervals from the start of output of the content unit to the end of output as the object identifiers of objects that have already been described. In addition, the algorithm for determining the object identifier is not limited. Note that the number of object identifiers determined in step S1610 is not necessarily one. The object identifier determined in step S1610 may be 2 or more, or 0. If the object identifier determined in step S1610 is 0, the processes in steps S1611 and S1612 are not performed.

(Step S1611) The already-explained information storage unit 1505 configures already-explained information based on the object identifier determined in Step S1610 and the content unit acquired in Step S1602. The already described information includes, for example, a content unit identifier for identifying a content unit and one or more object identifiers.
(Step S1612) The already-explained information accumulation unit 1505 accumulates the already-explained information configured in step S1611.
Note that the method of associating the output content unit with the already explained (output) person (object) is not limited to the processing in FIG.
Further, in the flowchart of FIG. 16, the process is terminated by power-off or a process termination interrupt.
Next, details of the content unit acquisition processing will be described with reference to the flowchart of FIG.

(Step S1701) The object identifier acquisition unit 1504 acquires an object identifier which is information for identifying a user. Here, it is assumed that the object identifier acquisition unit 1504 acquires one object identifier.
(Step S1702) The position information acquisition unit 1503 acquires position information that is information regarding the position of the instruction detected by the trigger detection unit 1502.
(Step S1703) The distance information acquisition unit 15061 acquires distance information, which is information regarding the distance between an exhibit (for example, a display on which display information is displayed) and the user.
(Step S1704) The content unit acquisition unit 15062 substitutes 1 for the counter i.

  (Step S1705) The content unit acquisition unit 15062 determines whether or not the i-th content unit exists in the content unit storage unit 1501. If the i-th content unit exists, the process goes to step S1706, and if the i-th content unit does not exist, the process returns to the upper function. If the content unit cannot be acquired and the process returns to the upper function, the processing after step S1703 in FIG. 16 is not performed. That is, the content unit is not output.

  (Step S1706) The content unit acquisition unit 15062 determines whether or not the i-th content unit satisfies the condition of the position information acquired in step S1702. A specific example of such determination processing will be described below. The content unit has position information as its attribute value. In other words, the position information is information indicating the position of the information (exhibit, display information, etc.) described or discussed by the content unit, such as information to be described or discussed. Then, the content unit acquisition unit 15062 determines whether or not the position information that is the attribute value of the content unit has a predetermined relationship with the position information acquired in step S1702. The predetermined relationship means, for example, matching or within a predetermined distance. Here, if the two pieces of position information have a predetermined relationship, the content unit acquisition unit 15062 determines that the condition of the position information is satisfied. If the condition of the position information is satisfied, the process goes to step S1707. If the condition of the position information is not satisfied, the process jumps to step S1710.

  (Step S1707) The content unit acquisition unit 15062 determines whether or not the i-th content unit satisfies the condition of the distance information acquired in step S1703. Specifically, for example, the content unit acquisition unit 15062 first acquires the output time of the i-th content unit. The output time may be stored as an attribute value of the content unit, or may be calculated from the data length of the content unit. Next, the content unit acquisition unit 15062 determines the output time (usually having a width) of the content unit to be output from the distance information. If the output time of the i-th content unit satisfies the determined output time, the distance information condition is satisfied. In other words, in this example, a user who is close to an exhibit can determine that he / she is interested in the exhibit, so a long content unit may be output, but a user who is away from the exhibit may Since it can be determined that the exhibit is not very interested, a short content unit is selected and output. If the condition for distance information is satisfied, the process goes to step S1708. If the condition for distance information is not satisfied, the process goes to step S1710.

(Step S1708) The content unit acquisition unit 15062 determines whether or not the i-th content unit has already been output to the user identified by the object identifier acquired in step S1701. If it is determined that it has not been output, the process goes to step S1709. If it is determined that it has already been output, the process goes to step S1710.
(Step S1709) The content unit acquisition unit 15062 acquires the i-th content unit. Return to upper function.
(Step S1710) The content unit acquisition unit 15062 increments the counter i by 1. The process returns to step S1705.

  In the flowchart of FIG. 17, the content unit is acquired based on the three conditions of position information, distance information, and whether or not it has already been output, but the content unit that satisfies one or two conditions is acquired. You may do it.

In the flowchart of FIG. 17, only one object identifier is handled. However, two or more object identifiers may be handled. Handling two or more object identifiers means acquiring two or more object identifiers in step S1701, and whether or not all of the two or more acquired object identifiers have already been output. The content unit is acquired when one or more of the three conditions are satisfied. It should be noted that any one of the three conditions may be used to acquire the content unit.
Hereinafter, a specific operation of the information output apparatus in the present embodiment will be described.

  FIG. 18 is a content unit management table stored in the content unit storage unit 1501 of the information output apparatus. The content unit storage unit 1501 stores a content unit management table and raw content. Examples of raw content are shown in FIGS. The content unit management table includes “ID”, “content unit type ID”, and “content unit information”. The “content unit information” is information related to the content unit, and has “object identifier”, “start point (t)”, “end point (t)”, and “position information (x, y)”. “ID” is information for identifying a record and exists for record management in the table. The “content unit type ID” is information for identifying the type of content unit. Here, for example, “0” is a lecture content unit and “1” is an interaction content unit. “Object identifier” is information for identifying the object to which the raw content is sent. “Start point (t)” and “End point (t)” are a start point and an end point for cutting out the content unit from the raw content. Here, the start point and the end point are time information (relative time may be used or absolute). It may be a good time). “Position information (x, y)” is, for example, a coordinate position on the touch panel, and is a coordinate position designated by the user identified by the “object identifier”.

  FIG. 19 shows an example of agents stored in the agent storage unit 1507. Here, an agent for an explanation member with “ID = 0” and an agent for a visitor with “ID = 1” are included. Note that the agent storage unit 1507 may include other agents. By having various agents, the agent may be changed depending on whether the content unit to be output is a lecture content unit or an interaction content unit, or the agent may be changed depending on the size and type of the display to be output. .

  FIG. 20 is an already explained information management table having already explained information accumulated by the already explained information accumulating unit 1505. The already-explained information management table has “ID”, “content unit ID”, and “object identifier”. Each record of the already-explained information management table indicates that the content unit identified by the “content unit ID” has already been output to the person identified by the “object identifier”.

  FIG. 21 is a distance-time relationship management table held by the content unit acquisition unit 15062. The distance / time relationship management table is a table for managing the relationship between the distance between the user and the exhibit and the output time of the output content unit. The distance time relationship management table stores one or more records having “ID”, “distance (m)”, and “output time (seconds)”. “Distance (m)” is the distance between the user and the exhibit, and “Output time (seconds)” is the attribute value of the output time of the output content unit. In the record of “ID = 1” in the distance-time relationship management table, when the distance between the user and the exhibit is “0 m or more and less than 1.0 m” (close), a content unit (long content) with an output time of 100 seconds or more ) Is selected.

  In this case, it is assumed that the visitor with the object identifier “63” passes in front of the exhibit (in this case, the display panel) and stops. Then, the object identifier acquisition unit 1504 of the information output device acquires the object identifier “63”. It is assumed that the object identifier “63” is acquired by the information acquisition device installed in the exhibit.

  Next, it is assumed that the visitor with the object identifier “63” approaches the display panel and touches the position (20, 90) of the display of the display panel having the touch panel. And the trigger detection part 1502 detects the trigger which is pressing-down of a touch panel. Next, the position information acquisition unit 1503 acquires position information (20, 90).

  Next, the content unit acquisition unit 1506 acquires a content unit by the following processing. That is, the distance information acquisition unit 15061 acquires the distance “0.5 m” between the display panel and the visitor with the object identifier “63”. Then, the content unit acquisition unit 15062 acquires the first content unit from the content unit storage unit 1501. Here, the content unit information of “ID = 1” is acquired from the content unit management table of FIG.

  Next, the content unit acquisition unit 15062 determines whether or not the position information (20, 90) and the position information of the content unit information with “ID = 1” satisfy a predetermined condition. Here, the predetermined condition is that the distance between any one piece of position information of the content unit information and the acquired position information (20, 90) is within 10 or less. Here, since the distance between one position (20, 88) of the content unit information and the acquired position information (20, 90) is “2”, the condition is satisfied.

  Next, the content unit acquisition unit 15062 acquires the content output time “100” from the end point “500” and the start point “400” of the content unit information of “ID = 1”. Then, the content unit acquisition unit 15062 searches the distance-time relationship management table of FIG. 21 using the distance “0.5 m” between the display panel and the visitor with the object identifier “63” as a key, and outputs the output time “100 or more”. Get. Then, the content unit acquisition unit 15062 detects that the content unit information of “ID = 1” satisfies the distance condition.

Next, based on the already-explained information management table of FIG. 20, the content unit acquisition unit 15062 describes the content unit corresponding to the content unit information of “ID = 1” by the visitor with the object identifier “63”. It is determined whether or not it has been received. Since the record “ID = 1” with the content unit ID “1” and the object identifier “63” exists in the already described information management table of FIG. 19, the visitor with the object identifier “63” has already “ID = 1”. It is determined that the description of the content unit corresponding to the content unit information is received.
Then, the content unit acquisition unit 15062 does not select the content unit corresponding to the content unit information of “ID = 1”.
Next, the content unit acquisition unit 15062 acquires content unit information of “ID = 2” from the content unit management table.

Then, the content unit acquisition unit 15062 determines whether or not the content unit information of “ID = 2” satisfies the conditions such as the position information and the distance information described above. Here, since the content unit information of “ID = 2” does not satisfy the condition of the position information (has no position information), the content unit acquisition unit 15062 corresponds to the content unit information of “ID = 2”. No content unit is selected.
Next, the content unit acquisition unit 15062 acquires content unit information of “ID = 3” from the content unit management table.

  Such content unit information of “ID = 3” satisfies the conditions of position information and distance information. Furthermore, based on the already-explained information management table of FIG. 19, it can be determined that the visitor with the object identifier “63” has not received the explanation of the content unit corresponding to the content unit information with “ID = 3”. That is, the already-explained information management table does not have a record with the content unit ID “3” and the object identifier “63”. Then, the content unit acquisition unit 15062 selects and acquires the content unit corresponding to the content unit information of “ID = 3”.

  Next, the content unit output unit 1508 acquires an agent corresponding to the previously acquired content unit from the agent storage unit 1507. In other words, here, the content unit output unit 1508 acquires the agent for the instructor with ID “0” in FIG. Then, the content unit output unit 1508 configures content to be output based on the acquired content unit and the acquired agent. Here, the content unit output unit 1508 configures display information to be output by overlapping an agent on the display information, and configures content having the display information to be output and sound to be output.

Next, the content unit output unit 1508 starts outputting the configured content unit. An example of such output is shown in FIG. In FIG. 22, the sound of the content unit is also output. In addition, the content unit output unit 1508 displays not only the agent but also a character string that recognizes the voice in a balloon form. The character string may be a character string such as “XXXXYYYY...” Appropriately without performing voice recognition. Note that the voice recognition technique and the technique for outputting a speech balloon are known techniques, and thus detailed description thereof is omitted. As a technique for outputting a speech balloon, for example, it may be possible to hold a speech balloon pattern in advance and output it around the mouth of the agent (which also manages the coordinates of the mouth).
Note that the content unit acquisition unit 15062 may select only the lecture content unit having the content unit type ID “0” when there is an instruction to the touch panel.
Next, it is assumed that the visitor with the object identifier “8” passes near the display panel and stops.

Then, the information output device detects that the user has stayed nearby for a predetermined time or more. This is because the trigger detection unit 1502 detects the trigger because the IR tracker installed in the exhibit continues to receive the object identifier “63” for a predetermined time or more.
Next, the distance information acquisition unit 15061 acquires the distance “0.3 m” between the display panel and the visitor with the object identifier “8”.
Then, based on the distance “0.3 m”, the output time “100 seconds or more” is acquired.

Next, the content unit acquisition unit 15062 examines the content unit information in order from the content unit management table, and the visitor with the object identifier “8” that is the content unit that matches the output time “100 seconds or more” A content unit that has not been explained in the past (excluding a content unit identified by a content unit ID corresponding to the object identifier “8” in FIG. 20) is acquired.
Here, for example, the content unit acquisition unit 15062 acquires the content unit corresponding to the ID “72” content unit information in FIG.

  Then, the content unit output unit 1508 acquires an agent corresponding to the previously acquired content unit from the agent storage unit 1507. That is, here, the content unit output unit 1508 acquires the agent for the explanation member with ID “0” and the agent for the visitor member with ID “1” in FIG. 19. Then, the content unit output unit 1508 configures content to be output based on the acquired content unit and the acquired agent. Here, the content unit output unit 1508 configures display information to be output by overlapping an agent on the display information, and configures content having the display information to be output and sound to be output.

Next, the content unit output unit 1508 starts outputting the configured content unit. An example of such output is shown in FIG. Note that the output example is not limited to FIG. 23, and as shown in FIG. 24, the video (right side of the window) of the content unit may be output. That is, the content unit output unit 1508 outputs audio information included in the content unit, but it does not matter which information is output among other information. Further, the content unit output unit 1508 may or may not output the agent.
As described above, according to the present embodiment, the information output device outputs a content knit that matches a predetermined condition.

  Note that according to the present embodiment, the predetermined condition is a combination of the above-described position information, distance information, and three conditions indicating whether or not an explanation has been received, but other conditions may be considered. . For example, content that has a long output time is not output just before the end of the exhibition. In such a case, it is assumed that the information output device stores information regarding the time that can be determined to be just before the end of the exhibition.

Furthermore, the software that realizes the information output apparatus according to the present embodiment is the following program. That is, this program stores in the computer one or more content units that are contents related to explanations or discussions on the displayed object, and a trigger detection step for detecting a trigger of output of the content unit, and the trigger detection step A program for executing a content unit acquisition step of acquiring one or more content units from the content unit and a content unit output step of outputting the content unit acquired in the content unit acquisition step when a trigger is detected in is there.
Furthermore, in the program, a display step for displaying the display information may be further executed.
In the above program, the trigger detected in the trigger detection step may be an instruction for the display information displayed in the display step.

  In the above program, the content unit has audio information and position information, and causes the computer to further execute a position information acquisition step of acquiring position information that is information regarding the position of the instruction detected in the trigger detection step. The content unit acquisition step may be a program for acquiring a content unit corresponding to the position information acquired by the position information acquisition unit.

  In the program, in the content unit acquisition step, a distance information acquisition substep for acquiring distance information that is information on a distance between the display on which the display information is displayed and a user, and the distance information acquisition substep. The program may include a content unit acquisition substep for acquiring one or more content units based on the acquired distance information.

In the above program, the object identifier acquisition step for acquiring an object identifier that is information for identifying the user to the computer, and the content unit output in the content unit output step are acquired in the object identifier acquisition step. In the content unit acquisition step, the content unit associated with the object identifier acquired in the object identifier acquisition step is further executed. It may be a program that is not acquired.
In the above program, one or more agents may be stored, and the content unit output step may output the agent when the content unit is output.

  FIG. 25 shows the external appearance of a computer that executes the program described in this specification to realize the information generation apparatus and / or the information output apparatus according to the various embodiments described above. The above-described embodiments can be realized by computer hardware and a computer program executed thereon. FIG. 25 is an overview diagram of the computer system 250, and FIG. 26 is a block diagram of the computer system 250.

  In FIG. 25, a computer system 250 includes a computer 251 including a FD (Flexible Disk) drive and a CD-ROM (Compact Disk Read Only Memory) drive, a keyboard 252, a mouse 253, a monitor 254, a touch panel 255, and a microphone 256. And a speaker 257 and an IR tracker 258.

  26, in addition to the FD drive 2511 and the CD-ROM drive 2512, the computer 251 includes a CPU (Central Processing Unit) 2513, a bus 2514 connected to the CPU 2513, the CD-ROM drive 2512, and the FD drive 2511, and a boot. A ROM (Read-Only Memory) 2515 for storing programs such as an up program, and a RAM (Random Access Memory) connected to the CPU 2513 for temporarily storing instructions of application programs and providing a temporary storage space 2516 and a hard disk 2517 for storing application programs, system programs, and data. Although not shown here, the computer 251 may further include a network card that provides connection to the LAN.

  A program that causes the computer system 250 to execute the functions of the information generation apparatus and the information output apparatus according to the above-described embodiments is stored in the CD-ROM 2601 or the FD 2602, inserted into the CD-ROM drive 2512 or the FD drive 2511, Further, it may be transferred to the hard disk 2517. Alternatively, the program may be transmitted to the computer 251 via a network (not shown) and stored in the hard disk 2517. The program is loaded into the RAM 2516 when executed. The program may be loaded directly from the CD-ROM 2601, the FD 2602, or the network.

  The program does not necessarily include an operating system (OS) or a third-party program that causes the computer 251 to execute the functions of the information generation apparatus and the information output apparatus according to the above-described embodiments. The program only needs to include an instruction portion that calls an appropriate function (module) in a controlled manner and obtains a desired result. How the computer system 250 operates is well known and will not be described in detail.

  In each of the above embodiments, each process (each function) may be realized by centralized processing by a single device (system), or by distributed processing by a plurality of devices. May be. That is, for example, the information generation device described in the first embodiment and the information output device described in the second embodiment may be realized as a single system (information processing system).

In each of the above embodiments, each component may be configured by dedicated hardware, or a component that can be realized by software may be realized by executing a program. For example, each component can be realized by a program execution unit such as a CPU reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory.
Further, the computer that executes the program may be singular or plural. That is, centralized processing may be performed, or distributed processing may be performed.
Further, in each of the above embodiments, it goes without saying that two or more communication means (such as an information transmission unit) existing in one apparatus may be physically realized by one medium.
The present invention is not limited to the above-described embodiments, and various modifications are possible, and it goes without saying that these are also included in the scope of the present invention.

  As described above, the information output device according to the present invention has an effect of appropriately outputting the content unit related to the explanation or discussion of the displayed object, and is useful as a presentation device or the like.

Block diagram of an information generation apparatus according to Embodiment 1 A flowchart for explaining the operation of the information generation apparatus A flowchart for explaining the operation of the information generation apparatus A flowchart for explaining the operation of the information generation apparatus A flowchart for explaining the operation of the information generation apparatus A flowchart for explaining the operation of the information generation apparatus Explanatory drawing of extraction processing of the same interaction content unit The figure which shows the example of the same information acquisition device The figure which shows the conceptual diagram of the raw content which the same information acquisition apparatus acquired The figure which shows the conceptual diagram of the raw content which the same information acquisition apparatus acquired The figure which shows the example of the typical description information Schematic diagram showing the typical explanation instructions The figure which shows the example of the same content unit information The figure which shows the example of an input screen of the same content unit output instruction | indication Block diagram of an information output apparatus according to Embodiment 2 Flow chart for explaining the operation of the information output device Flow chart for explaining the operation of the information output device Figure showing the same content unit management table Figure showing the agent management table Figure showing the same explanation information management table Figure showing the same distance time relationship management table The figure which shows the output example of the same information output device The figure which shows the output example of the same information output device The figure which shows the output example of the same information output device External view of the computer Block diagram of the computer system

Explanation of symbols

101 Target identifier storage unit 102, 1509 Display information storage unit 103, 1510 Display unit 104 Instruction reception unit 105, 1503 Position information acquisition unit 106 Video information reception unit 106 Video reception unit 107 Audio information reception unit 108, 1504 Object identifier acquisition unit 109 Content unit extraction unit 110 Content unit storage unit 111, 1508 Content unit output unit 1091 Explanation voice detection means 1092 Typical explanation information storage means 1093 Similarity detection means 1094 Lecture content unit extraction means 1095 Dialogue voice detection means 1096 Interaction content unit extraction means 1501 Content unit storage unit 1502 Trigger detection unit 1505 Already described information storage unit 1506 Content unit acquisition unit 1507 Agent Storage unit 15061 distance information acquisition unit 15062 content units acquiring means

Claims (8)

It is an information generation device that accumulates content units, which are part of content, from raw content related to explanations or discussions on objects being exhibited,
A voice information receiving unit that receives voice information that is information having voice;
A content unit extraction unit that extracts content units having audio information for a time that matches a predetermined condition based on the audio information received by the audio information reception unit;
An information generation apparatus comprising a content unit storage unit that stores content unit information, which is information about content units extracted by the content unit extraction unit. A display information storage unit that stores display information that is information to be displayed;
A display unit for displaying the display information;
An instruction receiving unit for receiving an instruction for display information displayed by the display unit;
A position information acquisition unit that acquires position information that is information related to the position of the instruction received by the instruction reception unit;
The content unit extraction unit includes:
Based on the instruction received by the instruction receiving unit and the audio information received by the audio information receiving unit, audio information for a time that matches a predetermined condition, and the position information acquisition unit acquired during the time The information generation apparatus according to claim 1, wherein a content unit having position information is extracted. The content unit extraction unit includes:
Explanation voice detecting means for detecting explanation voice, which is voice determined to be spoken by a single speaker for a predetermined time or more,
Typical explanation information storage means for storing typical explanation information having one or more pieces of position information indicating the position of an instruction with respect to the display information;
A description corresponding to the one or more position information acquired by the position information acquisition unit based on the one or more position information acquired by the position information acquisition unit and the one or more position information included in the typical description information; Similarity detection means for detecting the similarity to the explanation corresponding to the explanation information, the explanation voice detection means detects the explanation voice, and the similarity detected by the similarity detection means is equal to or higher than a predetermined similarity 3. The information generating apparatus according to claim 1, further comprising: a lecture content unit extracting unit that extracts a lecture content unit having audio information corresponding to the time of the explanation audio when the degree of similarity is. The content unit extraction unit includes:
Dialogue voice detecting means for detecting dialogue voice having a voice paired with one speaker identifier and a voice paired with another substantially continuous speaker identifier;
3. The information generating apparatus according to claim 1, further comprising: an interaction content unit extracting unit that extracts an interaction content unit having audio information corresponding to a time of the dialogue voice when the dialogue voice detecting unit detects the dialogue voice. . A content unit storage unit that stores at least one content unit that is content related to an explanation or discussion of an object on display;
A trigger detection unit for detecting a trigger of output of the content unit;
A content unit acquisition unit that acquires one or more content units from the content unit storage unit when the trigger detection unit detects a trigger;
An information output device comprising a content unit output unit that outputs the content unit acquired by the content unit acquisition unit. A display information storage unit that stores display information that is information to be displayed;
The information output apparatus according to claim 5, further comprising a display unit that displays the display information. A program for accumulating content units, which are part of content, from raw content related to explanations or discussions about objects on display,
On the computer,
A voice information receiving step for receiving voice information which is information having voice;
A content unit extraction step for extracting a content unit having audio information for a time that matches a predetermined condition based on the audio information received in the audio information reception step;
A program for executing a content unit accumulation step for accumulating content unit information, which is information relating to the content unit extracted in the content unit extraction step. On the computer,
Contains one or more content units that are content related to the explanation or discussion of the objects on display,
A trigger detection step for detecting an output trigger of the content unit;
A content unit acquisition step of acquiring one or more content units from the content unit when a trigger is detected in the trigger detection step;
A program for executing a content unit output step of outputting the content unit acquired in the content unit acquisition step.

Images