JPH1153385A - Conference information recording reproducing device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make it possible to arrive at desired information as efficiently as possible in a conference information recording reproducing device that visually expresses a structure of comment at a conference as a speaker chart and uses it as an index of an access to conference information. SOLUTION: A structure of comments by plural conference participants is extracted from sound data recorded and this structure of comments is displayed as a speaker chart on a display picture. A retriever executes a comment or a reproduction section that he or she wants to reproduce through an instruction input means 12 on the speaker chart. Reproduction means 13 and 14 reproduce sound data corresponding to the instructed comment or the reproduction section. An intention extraction means 17 extracts an intention of an instruction operation of the retriever by using values of attribute such as name of the speaker, time of speech, name of the previous speaker, name of the following speaker and so on that are predetermined for extracting the intention. A similar comment candidate having an intention similar to the comment instructed by the instruction input means is detected and the detected similar comment candidate is visually displayed on a display device.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an apparatus for recording and reproducing conference information such as audio information or video information in a conference. In particular, the present invention relates to an apparatus that can efficiently search for an access location suitable for the searcher's intention as much as possible.

[0002]

2. Description of the Related Art In a conference, a great deal of information is generated as speech information by conversation. Of these, only a small amount of information is recorded as text information on whiteboards or minutes, and there is a problem that many important information is not recorded or cannot be accurately remembered.

To solve this problem, there is a conference recording device for recording all information generated in a conference, and an example of this conference recording device is described in Japanese Patent Laid-Open No. Hei 6-343146. Here, all multimedia information such as audio information input from a microphone, video information input from a video camera, text information and graphic information input from a pen, and the like are recorded without exception.

In such a meeting recording device, when trying to remember the contents of a meeting, it is important how to properly access a necessary place. However, it is extremely difficult for participants to attach an index to each conference scene in real time. In this regard, effective indexing is possible if proper indexing is performed manually by humans after the meeting.

[0005] However, such indexing work is enormous. Further, the information required later often changes depending on each person who performs the search or over time, and it is difficult to perform a sufficient search using a predetermined index. Therefore, a method of automatically providing an effective index from various kinds of clue information generated during a meeting without any human intervention is being studied.

Japanese Patent Application Laid-Open No. Hei 6-343146 provides a means for retrieving audio and video information by using a time when a text or gesture is input by an input means using a pen as an index. Meeting participants often take handwritten notes when significant remarks occur. From this,
By using the time when the handwritten memo was performed as an index,
It is possible to access important information of the conference effectively.

[0007] However, there is a problem that a conference participant cannot take notes when he is enthusiastic about the discussion. Therefore, while indices that require such active instructions and actions of conference participants are many effective, there are many omissions. Also, if you want to create a sufficient index, meeting participants have to take a lot of notes, increasing the burden. In addition, if sufficient memo writing exists, there is a contradiction that the necessity of multimedia recording is reduced.

[0010] Several other methods have been studied as methods for automatically extracting a sufficient index while minimizing the burden on conference participants as much as possible. JP-A-2-
In Japanese Patent No. 113790, a search scene is extracted from a moving image by extracting features of image information, and a menu is displayed on the search scene. It is possible to efficiently access necessary data from image data. There are situations where such technology is effective even in meetings, such as "when a specific person goes out and talks on the blackboard." However,
In general, video information in a conference does not change much, and it is difficult to extract sufficient clues from the video information to recall the content of the conference.

[0009] The most important information in a conference is voice data from a conversation. Attempts have been made to extract a clue for retrieval from the voice data. Japanese Patent Application Laid-Open No. 3-250481 discloses that in order to access an image when a user falls into trouble from an image in which a user uses a tool, a keyword that is frequently issued at the time of the trouble is used to access the corresponding data. A method for accessing a place where is recorded is described. However, the situation is quite specific here and cannot be general clue information.

Japanese Unexamined Patent Publication No. Hei 6-236410 discloses an apparatus utilizing voice information. Here, the language of the speaker is analyzed, the topic of the utterance content and its field are identified, and a group of information suitable for the topic is automatically selected from the database.
Here, a topic conversion point and a topic candidate there are detected using a dictionary for speech expression. The turning point of the topic is
It is very important as a clue to access to meeting records.

[0011] However, although the topic turning point is important, there is a problem that fine-grained access is not possible because the granularity of the access clue information is too large. Furthermore, in order to find a practical topic turning point, it is difficult to cope with the natural speech recognition technology for natural utterance at the present time, and there is a high difficulty in enriching a dictionary for utterance expression.

On the other hand, Japanese Patent Application Laid-Open No. 8-317365 discloses a technique for graphing voice data of a speaker of a conference in a time series in a length corresponding to the amount of stored data. .
This makes it possible to visualize, as a graph, in which order and by whom and for how long. Hereinafter, in this specification, this statement structure diagram is referred to as a speaker chart.

From the speaker chart, a conference participant can recall to some extent the contents of the conference in which he or she participated even after the conference, and can access important or necessary information storage locations. It becomes possible. The advantages of this technology are that it does not require advanced speech recognition technology or dictionaries, and does not require explicit instructions from conference participants, and can be created automatically from only recorded information.

[0014]

However, in searching in a conference record using a speaker chart,
The following problems exist.

[0015] One is a problem caused by accessing "partial information" in the recorded conference information. Specifically, there is a problem of a loss of the absolute position of the access, which makes it impossible to know what information is currently being accessed. In addition, there is a problem that the relative location in the whole meeting is lost, because it is not known which side of the whole meeting is currently being accessed. In addition, there is a weakness against a repetition of logic development that trusts the accessed partial information to make a conclusion and later misses the information of the part that the conclusion has overturned.

The second point is that when an incorrect reproduction location is accessed, it is not possible to know where else the necessary information is located.

To solve these problems, Japanese Patent Application Laid-Open No. 8-317365
The issue does not address this issue. In contrast, Xerox PA
RC Audio browsing Tool (Donald G. Kimber, lynn D.Wi
lcox, Francine R. Chen, and Thomas Moran: "Speaker
Segmentation for BrowsingRecorded Audio ", CHI '9
5 Proceedings (short paper), pp. 212-213) indicates that the currently accessed location is explicitly shown on the speaker chart, and which part of the whole is displayed as the speaker chart By displaying the two pieces of information described above, the problem of the loss of the absolute and relative access positions, which is a problem caused by accessing the “partial information”, is solved.

However, two other problems remain. That is, during a meeting, the logical development can be two or three points, and when one accidentally accesses the first conclusion, it is easy to overlook the correct information that exists thereafter. Therefore, it is necessary to support such a change in logical development so as to eliminate access omission.

Also, the speaker chart itself is not necessarily 1
It is not an index that can access the location of necessary information exactly at the time. Actually, accuracy can be increased by being used in combination with a handwritten memo or the like. However, as mentioned earlier, handwritten notes are a heavy burden on participants, so it is more important to support how to find the appropriate information from a speaker chart with ambiguity. Become. That is, even if an incorrect place is accessed, information is needed so that it is possible to know on which side the required information exists.

In view of the above problems, the present invention provides a conference information recording / reproducing apparatus capable of visualizing and displaying a statement structure in a conference and using the visualized structure as an index for accessing recorded conference information. It is an object of the present invention to provide a device which can reduce the load on conference participants, reduce omission in retrieval, and reach desired information as efficiently as possible.

[0021]

In order to solve the above-mentioned problems, a conference information reproducing apparatus according to the first aspect of the present invention comprises:
Recording means for recording audio data when a plurality of conference participants hold a conference; statement structure extraction means for extracting a statement structure by the plurality of conference participants from the speech data; and visualizing the statement structure. Information generating means for generating visualization information for visualizing, a display device for visualizing the speech structure based on the visualization information, and an instruction input in the speech structure visualized on the display device Instruction input means for performing, a reproducing means for reproducing the audio data corresponding to the position or portion designated by the instruction input means, and an intention of the searcher's instruction operation through the instruction input means, Intention extraction means for extracting using the value of the attribute determined for extraction, similar candidate detection means for detecting a voice data section having an intention similar to the intention extracted by the intention extraction means, Serial characterized by comprising a similarity candidate display means for visualizing on the display device the detected similar candidates similar candidate detection means.

A conference information reproducing apparatus according to a second aspect of the present invention provides a conference information reproducing apparatus provided for each conference participant for inputting audio data of conference information, and storage means for storing the audio data. Utterance data extraction means for extracting utterances from the voice data, utterance structure table generation means for generating an utterance structure table from the extracted utterance data and a timer, storage means for storing the utterance structure table, Storage means for storing a conference participant table for holding a correspondence relationship between a voice input device and the conference participant, and a speaker chart generation for generating a speaker chart for visualizing the statement structure table on a display device Means, and a speaker chart display for displaying the speaker chart generated by the speaker chart generating means on the display device A step, on the speaker chart, instruction input means for instructing an arbitrary statement that the searcher intends to reproduce, statement specifying means for specifying the statement specified by the instruction input means, Intention extraction means for extracting the search intention of the searcher using the attribute value determined to extract the intention in advance, and information of the search intention of the searcher extracted by the intention extraction means. A similar utterance detecting means for detecting a similar utterance candidate having an intention similar to the utterance instructed by the instruction input means, and visualizing the similar utterance candidates detected by the similar utterance detecting means on a display device. And a similar utterance candidate display unit for displaying

According to a third aspect of the present invention, in the conference information recording / reproducing apparatus of the second aspect, the instruction intention extracting means includes: a speaker name, a speaking time, Previous speaker name, post speaker name 4
It is characterized in that the intention of the searcher is extracted from the values of the two attributes.

According to a fourth aspect of the present invention, there is provided the conference information reproducing apparatus according to the second aspect, wherein the similar utterance detecting means receives the instruction extracted by the instruction intention extracting means. An utterance similarity calculating unit that calculates a similarity between the intention of the utterance and another utterance in the utterance structure table, and the similarity calculated by the utterance similarity calculating unit is a similarity equal to or more than a predetermined value. And a utterance similarity determination unit that determines whether or not the utterance similarity has a degree. The similarity utterance candidate is detected based on a determination result of the utterance similarity determination unit.

According to a fifth aspect of the present invention, there is provided the conference information reproducing apparatus according to the second aspect, wherein the instruction input means allows the searcher to specify a reproduction section. The intention extraction means includes a reproduction operation monitoring means for monitoring a reproduction action of the searcher, and a reproduction intention for extracting a searcher's reproduction intention from a predetermined attribute value relating to a series of utterance groups in the reproduced audio data section. It is characterized by comprising extraction means.

According to a sixth aspect of the present invention, in the conference information recording / reproducing apparatus of the fifth aspect, the attribute value used in the reproduction intention extracting means is a series of the reproduced audio data section. , The intent of the reproduction start utterance of the utterance group, the name of the uttered utterer, the total number of utterances, the total utterance time, the utterer set, and the utterance transition matrix.

According to a seventh aspect of the present invention, in the conference information recording / reproducing apparatus of the fifth aspect, the similar utterance detecting means uses the reproduction intention from the reproduction intention extracting means. For another series of utterance groups in the utterance structure table, utterance structure similarity calculating means for calculating similarity of utterance structures, and similarity of the utterance structures calculated by the utterance structure similarity calculating means, And a utterance structure similarity determining unit that determines whether or not the utterance structure similarity is equal to or greater than a predetermined value, and detects the similar utterance structure candidate based on the determination result of the utterance structure similarity determining unit. It is characterized by the following.

In the conference information reproducing apparatus according to the present invention, in the conference information recording / reproducing apparatus according to the fifth aspect, the similar utterance detection means detects a similar utterance in accordance with a state of the reproduced utterance. A similarity determination method selecting means for automatically selecting a means and a similar utterance structure detecting means.

According to a ninth aspect of the present invention, in the conference information recording / reproducing apparatus according to the second aspect, the similar utterance candidate display means displays the conference time information in time series. It has two display areas, a conference time display area and a similar candidate reduced view display area for displaying a reduced view of a plurality of speech structures, and the entire meeting time display area has a display area from the instruction input device of the searcher. Means for displaying, as the partial display area, the reproduction section determined by the input instruction and the similar candidate of the reproduction section as the partial display area, and displaying the similar candidate reduced figure display area in the entire conference time display area List display means for displaying a reduced list of similar candidates obtained by reducing the comment structure of the section of the displayed partial display area by the number of the partial display areas, and further comprising: One of the similar candidate reduction diagram detects that the selected instruction by said searcher, characterized in that it comprises a means for reproducing the audio data of the selected designated interval.

[0030]

In the conference information recording / reproducing apparatus according to the first aspect of the present invention,
The speech structure is extracted from the voice input data of the conference information and recorded. Here, the utterance structure is, for example, extracting a utterance from voice input data,
It can be extracted by specifying the utterance end time and further specifying the utterance order. This comment structure is visualized on the display device by the visualization information generated by the visualization information generation means.

When an arbitrary position on the visualization information is designated by an instruction input means such as a pointing device such as a mouse, an arbitrary position of the conference information data recorded in audio and video is reproduced. Is done. At this time, the search behavior of the searcher is monitored, and the search intention of the searcher is automatically extracted from the search behavior. Then, it is detected whether or not there is a statement having an intention similar to the extracted searcher's intention with respect to the other part during the meeting, and the detected similar candidate is displayed on the display device.

Thus, similar candidates can be automatically presented to the searcher. This information indicates the presence of information to be accessed next when the search fails, and can support efficient search. In addition, even when the search is successful, the searcher is notified that other correct candidates exist, and the search leakage is reduced.

In the conference information recording / reproducing apparatus according to the second aspect of the present invention, the speech structure is extracted from the speech input data of the conference information, and the speech structure data is recorded. As means for visualizing the utterance structure data, for example, a utterer chart that displays utterance structure information such as a utterer, utterance time, and utterance transition information in a time-series manner is used.

When an arbitrary position on the speaker chart is designated and input by the searcher, the intention of the searcher is automatically extracted. Here, the instruction intention is an intention of a search relating to a specific utterance instructed and reproduced by the searcher, and is composed of characteristic values of a plurality of attributes related to the utterance. After the intention of the instruction statement is extracted, it is evaluated whether another statement in the statement structure data file has a statement having an intention similar to the instruction intention. When a similar utterance is detected, the utterance extracted as the similar utterance is visualized at a corresponding position on the utterer chart.

As a result, a statement having a structure similar to the search intention of the searcher of the conference information can be automatically extracted without additional input by the searcher. Further, by visually presenting a similar utterance candidate to the searcher, it becomes possible to inform the presence of the candidate.

In the conference information recording / reproducing apparatus according to the third aspect of the present invention, in the extraction of the instruction intention, the name of the speaker, the speaking time, the name of the previous speaker, and the name of the subsequent speaker related to the statement specified by the searcher through the instruction input. By extracting the four attribute values,
The intention of the instruction input performed by the searcher can be calculated. Thus, it is possible to extract an appropriate search intention with a small amount of information by specifying four representative attributes representing the design intention from the complicated structure of the searcher's intention. .

In the conference information recording / reproducing apparatus according to the fourth aspect of the present invention, the similarity with the instructed utterance other than the utterance instructed by the searcher is calculated. Then, by determining whether or not the similarity satisfies a certain criterion, a similar utterance is extracted. This makes it possible to automatically extract a statement similar to the statement that the searcher has instructed to reproduce.

In the conference information recording / reproducing apparatus according to the fifth or sixth aspect of the present invention, a search intention is automatically extracted not only from an instruction inputting action but also from a reproducing action by a searcher.

The searcher instructs an arbitrary statement on the speaker chart to reproduce the audio and video data in which the conference information is recorded. Then, after playing for a while, it is possible to perform a playing action of stopping the playing. Here, after the reproduction stop instruction is input, the reproduction section is specified, and both the instruction input intention and the reproduction intention are automatically extracted from the reproduction section. Extracting intent from the playback section is simply a matter of 1
This means that not only one utterance, but also a series of uttered utterances reproduced and its utterance structure extract a search intention.

Here, the reproduction intention is an attribute relating to the six statement structures of the instructing intention of the start statement, the name of the stop speaker, the total number of statements, the total statement time, the speaker set, and the statement transition matrix. Can be calculated by This makes it possible to infer the search intention of the searcher more accurately by comparing when only the instruction intention is used.

In the conference information recording / reproducing apparatus according to the seventh aspect of the present invention, the similarity between the reproduced section and other utterance structures generated during the conference other than the utterance structure of the reproduced section is calculated. It is determined whether this similarity satisfies a certain condition,
Those satisfying the conditions are detected as similar utterance structure candidates. Thus, a series of utterance groups having a utterance structure similar to the search intention of the searcher can be automatically extracted.

In the conference information recording / reproducing apparatus according to the eighth aspect of the present invention, it is determined from the search action of the searcher whether the searcher's intention is a specific statement or a series of statements, and an appropriate similarity is determined for each. Is automatically determined. This allows
Means for determining an appropriate similarity can be selected without additional input by the searcher, and more appropriate similar candidates can be presented.

In the conference information recording / reproducing apparatus according to the ninth aspect of the present invention, regarding a display method for presenting the detected similar candidates to a searcher, a display area showing the entire meeting in chronological order, and a statement structure of the similar candidates Has a display area that can be listed in a reduced display, so that the relative positional relationship of similar candidates in the meeting can be grasped on the time axis, and the details can be listed in a reduced display, so that the details of the remarks can be And two pieces of information about the relative positional relationship in time series can be organically connected and displayed.

As a result, the ability to recognize the speech structure is improved,
Searching can be performed more efficiently. In addition, by listening to or viewing reproduction information while referring to such information, it is possible to promote understanding of the reproduction contents.

[0045]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of a conference information recording / reproducing apparatus according to the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram showing a system configuration of a conference information recording / reproducing apparatus according to an embodiment of the present invention. The conference information recording / reproducing apparatus according to this embodiment records audio and video data as conference information, and has access means for accessing an arbitrary position of the recorded audio and video data file, and is accessed by this access means. It reproduces the audio and video data at the location.

In the conference information recording / reproducing apparatus according to this embodiment, it is possible to access an arbitrary position in the audio and video data file recorded as the conference information in response to a reproduction instruction of the searcher. It is assumed that the apparatus has an access index that visualizes a speech structure such as a speaker chart. Then, when the searcher gives a playback instruction through the speaker chart, the audio and video data corresponding to the designated position is of course played back, and the searcher's playback instruction is given on top of that. An intention is extracted, a search candidate similar to the intention is detected, and it is detected. By displaying the search candidate, omission of search by the searcher is reduced.

As shown in FIG. 1, the conference information recording / reproducing apparatus of this embodiment comprises a plurality of audio input devices 1a, a video input device 1b, and an A / D for audio signals from the audio input device 1a. Conversion device 2, audio data synthesis device 3, file storage unit 4, speaker chart generation control unit 5, display device 11, instruction input device 12, video playback device 13
And an audio playback device 14.

The speaker chart generation control unit 5 includes an utterance data extraction unit 6, a timer 7, an utterance structure table generation unit 8, an utterer chart generation unit 9, and a part of the utterer chart display unit 10. Prepare. Speaker chart search control unit 1
5 includes a statement specifying unit 16, a searcher intention extracting unit 17, a similar candidate detecting unit 18, a similar candidate display unit 19, and a part of the speaker chart display unit 10.

In this embodiment, the speaker chart generation controller 5 and the speaker chart search controller 15
Is a configuration of a computer processing device. That is,
Each unit of the speaker chart generation control unit 5 and the speaker chart search control unit 15 has a configuration of a functional unit realized by software of a computer.

The voice input device 1a is a device for inputting a voice of a conference participant, such as a microphone, and is assigned to each of the conference participants. The output audio signals of each of the plurality of audio input devices 1a are converted into digital signals in the A / D converter 2. This A / D
The plurality of digital audio data from the conversion device 2 are synthesized as audio data of all the conference participants by the audio data synthesizing device 3 and stored in the file storage unit 4 as an audio data file.

The video input device 1b comprises, for example, a digital video camera. Digital video data from the video input device 1b is stored in the file storage unit 4 as a video data file. The number of digital video cameras of the video input device 1b may be one or more.

FIG. 2 is a diagram for explaining a data file stored in the file storage unit 4. In this example, the file storage unit 4 stores four data files. The statement structure table 41 is a data file generated by extracting a statement structure of a conference participant in a conference from input voice data. This data holds information serving as an index for accessing the audio data file 43 and the video data file 44. Further, it is also data for generating a speaker chart. The comment structure table 41 will be described later in detail.

The audio data file 43 and the video data file 44 are data files for holding audio data and video data recorded as conference information. These audio data file 43 and video data file 4
4 holds a link relationship with the comment structure table 41. The conference participant table 42 is a data file for identifying a conference participant. The conference participant table 42 holds, as data, a relationship between an input device number assigned to each of the voice input devices 1a and a conference participant name. I have.

FIG. 3 is a diagram for explaining the data structure of the conference participant table 42. The conference participant table
It is a data file that holds the correspondence between conference participants and input device numbers. The field 42a is an input device number, and means a device number which is an identifier held by the voice input device 1a. The field 42b is a conference participant name, and the name of the conference participant assigned to each voice input device 1a is stored as text data.

The digital audio data from the A / D converter 2 for each of the plurality of audio input devices 1a is passed to the speaker chart generation control unit 5 and processed. The speaker chart generation control unit 5 includes a file storage unit 4
This is an apparatus for generating a speaker chart, which is one of access means for accessing an arbitrary position of the audio data file stored in the file. Details of the speaker chart generation processing will be described later.

The display device 11 visually displays the speaker chart generated by the speaker chart generation controller 5 on its screen. Also, the video reproduced by the video reproducing device 13 may be further displayed. That is,
Since the video reproducing device 13 includes a display unit, the reproduced video is displayed on the display unit, but may be displayed on the display screen of the display device 11. Of course, only the speaker chart may be displayed on the display device 11, and the video may be shared and displayed on the display unit of the video playback device 13.

The instruction input device 12 is for instructing an utterance or a utterance structure in the speaker chart displayed on the display screen of the display device 11, and is constituted by a mouse or a pointing device.

The video reproducing device 13 is a device for reproducing the video data of the portion specified by the user from the speaker chart in the video data file in the file storage unit 4. Similarly, the audio reproducing device 14 is a device that reproduces audio data of a portion specified by the user in the audio data file in the file storage unit 4. Using the speaker chart, an arbitrary portion of the video data can be reproduced by the video reproduction device 13 in synchronization with the audio data.

The speaker chart search controller 15 searches and reproduces voice data and image data corresponding to an arbitrary position designated by the instruction input device 12 on the speaker chart on the display screen of the display device 11. It is.

In the following description, retrieval of the designated audio data from the audio data file will be described for the sake of simplicity. However, the reproduction of the conference information data file is the same for video data.

First, the processing operation in the speaker chart generation control unit 5 will be described.

The digital voice data from the A / D converter 2 for each of the plurality of voice input devices 1 a is input to the utterance data extraction unit 6. The utterance data extracting unit 6 extracts a utterance section for each of the input voice data, assuming that a sound volume level exceeding a certain level continues for a certain time or more as a utterance, and uses the utterance section data in the utterance structure table. The information is transmitted to the generation unit 8. The utterance section data is composed of an input device number indicating which of the voice input devices 1a has voice data, utterance start timing, and utterance end timing information.

The comment structure table generating section 8 generates a comment structure table which is an access index to the audio data file in which the comments of the conference are recorded. That is,
From the speech section data from the speech data extraction unit 6 and the time information of the timer 7, the input device number, the speech start time,
Information about the speech section of the conference participant, such as the speech end time, is extracted, a speech structure table is generated, and stored in the file storage unit 4.

FIG. 4 is a diagram for explaining the data structure of the comment structure table. The speech structure table is a data file that holds the structure of speeches of conference participants in the conference and is used as an access index to an audio data file and a video data file in which conference information is recorded.

In FIG. 4, a field 51 is a statement number, and an identifier is assigned in order of the time of the statement. The field 52 is an input device number as an identifier of the voice input device 1a from which the utterance was detected. The field 53 is a speech start time, and records the start time of the detected speech as the elapsed time from the start of recording. The field 54 is the utterance end time, and the end time of the detected utterance is similarly recorded.

As described above, the audio data file 4
3 is associated with the comment structure table.
For example, a description will be given of the correspondence between the utterance number 7 and the utterance number 7 in FIG. 4. Reference numeral 56 denotes a recording location of the utterance number 7 recorded in the audio data file 43;
The link 55a points to the start point of the recording position of the utterance number 7. Similarly, the link 55b indicates the end point of the recording position of the utterance number 7.

The speaker chart generating section 9 receives the information of the statement structure table stored in the file storage section 4 and generates information of the speaker chart for visualizing and displaying the statement structure table. The generated speaker chart information is passed to the speaker chart display unit 10, and the speaker chart display unit 10 displays the speaker chart on the display device 11.

FIG. 5 is a diagram showing one embodiment of the speaker chart. Reference numeral 101 denotes a speaker chart display area. The speaker chart displays the entire conference time display area 102 displayed as an overview of the entire conference and the details of the statement structure of the portion corresponding to the detail display location 104 displayed in the entire conference time display area 102. The structure display area 103 includes two areas.

The total conference time display area 102 has a time display in which the start time of the conference is "00: 00: 00: 00" and the time from then to the end of the conference is displayed as a relative time. In the example of FIG. 5, the relative time on the way displays only the time at the intermediate point. The detailed display portion 104 indicates a specific time section of the total conference time.

The details of the comment structure in the time section indicated by the detail display portion 104 are displayed in the comment structure display area 103. In other words, the statement structure displayed in the statement structure display area 103 indicates which time section in the entire meeting time is based on the position of the detailed display location 104 in the entire meeting time display area 102. You can know.

The speech structure display area 103 includes a speaker name area 106 for displaying a speaker name for identifying the speaker,
And an utterance transition display area 107 for visually displaying the state of the utterance transition. As shown in FIG. 5, the start time and the end time of the section displayed in detail in the comment transition display area 107 of the comment structure display area 103 are also displayed, and the total conference time is displayed. Which time portion of the message structure is displayed in detail is displayed.

In each column of the speaker transition display area 107 for each speaker, each conference participant (speaker) displays during the conference time.
When and how long the utterance was made are indicated by the display position and length of the utterance section bar VB. By reading the utterance structure displayed as the transition of the utterance section bar for all the conference participants in the utterance transition display area 107, a detailed display portion 104 indicating which utterance has transitioned to which utterance is displayed. It becomes possible to read the utterance transition structure in the time section.

A triangle point 105a in the all meeting time display area 102 or a broken line 105b in the speech transition display area 107 in FIG. 5 indicates a time position on the speaker chart corresponding to the audio data being reproduced at that time.

By designating an arbitrary position on the speaker chart displayed on the display device 11 by the instruction input device 12, it is possible to reproduce an arbitrary position of the recorded audio data of the conference. Speaker chart search control unit 15
Retrieves and reproduces the voice data at the specified position.

The utterance identification unit 16 of the utterer chart search control unit 15 uses the position information specified on the display device 11
This is a device that performs a process of specifying a corresponding utterance (utterance section) in the utterance structure table 41 of the file storage unit 4. Then, as shown in FIG. 4, according to the index recorded in the statement structure table 41, the audio data file 4
3 is searched, audio data corresponding to the specified utterance (utterance section) is extracted from the audio data file 43, and is reproduced by the audio reproducing device 14.

The searcher intention extraction unit 17 extracts the intention (instruction intention) of the instruction input of the searcher who has performed the instruction input. Here, the instruction intention means a search intention of an instruction input when a user who is a searcher who wants to reproduce an arbitrary position of audio and video data has given an utterance to reproduce. In this embodiment, the intention of the searcher to specify is extracted from the four attributes related to the utterance, the name of the utterer regarding the utterance whose reproduction is instructed, the utterance time, the previous utterer's name, and the subsequent utterer's name. The first speaker name and the second speaker name are
This is an attribute related to the statement transition structure. Searcher intention extraction unit 1
7 retrieves the file storage unit 4 based on the information on the utterance specified by the utterance specifying unit 16 to obtain the above four attributes, and thereby extracts the instruction intention.

The similar candidate detecting unit 18 receives the information on the instruction intention extracted by the searcher intention extracting unit 17 and searches for a similar candidate that is a statement similar to the instruction intention. If there is a similar candidate, the similar candidate display unit 1
9 and send the information. In response to this, the similar candidate display unit 1
9 displays a similar candidate on the display device 11.

FIG. 6 is a diagram for explaining a method of instructing a comment that the searcher wants to reproduce. FIG. 6 shows an enlarged part of the speaker chart. The searcher designates an area corresponding to a statement to be reproduced by using a pointing device such as a mouse constituting the instruction input device 12.

FIG. 6 shows the number 1 in FIGS. 5 and 7.
A speech section bar of the speaker “Sato”, which is denoted by 08, is illustrated, and the position pointed by the instruction input device 12 is indicated by an arrow cursor 110. When an instruction is given by the instruction input device 12 such as a click of a mouse button at the position indicated by the arrow cursor 110, audio data corresponding to the speech section bar 108 is reproduced as described later.

FIG. 7 is a diagram for explaining the relative coordinate position of the searcher's instruction input position in the speaker chart display area 101. In this embodiment, the instruction input position is not treated as coordinates on the display device 11, but as relative coordinates in the speaker chart display area 101. FIG.
, 121 indicates the coordinates (0, 0) of the starting point in the speaker chart.

The meeting time corresponding to the start point (coordinates (0, 0)) of the section displayed in the comment transition display area 107 is represented by Origin. Further, the time width of the conference section corresponding to the portion displayed in the statement transition display area 107 is ΔTm, and the display width of the statement transition display area 107 is ΔXm. Therefore, the time width ΔTm has a value corresponding to the conference section displayed in the comment structure display area 103 at that time. ΔXm varies depending on the size of the display frame of the speaker chart display area 101 displayed at that time.

In FIG. 7, reference numeral 122 denotes an instruction input position of the searcher using the instruction input device 12, and the value of the meeting time corresponding to the instruction input position 122 is defined as Tpoint. Δx indicates the relative coordinates of the instruction input position 122 in the x direction (lateral direction) from the starting point 121 in the speaker chart display area 101.

The formula for calculating the instruction input time Tpoint is as follows: Tpoint = Torigin + ΔTm (Δx / ΔXm) (1)

Next, FIG. 8 is a flowchart showing the flow of processing in the speaker chart search control section 15.

In step 201, it is monitored whether or not there is a reproduction instruction input from a user who is a searcher. Step 202
Then, it is determined whether or not there is an instruction input, and if there is no instruction input, the process returns to step 201, and the monitoring of the user's instruction input is repeated.

When there is an instruction input from the user,
In step 203, the user's instruction input coordinates Ppoint
To win. This is an absolute coordinate on the display screen. Next, in step 204, the utterance corresponding to the instruction input position is specified. At this time, a process of converting the user's instruction input coordinate Ppoint acquired in step 203 into a relative coordinate position in the utterance transition display area 107 is also performed. The above processing is performed by the comment specifying unit 16.
The details of the processing in step 204 will be described later with reference to the flowchart in FIG.

In step 205, processing for extracting the intention of the specified utterance is performed. The processing in step 205 corresponds to the processing performed by the searcher intention extracting unit 17. This step
Details of the processing of 205 will be described later using the flowchart of FIG.

In the next step 206, a process for detecting a utterance candidate similar to the extracted instruction intention is performed. The process of step 206 is performed by the similar candidate detecting unit 18. Details of step 206 will be described later with reference to the flowchart of FIG.

Next, the utterance identification processing in step 204 will be described with reference to the flowchart in FIG. In step 251, the input coordinate position Ppoint is converted into a relative coordinate position in the utterance transition display area 107, and the x coordinate Δx of the instruction input position is calculated. Then, in the next step 252, the instruction input time Tpoint is calculated from the aforementioned equation (1).

In the next step 253, the file storage unit 4
Is read from the comment structure table 41 for one record and substituted into the variable R1. This is data corresponding to one arbitrary statement. In the next step 254, the read record R1
Of the utterance start time field and the utterance end time field are substituted into variables T (start) and T (end), respectively.

At the next step 255, the instruction input time Tpo
It is determined whether int is a time between the utterance start time and the end time of the record R1. If the input instruction time Tpoint exists between the utterance start time and the utterance end time, it is determined that the instruction utterance has been identified, and in step 256,
The value of the comment number field of the record R1 of the corresponding comment in the comment structure table 41 is obtained, the value is substituted for the variable ID, and the value of the variable ID is returned. If it is determined in step 255 that the instruction input time Tpoint does not exist between the utterance start time and the end time of the record R1, the process proceeds to step 255.
Returning to step 253, the next record is read, and the processing related to the next statement is performed.

Next, the instruction intention extracting process will be described. As described above, the instruction intention is defined by the four attributes related to the utterance, the name of the utterer, the utterance time, the name of the previous utterer, and the name of the latter utterer. Using these attributes, the instruction intention is
In this specification, it is described as Iinst (speaker name, speech time, previous speaker name, subsequent speaker name).

FIG. 10 shows a part of the speaker chart.
In FIG. 10, as indicated by the arrow cursor 110, it is indicated that the remark of the conference participant name “Tanaka” has been instructed by the searcher. At this time, the searcher's intention to specify is defined as Iinst (Tanaka, 65 seconds, Suzuki, Sato). This means that the speech of "Tanaka" has a speech time of 65 seconds, speaks after "Suzuki", and "Sato" after "Tanaka"
Means that In this embodiment, it is interpreted that the searcher has designated a specific utterance with an intention expressed by these four attributes.

When not indicating the entire instruction intention but the instruction intention with respect to an individual attribute, each attribute is described in () of the instruction intention Iinst (). For example, the speaker name attribute of the instruction intention is
Marked as Iinst (speaker name). Attributes of other utterance times, preceding speaker names, and subsequent speaker names are described in the same format.

Next, using the flowchart of FIG.
The instruction intention extraction process in step 205 will be described.

FIG. 11 is a flowchart for explaining a process of extracting an instruction intention. Step 311 is an initial setting, in which the statement number of the statement specified by the statement specifying process is substituted for the variable ID. In the next step 312, the record of the statement number indicated by the variable ID from the statement structure table 41 is substituted for the read variable Ri. Similarly, records related to the utterances before and after the utterance number indicated by the variable ID are read and assigned to the variables Rp and Rn, respectively.

In the next step 313, an instruction intention Iinst (speaker name) relating to the speaker name attribute is derived from the variable Ri. In the next step 314, the instruction intention Iinst (speech time) of the speech time attribute is similarly derived.

In the next step 315, the instruction intention related to the statement transition structure is calculated. First, a conference participant name corresponding to the input device number of the variable Rp is extracted from the conference participant table 42 of the file storage unit 4 to derive an instruction intention Iinst (previous speaker name) of a previous speaker name attribute. Similarly, the conference participant name corresponding to the input device number of the variable Rn is
It is extracted from the conference participant table 42 of the file storage unit 4 to derive the instruction intention Iinst (name of the poster speaker) of the poster speaker name attribute.

In the next step 316, the value of the specified instruction intention Iinst (speaker name, speech time, previous speaker name, subsequent speaker name) is sent to the similar candidate detecting unit 18.

Next, the similar utterance detection process in step 206 will be described. In the following description, the similarity of remarks is referred to as DIarti. The utterance similarity DIarti is defined as a composite function of the similarities of the four attributes constituting the utterance intention Iinst. When the similarity is expressed for individual attributes, the similarity DIarti ()
Each attribute is described in parentheses. For example, the speaker name attribute of the similarity is marked as DIarti (speaker name). Other utterance times, names of previous speakers, and names of subsequent speakers are described in the same format.

The similarity DIarti has a smaller value as the similarity is higher. DIarti (A, B) says remark A
And the similarity of the instruction intention of remark B. The similarity for each attribute of the intent to instruct A and B is DIarti (A,
B) Each attribute is described in parentheses in parentheses.

Similarity DIarti (A, B) of speaker name attribute
(Speaker name) has a value of 0 when the names of the speakers A and B are equal. If not, a very high similarity value of DImax is assigned. That is,
When evaluating the similarity, if the similarity of the speaker name attribute is not 0, it is determined that there is no similarity at all. The similarity degree DIarti (A, B) (speech time) of the speech time attribute is evaluated by the absolute value of the difference between the speech times. The similarity between the name of the preceding speaker and the name of the following speaker takes a value of 0 when they match, and a value of 1 when they do not match.

The similarity DIarti of the statement is expressed as a weighted composite function of the similarity for each of the other attributes, using the speaker name attribute as a condition part. The definition formula of the similarity DIarti of this remark is as follows.

(I) When DIarti (speaker name) = 0, DIarti = w1 × DIarti (speech time) + w2 × DIar
ti (previous speaker name) + w3 × DIarti (post speaker name) (ii) When DIarti (speaker name)> 0, it can be expressed as: DIarti = DImax (2) Note that w1, w2, and w3 are weight coefficients.

FIG. 12 shows the definition formula of the similarity DIarti of the utterance and the definition of the similarity of each attribute of the intent of the utterance A and the utterance B for each attribute.

As shown in the equation (2), regarding the similarity DIarti of the utterance, the similarity DIarti (A,
B) (speaker name) is a condition part, and matching is a necessary condition. When DIarti (speaker name) = 0 and the names of the speakers are the same, the attribute is defined as a composite function of the other three attributes, the speaking time, the name of the preceding speaker, and the name of the following speaker. In this case, with respect to the three attributes of the speech time, the name of the previous speaker, and the name of the subsequent speaker, weights of w1, w2, and w3 are assigned to the respective similarities, and these are added to obtain the similarity of the speech. DIarti is calculated. If the names of the speakers do not match, the similarity has an infinite value DImax, which means that they are not similar at all.

FIG. 13 is a flowchart for explaining a process for detecting a similar utterance. Step 351 is an initial setting value, a variable List that holds a list of similar utterance candidates.
Is set to the initial value (). From step 352 to step 356, a series of processing such as calculation and determination of similarity is repeated for each record in the comment structure table 41, that is, for each comment.

In step 352, the statement structure table 41
, One record is read and assigned to the variable R1. In step 353, if the variable R1 is not nil, that is, if there is a record to be processed, a statement similarity calculation process in step 354 is performed. Next, in step 355, a statement similarity determination process is performed to evaluate whether the similarity of the statement satisfies a certain criterion for determining similarity. In the next step 356, the location of the data file corresponding to the speech candidate determined to be similar (the position in the audio data file or the video data file) is detected. Step 35
If it is determined in step 3 that there is no record to be read, the process ends.

FIG. 14 is a flowchart for explaining the utterance similarity calculation process in step 354 of FIG.
In step 401, the initial value of the variable is indicated, and the variable inpu
The utterance number of the utterance identified by the utterance identification processing is substituted for t, and the currently processed utterance number, which is a similarity comparison with input, is substituted for the variable R1.

In step 402, the variable input and the variable
Instruction intention Iinst (input) of speech of two speech numbers of R1
And Iinst (R1) are calculated. In the next step 403, variables related to the speaker name attribute are defined according to the definition equation (2).
The similarity DI between the instruction utterance of input and the similar utterance candidate of the variable R1
Calculate arti (input, R1) (speaker name).

Then, in the next step 404, it is determined whether or not the value of the similarity DIarti (input, R1) (speaker name) of the speaker name attribute is 1. Similarity DIar of speaker attributes
If the value of ti (input, R1) (speaker name) is a value other than 1, that is, if they do not match, the similarity thereafter is not calculated, and in step 407, the similarity DIarti (input,
R1) The extremely large value of DImax described above is substituted as the value of (speaker name), and the process is terminated.

On the other hand, if it is determined in step 404 that the names of the speakers match, the flow shifts to step 405. Steps
In 405, the similarity DIarti for the remaining three attributes
(Input, R1) (utterance time), DIarti (input, R1)
(Previous speaker name) and DIarti (input, R1) (post speaker name) are calculated individually. Then, in step 406, the similarity DIarti (input, R1) between the instruction utterance of the utterance number input and the similar utterance candidate of the utterance number R1 is calculated according to the definition formula (2), and the value is calculated. Pass to.

FIG. 15 is a flowchart for explaining the utterance similarity determination processing. In step 451, as the initial setting, the statement number inpu
The similarity between the input instruction utterance t and the similar candidate utterance of the utterance number R1 is calculated. In the next step 452, it is determined whether or not the value of the calculated similarity DIarti (input, R1) is smaller than the similarity DIlimit of the evaluation criterion of similarity. If it is smaller than the evaluation reference value DIlimit, it is determined that these two remarks are similar, and
Returns a value of "True" at 453. If it is larger than the reference value DIlimit, it is determined that the two remarks are not similar, and the value of "False" is returned in step 454.

FIG. 16 is a flowchart for explaining a process of detecting the location of a data file corresponding to the similar utterance candidate detecting process of step 356 in FIG.

In step 471, initialization is performed, and the variable
The record of the currently processed comment structure table 41 is substituted for R1. At step 472, the result of the similarity determination processing is determined. If the return value is "True", at step 473, the audio data corresponding to the utterance determined to be similar to the instruction input utterance The location of the file is represented by a section between the start time and the end time of the comment, and is added to the variable List. If the return value is "False" in step 472, the process ends.

As described above, voice information of a participant in a conference or the like is recorded, speech structure data as index information for accessing a voice data file is extracted, and speech structure data is visualized as a speaker chart. In a multimedia conference recording / playback apparatus having a means, when a user who is a searcher of a conference record designates an arbitrary utterance position on a speaker chart with a pointing device or the like, the intention of the user's instruction is extracted and the intention is extracted. The user can easily search for utterances similar to what he intends when he / she judges that the utterance candidate is not his own by watching the reproduced sound or image. Can be.

[Second Embodiment] In the above embodiment, the user's search intention is extracted from an instruction input for instructing a specific utterance. However, the user ’s search intent
By extracting the reproduction intention by the reproduction action of the user, it is possible to more accurately extract the information required by the user.

In the second embodiment, in order to reproduce a specific utterance section, the user not only specifies a desired utterance (utterance section bar) on the utterance chart as described above, but also Instruction on the player chart
It is possible to give a reproduction end instruction while viewing the reproduction information. That is, the user can specify a playback section that extends over a plurality of comment sections. Then, in the second embodiment, a reproduction intention is extracted from the reproduction instruction action of the user, and information required by the user can be extracted based on the extracted reproduction intention.

FIG. 17 is a block diagram for explaining the details of the searcher intention extraction unit 17 in the case of the second embodiment. The searcher intention extraction unit 17 extracts the intention of inputting an instruction. It comprises an instruction intention extraction unit 17a and a reproduction intention extraction unit 17b for extracting a reproduction intention.

The instruction intention extracting unit 17a extracts the instruction intention from the instruction input information for the specified specific utterance as described in the first embodiment. The extraction unit 17b extracts the user's reproduction intention for the information to be searched from the statement structure of a series of statement groups included in the section from the reproduction start to the reproduction end.

FIG. 18 is a block diagram for explaining details of the similar candidate detecting section 18 in the case of the second embodiment. In the case of the second embodiment, the similar candidate detection unit 18 includes a similarity determination method selection unit 18a, a similar utterance candidate detection unit 18b, and a similar utterance structure candidate detection unit 18f.

The similarity determination method selection unit 18a determines whether the appropriate similarity between the similar utterance candidate detection unit 18b and the similar utterance structure candidate detection unit 18f is determined based on the searcher's instruction input information and the reproduction information. A process for selecting a determination method is performed. In this embodiment, the similarity determination method selection unit 18a
As will be described later, if only one utterance is included in the reproduction section specified in response to the user's instruction input, the similar utterance candidate detection unit 18b is selected, and a plurality of If there are multiple utterances, the similar utterance structure candidate detecting unit 18f is selected.

The similar remark candidate detecting section 18b is the same as the operation of the similar candidate detecting section of the first embodiment described with reference to FIG. 13, and includes a remark similarity calculating section 18c and a remark similarity determining section 18d. And a similar utterance detection unit 18e. The processes of the similar utterance candidate detection unit 18b, the utterance similarity determination unit 18d, and the similar utterance detection unit 18e are the same as those described with reference to FIGS. 14, 15, and 16.

The similar utterance structure candidate detecting section 18f includes an utterance structure similarity calculating section 18g and an utterance structure similarity determining section 18h.
And three parts of the similar utterance structure detecting unit 18i. The difference between the similar utterance candidate detection unit 18b and the similar utterance structure candidate detection unit 18f is as follows. That is, the similar utterance candidate detecting unit 18b detects the similarity with respect to the utterance inputted and instructed, and detects the similarity with respect to a series of utterance groups by adding the reproduction information. The detecting unit 18f.

FIG. 19 is a diagram for explaining the user's designation of the reproduction section in the speaker chart.
FIG. 19 shows a part of the speaker chart.

The reproduction instruction input position is also represented by relative coordinates in the utterance transition display area 107, as in the case of the instruction input of the first embodiment. In FIG. 19, the leftmost point in the x direction of the utterance transition display area 107 is defined as a starting point 501.
The relative coordinates are represented by (0,0). Then, the x coordinate 502 of the reproduction start point instructed by the user to start reproduction is represented by Δxst
art, the x coordinate 503 of the playback end point specified by the playback end is Δ
xstop.

A time corresponding to the starting point (0, 0) is represented as a starting time Torigin, a reproduction start instruction time at which a reproduction start instruction is inputted by the user is represented as Tstart, and a reproduction end instruction is inputted by the user. The playback end instruction time, which is the performed time, is represented as Tstop. A period between the reproduction start instruction time Tstart and the reproduction end instruction time Tstop is a reproduction section. The search intention of the searcher is extracted for a series of utterance groups included in the playback section.

FIG. 20 is a flowchart for explaining a process for detecting a similar utterance structure candidate.

In step 601, it is monitored whether or not there is an input of a reproduction start instruction from a user who is a searcher. Steps
At 602, it is determined whether or not there is an instruction input. If it is determined that there is no instruction input, the process returns to step 601 and the monitoring of the user's instruction input is repeated.

If it is determined in step 602 that a reproduction start instruction has been input by the user, in step 603, the reproduction start instruction input coordinates of the user are extracted, and the coordinates are input to a variable Pstart. An utterance identification process is performed on the coordinate variable Pstart to identify an utterance at the instruction input position. This utterance identification processing is the same as the processing described with reference to FIG.

Next, in step 605, the monitoring of the instruction input from the user is continued, and in the next step 606, it is monitored whether or not there is a reproduction ending instruction input. If there is no ending instruction input, the monitoring is continued in step 605. I do. If it is determined in step 606 that the reproduction end instruction has been input, in step 607, the reproduction end time is substituted for a variable Tstop. Next, at step 608, a reproduction section specifying process is performed. Here, a playback section is specified, and a series of utterance groups included in the playback section are specified. Details of the playback section identification processing will be described later with reference to FIG.

After the searcher inputs a reproduction end instruction, similarity determination processing is performed. First, in step 609, similarity determination processing for selecting a similarity determination method is performed. Details of the similarity determination processing will be described later with reference to FIG.

If it is determined in step 610 that the similarity determination is performed on the utterance as a result of the similarity determination processing in step 609, the process proceeds to step 611.
An instruction intention extraction process is performed, and a similar utterance detection process is performed in the next step 612. The processing of 611 and 612
In the first embodiment, this corresponds to a series of processes described with reference to FIGS. 11 to 16.

If it is determined in step 610 that the similarity is determined for the speech structure as a result of the similarity determination process, a process for extracting a reproduction intention is performed in step 613. In the next step 614, similar speech structure detection processing is performed. The processing for extracting the reproduction intention in step 613 will be described later with reference to FIG.
In addition, the similar utterance structure detection processing in step 614 is performed as follows.
This will be described later with reference to FIGS.

The process of specifying the playback section in step 608 will be described with reference to the flowchart of FIG.

At step 651, the variable IDstart and the variable I
This indicates the initial setting of Dstop, and the variable IDstart is set at step 604 from the reproduction start instruction input position Pstart.
Is assigned to the utterance number specified by the utterance specifying process. Similarly, the utterance number specified from the input time Tstop specified by the reproduction stop instruction input is substituted for the variable IDstop. The utterance identification process in this case refers to the processes of steps 253 to 256 shown in FIG.

As a result, the reproduction section designated by the user is obtained. However, in the playback end instruction act, although there is no intention to play,
There is also a possibility that an end instruction is input after the next utterance is reproduced. Therefore, in order to extract a reproduction section intended by the user as accurately as possible, it is better to perform a process of correcting an excessive portion of reproduction.

In general, it is considered that a user starts reproduction of an utterance and, when the utterance does not relate to his / her intended reproduction section, inputs a reproduction end relatively immediately. Therefore, in the second embodiment, the time from the start time of the utterance at the position where the user has input the reproduction end instruction (hereinafter referred to as a stop utterance) to the reproduction end instruction input time is longer than a predetermined time ΔTlimit. When the length is also shorter, the stop utterance, which is the last utterance, is corrected to be excluded from the reproduction section intended by the user, as a utterance irrelevant to the reproduction intention.

That is, in step 652, the reproduction end instruction time is substituted for the variable Tstop. In the next step 653, the statement number IDst of the stop statement specified at this time
The record of the comment structure table 41 corresponding to op is read, and it is substituted for the variable R1. Next, in step 654, the start time field in the record of the variable R1 is substituted for the variable T (start time).

Then, in the next step 655, the actual time Tstop at which the reproduction end instruction was input and the start time T (start time) of the speech number IDstop specified as the stop speech
Is smaller than a certain time ΔTlimit. If smaller, go to step 656,
The searcher unintentionally considers that the reproduction is excessive, and does not include the section of the stop comment in the reproduction section. That is, in step 656, the utterance at the end of the playback section is regarded as the utterance immediately before the stop utterance, and the variable IDstop is subtracted by "1".

If it is determined in step 655 that the difference between the actual time Tstop at which the reproduction end instruction was input and the start time T (start time) of the stop utterance is larger than ΔTlimit, no operation is performed. The section up to the time specified by the playback end instruction input position is regarded as a playback section as it is. Then, in the next step 657, the value of the reproduction section (IDstart, IDstop) obtained as described above is returned.

Next, a process for selecting a similarity determination method will be described with reference to the flowchart in FIG.

First, in step 671, the playback section (IDstart, IDsto
p) is specified. In the next step 672,
It is determined whether the reproduction start utterance IDstart is equal to the reproduction stop utterance IDstop. If they are equal, the playback section is not a section, but a single utterance. Therefore, "utterance" is returned as a return value, and the similarity determination for the utterance is performed. On the other hand, if they are not equal to each other, since a plurality of utterances are included in the reproduction section, "speech structure" is returned as a return value, and similarity determination for the utterance structure is performed.

FIG. 23 is a diagram for explaining a reproduction intention, which shows a part of a speaker chart.

FIG. 24 shows the definition and notation system of the reproduction intention. In this embodiment, the reproduction intention is defined by six attributes related to the statement structure of a group of statements in the playback section. The six attributes are the instruction utterance, the name of the uttered utterer, the total number of utterances, the total utterance time, the speaker set,
This is a statement transition matrix.

By using these attributes, the reproduction intention is
Play (instructed utterance, name of stopped utterer, total number of utterances, total utterance time, set of utterers, utterance transition matrix). In addition, when not describing the entire reproduction intention but the reproduction intention with respect to individual attributes, each attribute is described in parentheses of the reproduction intention Ireplay (). For example, the speaker name attribute of the reproduction intention is marked as Ireplay (speaker name). The names of other suspended speakers, the total number of speeches, the total speech time, the speaker set,
The attribute of the statement transition matrix is described in the same format.

To explain the details of the six attributes, in the case of a reproduction section instruction, the instruction utterance corresponds to the utterance (utterance section) at the reproduction start instruction position, and Ireplay (instruction utterance) =
Iinst (instruction remark). The stop speaker name is the name of the stop speaker. The total number of remarks is based on the playback section (IDstart
, IDstop). The total utterance time is the sum of the time of each utterance in the playback section (IDstart, IDstop). The speaker set is a playback section (ID
Start, IDstop) is a list of speaker names included in the list without duplication.

The utterance transition matrix is a matrix representing utterance transition among a plurality of utterers included in the utterer set. If the number of utterers in the utterer set is n, n rows × n columns are used. It is a matrix. That is, n persons are arranged in the order of the input device number for each speaker, and arranged in n columns. When there is a transition from one speaker A to another speaker B, an element of a row corresponding to the input device number of the speaker A and a column corresponding to the input device number of the speaker B Is added to. Thus, it is possible to indicate how many transitions have occurred from which speaker to which speaker.

FIG. 25 shows a description example of a reproduction intention corresponding to a reproduction section of the speaker chart shown in FIG.

First, since the instructed utterance is a utterance instructed to be reproduced and input, the utterance number 205 is specified. The stop speaker name is the speaker name of the speech corresponding to the stop speech of the specified reproduction section, and thus is the speaker “Suzuki” of the speech number 209 in the example of FIG. Since the total number of utterances is the total number of utterances included in the reproduction section, the total number is five in this example. The total utterance time is the sum of each utterance time of a group of utterances included in the reproduction section, and includes a reproduction instruction time Tstart and a re-stop time Tstop.
The difference time is not taken into account, and is from the head of the statement number 205 to the end of the statement number 209, for example, 3 minutes and 20 seconds.
The speaker set is (Tanaka, Suzuki, Sato) in this example. Suzuki has made three remarks, but counts only once, since duplication is eliminated.

In the example of FIG. 23, the statement transition matrix is changed from the speaker "Suzuki" to "Tanaka" once, the speaker "Tanaka" to "Suzuki" once, and the speaker "Suzuki" to "Sato". Once in a row, the speaker "Sato" goes once to "Suzuki".

FIGS. 26 and 27 are flowcharts for explaining the process of extracting a reproduction intention.

Steps 711 and 712 are processing for initial setting. First, in step 711, the speech number of the utterance instructed to start reproduction and the utterance number of the utterance instructed to end reproduction are assigned to the variables IDstart and IDstop by the reproduction section specifying process.

At the next step 712, initial values of various variables are set. The variable time holds the value of the total speech time. The variable List is a list for holding a set of speakers. An instruction utterance (start utterance) is set as an initial value in the variable id. The variable “transfer” is a variable that holds a statement transition matrix. As an initial value, when the number of conference participants is n,
An n × n zero matrix is set.

In step 713, a record in the statement structure table corresponding to the statement number IDstop of the playback stop statement is read and assigned to the variable R1. In the next step 714, the variable
The name of the conference participant corresponding to the input device number in the record of the read variable R1 is acquired from the conference participant table 42 and substituted into name-stop. This corresponds to the name of the suspended speaker.

In the next step 715, the record of the utterance immediately before the reproduction start utterance in the utterance structure table 41 is read, and it is substituted for the variable R1, and the subsequent steps 716 to 721 are prepared for the repetition processing. I do. The processing from step 716 to step 721 is a reproduction intention extraction processing that is repeatedly performed for each utterance in the reproduction section.

First, in step 716, a record that matches the statement number of the reproduction start statement indicated by the variable id is read from the statement structure table 41, and is substituted for the variable R2. Therefore, the records related to the preceding and following statements are substituted into the variable R1 and the variable R2. In addition,
The basic processing target in the following repetition processing is R2.

In step 717, it is determined whether the statement number in the record of the variable R2 is smaller than the statement number of the stop statement IDstop, that is, whether the statement number exists in the reproduction section. If it exists in the playback section, steps 718 and 71
9. In step 720, an attribute related to the reproduction intention is calculated.

First, at step 718, the total speech time time
Is added to the speech time in the record of the variable R2. The variable number of the total number of remarks is also incremented by +1. In step 719, processing relating to the speaker set is performed. As the variable name, the conference participant name corresponding to the input device number in the record of the variable R2 is extracted from the conference participant participant table 42. This is the speaker name of the statement currently being processed. Then, the speaker name indicated in this variable name is the speaker set Li
It is determined whether or not it already exists in st, and if it does not yet exist in the list, the variable na
The name of the speaker of me is added.

At step 720, processing of a statement transition sequence is performed. In the n × n matrix when the number of conference participants = n, the input device number of the previous statement R1 of the statement R2 is taken as the row number, and R2
Is added to the value of the element whose column number is the input device number. This means that there was a transition of the utterance from R1 to R2.

At step 721, post-processing for the next repetition is performed. That is, by adding +1 to the variable id, the preparation for processing the next utterance is made.
Further, the variable R2 becomes the previous utterance in the next processing loop, and is substituted for the variable R1.

If it is determined in step 717 that the comment number of the variable id does not exist in the comment section, step 72
The process proceeds to step 2 to derive the overall playback intention from the calculated intention attribute, and to set the playback intention Ireplay (IDstart, name-stop, nu
mber, time, List, transfer) as the return value.

FIG. 28 is a view for explaining the definition and notation method of the similarity of the statement structure. Similar to the similarity of the remark, the intention is represented by I and the similarity is represented by DI. DI is statement structure A,
B similarity. Also in this case, the similarity has a smaller value as the similarity is higher.

The similarity of the statement structure is defined as shown in the definition equation. In other words, the similarity DIa-stru
Is the similarity DIarti of the instruction utterance and the similarity D of the utterance structure
It can be defined as the sum of the values of the following expressions: DIa-stru = α1 × DIarti + α2 × DIstru (3) α1 and α2 are weighting factors, respectively.

Since the similarity of the instruction utterance has already been defined, the definition of the similarity regarding the other five attributes excluding the instruction utterance among the six attributes constituting the reproduction intention will be described here.

The similarity DIstru (A, B) of the stop speaker name
The (suspended speaker name) is to determine whether the final speaker names are the same in the speech sections of the speech structure A and the speech structure B. If the names of the stopped speakers match, the value takes a value of 0, and if the names are different, the value takes a large value of DImax. This means that, in the similarity of the statement structure, if the names of the stopped speakers do not match, the value of the similarity will increase indefinitely, and it will be determined that they are not similar, as in the similarity of the instruction statement. Means.

The similarity DIstru (A, B) (total utterance count) of the total utterance count is defined by the absolute value of the difference in the total utterance count.

Similarly, the similarity DIstru of the total utterance time
(A, B) (total utterance time) is defined by the absolute value of the difference between the total utterance times.

Similarity DIstru (A, B) of speaker set
(Speaker set) calculates a set of speakers that do not overlap between A and B among elements in the set in the sum of the speaker sets of the statement structure A and the statement structure B. The similarity is defined by the number of elements of the calculated set, and the numerical value increases as the number of speakers whose speaker sets do not match increases.

Similarity DIstru (A, B) of statement transition structure
(Speech transition matrix) calculates the absolute value of the difference between the speech transition matrices and is defined by the sum of the elements. This indicates the degree of coincidence of the pattern of the transition from the speaker X to the speaker Y, and it is interpreted that as the number of the same transition patterns increases, the value of the similarity decreases and the similarity increases.

As shown in the following definition expression (4), the similarity of the statement structure is expressed as a weighted composite function of the similarity for each of the other attributes, with the attribute of the stopped speaker name as a condition part. That is, the similarity DIstru of the statement structure is as follows: (i) When DIstru (name of the stopped speaker) = 0, DIstru = w1 × DIstru (total number of statements) + w2 × Dist
ru (total utterance time) + w3 × DIStru (speaker set) + w
4 × DIstru (A, B) (state transition matrix) (ii) When DIstru (stop speaker name)> 0, DIstru = DImax (4) is defined. Note that w1, w2, w3, and w4 are weight coefficients.

As shown in the expression (4), the similarity of the attribute of the uttered speaker name is a condition part with respect to the similarity of the utterance structure, and matching is a necessary condition. When the names of the stopped speakers are the same, they are defined as a composite function of the composite functions of the other four attributes. In other words, the similarity of the statement structure is a necessary condition that, in addition to the similarity of the instruction statements, the names of the stopped speakers match, and if they do not match, the degree of similarity is infinite. , Which means that they are not quite similar.

In the composite function of the four attributes of the total number of utterances, the total utterance time, the utterer set, and the utterance transition matrix, each similarity is weighted by w1, w2, w3, and w4 and added. Calculate the similarity.

FIG. 29 is a flowchart for explaining a process for detecting a similar utterance structure.

Step 781 is a step for performing initial setting, in which an initial value () is set in a variable List that holds a list of values of locations where similar speech structures exist. In step 782, one record is read from the comment structure table 41, and the variable
Substitute for R1. In the next step 783, if the variable R1 is not nil, that is, if there is a record to be processed,
In the next step 784, a section of a similar utterance structure candidate is extracted. Next, a comment structure similarity calculation process of step 785 is performed.

In the next step 786, a statement structure similarity determination process for evaluating whether or not the calculated similarity of the statement structure satisfies a certain criterion of similarity is performed. The location of the data file corresponding to the statement structure candidate determined to be present is detected. If there is no record to be read in step 783, the process ends.

FIG. 30 is a flowchart for explaining a process for extracting a speech section of a similar candidate having a speech structure.

In step 801, the initial value of the processing is
The reproduction section A is calculated by the reproduction section identification processing, and the reproduction intention Ireplay (A) (instruction utterance, stop speaker name, total utterance number, total utterance time, utterer set, utterance transition matrix) is calculated by the reproduction intention extraction processing.

In the next step 802, an empty list () is substituted for a variable KList to which the detected similar speech structure candidate is substituted. In step 803, the statement number of the record R1 of the statement structure table that is currently being processed is extracted and assigned to a variable id.
In step 804, the similarity between the utterance of the utterance number IDstart which is the start utterance of the playback section and the utterance of the utterance number id is calculated, and it is determined whether or not the similarity is smaller than a certain similarity DIlimit. Is done. The similarity of the instruction intentions of the start utterances is a necessary condition for the utterance structures to be similar. Therefore, if it is determined that the similarity is larger than a certain value, that is, it is determined that they are not similar, the process proceeds to step 813, where KList is returned as a return value, and the process ends.

At step 804, a remark of the start of the reproduction section,
When it is determined that the utterances with the utterance number id are similar, the section of the utterance structure is specified by the processing of steps 805 to 812.

That is, in step 805, id + 1 is substituted as the initial value of the counter variable n. This means that processing is performed from a statement next to a statement currently being processed. Also, 1 is set as the initial value of the counter variable m relating to the processing of the stopped speaker, and the variable M is set to the maximum number of loops of the processing relating to the name of the stopped speaker, and the number of times of the name of the stopped speaker in the playback section. Set. This is a case in which the number of appearances of stopped speakers is used as one criterion for limiting the range of the section to be investigated when extracting sections having a similar utterance structure.

In the next step 806, the record with the comment number n is read from the comment structure table 41 and is substituted for the variable R2. In the next step 807, it is determined whether or not the variable R2 is nil.
If there is no record to be read, it is determined that an appropriate utterance structure could not be extracted, and the process proceeds to step 813 where the variable K
List is returned as a return value, and the process ends.

If it is determined in step 807 that the variable R2 is not nil, the flow shifts to step 808. At step 808, it is determined whether or not the variable m has exceeded the maximum value of the loop relating to the name of the stopped speaker. If so, the processing relating to the statement id is terminated. At step 813, KList is returned as a return value, and Ends. If not, go to step 809.

In step 809, it is determined whether or not the value of the attribute of the name of the paused speaker who intends to reproduce is equal to the name of the conference participant corresponding to the statement number of the variable R2. If they do not match, in step 811, 1 is added to the counter variable n, and the flow shifts to step 806 in order to process the next utterance. If they match, it is determined that the section of the similar utterance candidate has been specified, and the routine proceeds to step 810, where the variable KList
In the KList, the section (id, n) of the identified similar utterance candidate is
Add to Then, in step 812, 1 is added to the counter variable m of the processing relating to the name of the stopped speaker, and the processing for searching for the next statement structure for the statement with the statement number id is continued.

FIG. 31 is a flowchart for explaining the similarity calculation process of the utterance structure. First, step 85
In 1, a list of extracted sections is substituted into KList by a similar utterance structure section extraction process as an initial setting. Next, at step 852, the reproduction section is set to a variable A.

Steps 853 to 858 correspond to the KList
Is performed for each element. In step 853, one section (IDstart, IDstop) as a similar utterance structure candidate is extracted from the KList and assigned to a variable B. In step 854, it is determined whether or not the processing has been completed for all the comment structures in the KList. If it is finished, go to step 859.

If it is determined in step 854 that there is a playback section to be processed, in step 855, the similarity between the instruction intentions regarding the start utterances of the playback section of variable A and the playback section of variable B is defined as a definition expression. Then calculate. Next, in step 856, the similarity for each attribute defining the comment structure is calculated. At this time, the similarity regarding the stopped speaker name has already been determined at the time of section extraction, and here, four attributes of the total number of statements, the total statement time, the total speaker set, and the statement transition matrix are calculated.

In the next step 857, according to the definition formula,
The similarity between the utterance structures of the playback section A and the playback section B is calculated according to the definition formula. In the next step 858, the overall similarity of the utterance structure is calculated by combining both the similarity of the instruction intention of the start utterance and the similarity of the utterance structure, and is added to the variable DList holding a list of similarities. . Thereafter, the process returns to step 853, and the process is repeated.

Finally, in step 859, the process is terminated with the list KList of similar utterance candidates and the list DList of similarity as return values.

FIG. 32 is a flowchart for explaining a process of determining the similarity of a similar utterance structure candidate and detecting the location of the corresponding audio data file.

In step 871, initialization is performed. A list of similar utterance candidate sections, which are return values of the similarity calculation process, is set as a variable KList, and a list of similarity values with the reproduction intention is set as a variable DList. substitute.

From step 872 to step 875, similarity judgment processing is performed for each element in the list. First, at step 872, one element is taken out from each of the lists DList and KList, and substituted into variables D and K. In the next step 873, it is determined whether or not the element to be processed has been completed. If the processing has been completed, the process proceeds to step 876. In step 876, the value of the variable List that holds the section of the similar utterance structure is returned as a return value, and the process ends.

If it is determined in step 873 that the processing of the elements in the list has not been completed, in step 874,
It is determined whether or not the value of the similarity is smaller than a certain limit value DIlimit. If the value is smaller than a certain similarity, it is determined that they are similar, and the process proceeds to step 875, and the section K corresponding to the variable D is added to the list List holding the similar utterance structure candidates. And step 872
And the process is repeated for the next element. Step 87
In step 4, if the value of the variable D is larger than DIlimit, it is determined that they are not similar, and the flow returns to step 872 to proceed to the processing of the next element.

FIG. 33 is a diagram for explaining an embodiment of a method for displaying detected similar speech structure candidates.

Reference numeral 901 denotes a similar utterance structure candidate display area.
This area 901 is composed of two areas, an all meeting time display area 902 and a similar utterance structure candidate reduced figure display area 903. When the similar speech structure is detected, the place where the similar speech structure candidate exists is shown in vertical bar displays 904 and 905 in the entire conference time display area 902. Vertical bar display 904
Indicates a playback section.

A vertical bar display 905 indicates the location of a similar utterance candidate. By indicating the location of the similar utterance structure candidate in the all meeting time display area 902, it is possible to list in which part of the whole the similar utterance candidate exists.

[0198] The similar utterance structure candidate reduced figure display area 903 is
It is composed of a plurality of rectangular areas. In each rectangular area, a reduced view of the statement structure is displayed. The reduced view displayed on 906 is
This is a speech structure corresponding to 904 playback sections. Rectangular area 907
In the other rectangular areas including, a reduced view of a speech structure corresponding to a similar speech structure candidate existing during another meeting time, such as a vertical bar display 905, is displayed in chronological order. The searcher can select and play similar candidates by clicking on the displayed reduced view with a pointing device such as a mouse.

In the total conference time display area 902,
By changing the display color of the rectangular area as well as indicating its location, the degree of similarity can also be presented as information. Although a display example of similar utterance candidates is shown here, a similar display can be performed for a similar utterance display method for a portion including a similar utterance and a transition utterance structure before and after.

[0200]

As described above, according to the conference recording / playback apparatus of the present invention, the search intention of the searcher is automatically extracted from the search instruction input action and the playback action, and a similar statement and a series of statement groups Is detected and visualized and presented on a display screen. As a result, remarks having a structure similar to the search intention of the searcher of the conference information can be automatically extracted without additional input by the searcher. Further, by visually presenting a similar utterance candidate to the searcher, it becomes possible to inform the presence of the candidate.

By presenting the similar utterance and the similar utterance structure candidate to the searcher, the searcher who wants to access the information required for the conference information accesses without sufficient clues for the access, and corrects it. Even when the location cannot be accessed, another candidate similar to the search intention is automatically presented, so that it is possible to efficiently reach the correct access location.

Conversely, even if the reproduction location is erroneously determined to be correct based on the ambiguous memory, by indicating to the searcher that other similar candidates exist, other candidates that are considered to be correct can be obtained. Can be known by a searcher, and search omission can be reduced.

[0203] Further, on the display screen of the similar candidates, a relative position in the whole of the chronological order and a list display of a reduced view of detailed information which can grasp the contents of each similar candidate are simultaneously displayed. Relative position information and information on absolute contents can be organically linked. As a result, the recognizing ability of the comment structure is improved, the search action of the searcher is appropriately navigated, and the search can be efficiently performed.
Listening to playback information while referring to such information,
Or, by watching, it is possible to promote the understanding of the reproduction contents.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a system configuration diagram of a conference information recording / reproducing apparatus according to an embodiment of the present invention.

FIG. 2 is a diagram illustrating a data file stored in a file storage unit of the conference information recording / playback apparatus according to one embodiment of the present invention.

FIG. 3 is a diagram for explaining a data structure of a conference participant table in a file storage unit of FIG. 2;

FIG. 4 is a diagram for explaining a data structure of a statement structure table in a file storage unit in FIG. 2;

FIG. 5 is a diagram showing an example of a speaker chart.

FIG. 6 is a diagram for explaining a method in which a searcher instructs a statement to be reproduced;

FIG. 7 is a diagram for explaining a relationship between a searcher's instruction input position and a relative coordinate position in a speaker chart display area.

FIG. 8 is a flowchart showing an outline of a process for detecting a similar utterance candidate in the conference information recording / reproducing apparatus according to one embodiment of the present invention;

FIG. 9 is a flowchart for explaining a statement specifying process in the conference information recording / reproducing apparatus according to one embodiment of the present invention;

FIG. 10 is a diagram for explaining an instruction intention in the conference information recording and reproducing apparatus according to one embodiment of the present invention;

FIG. 11 is a flowchart illustrating a process of extracting an instruction intention in the conference information recording and reproducing apparatus according to the embodiment of the present invention;

FIG. 12 is a diagram illustrating a definition and a notation method of a similarity between utterances in the conference information recording and reproducing apparatus according to the embodiment of the present invention;

FIG. 13 is a flowchart illustrating a process for detecting a similar utterance in the conference information recording / reproducing apparatus according to the embodiment of the present invention.

FIG. 14 is a flowchart illustrating a statement similarity calculation process in the conference information recording and reproducing apparatus according to one embodiment of the present invention;

FIG. 15 is a flowchart for explaining a statement similarity determination process in the conference information recording and reproducing apparatus according to one embodiment of the present invention;

FIG. 16 is a flowchart illustrating a process of detecting a location of a data file corresponding to a similar statement candidate in the conference information recording and reproducing apparatus according to one embodiment of the present invention;

FIG. 17 is a block diagram illustrating details of a searcher intention extracting unit in the conference information recording / reproducing apparatus according to one embodiment of the present invention;

FIG. 18 is a block diagram illustrating details of a similar candidate detecting unit in the conference information recording / reproducing apparatus according to one embodiment of the present invention;

FIG. 19 is a diagram for describing a playback section in a speaker chart in the conference information recording / playback apparatus according to one embodiment of the present invention.

FIG. 20 is a flowchart illustrating a process for detecting a similar utterance structure candidate in the conference information recording / reproducing apparatus according to one embodiment of the present invention;

FIG. 21 is a flowchart illustrating a process of specifying a playback section in the conference information recording / playback apparatus according to one embodiment of the present invention.

FIG. 22 is a flowchart illustrating a process for selecting a similarity determination method in the conference information recording and reproducing apparatus according to one embodiment of the present invention;

FIG. 23 is a diagram for describing a reproduction intention in the conference information recording / reproducing apparatus according to one embodiment of the present invention;

FIG. 24 is a diagram for explaining a reproduction intention in the conference information recording / reproducing apparatus according to one embodiment of the present invention;

FIG. 25 is a diagram for describing a playback intention in the conference information recording / playback apparatus according to one embodiment of the present invention.

FIG. 26 is a diagram showing a part of a flowchart for explaining a process of extracting a reproduction intention in the conference information recording / reproducing apparatus according to one embodiment of the present invention;

FIG. 27 is a diagram showing a part of a flowchart for explaining a process of extracting a reproduction intention in the conference information recording / reproducing apparatus according to one embodiment of the present invention;

FIG. 28 is a diagram illustrating a definition and a notation method of a similarity of a statement structure in the conference information recording / reproducing device according to the embodiment of the present invention.

FIG. 29 is a flowchart illustrating a process for detecting a similar utterance structure in the conference information recording and reproducing apparatus according to one embodiment of the present invention;

FIG. 30 is a flowchart illustrating a process for extracting a speech section of a similar candidate having a speech structure in the conference information recording / playback apparatus according to one embodiment of the present invention;

FIG. 31 is a flowchart illustrating a similarity calculation process of a statement structure in the conference information recording and reproducing apparatus according to one embodiment of the present invention;

FIG. 32 is a flowchart illustrating a process of determining a similarity of a similar utterance structure candidate and detecting a location of a corresponding audio data file in the conference information recording / reproducing apparatus according to the embodiment of the present invention; .

FIG. 33 is a diagram for explaining an example of a method of displaying detected similar speech structure candidates in the conference information recording / reproducing apparatus according to one embodiment of the present invention;

[Explanation of symbols]

 Reference Signs List 1a Voice input device 2 A / D converter 4 File storage unit 5 Speaker chart generation control unit 6 Utterance data extraction unit 7 Timer 8 Utterance structure table generation unit 9 Utterance chart generation unit 10 Utterance chart display unit 11 Display device 12 Instruction input device 13 Video playback device 14 Audio playback device 15 Speaker chart search control unit 16 Statement identification unit 17 Searcher intention extraction unit 18 Similar candidate detection unit 19 Similar candidate display unit

Claims (9)

[Claims] Recording means for recording voice data when a plurality of conference participants hold a conference; statement structure extracting means for extracting a statement structure of the plurality of conference participants from the voice data; Visualization information generating means for generating visualization information for visualizing the statement structure; a display device for visualizing the statement structure based on the visualization information; and a statement visualized on the display device. Instruction input means for inputting an instruction in a structure; reproducing means for reproducing audio data corresponding to the position or portion indicated by the instruction input means; and intention of a searcher's instruction operation through the instruction input means To
An intention extraction unit that extracts using an attribute value determined in advance to extract an intention; a similar candidate detection unit that detects a voice data section having an intention similar to the intention extracted by the intention extraction unit; A similarity candidate display means for visualizing the similarity candidates detected by the similarity candidate detection means on a display device. 2. A speech input device provided for each conference participant for inputting speech data of conference information, storage means for storing said speech data, and speech data extraction means for extracting speech from said speech data. Utterance structure table generation means for generating an utterance structure table from the extracted utterance data and a timer; storage means for storing the utterance structure table; holding a correspondence relationship between the voice input device and the conference participant Storage means for storing a conference participant table to be executed; speaker chart generation means for generating a speaker chart for visualizing the speech structure table on a display device; and the speaker chart generation means generated by the speaker chart generation means. Speaker chart display means for displaying a speaker chart on the display device, and a searcher performs reproduction on the speaker chart. An instruction input means for instructing an arbitrary utterance that figure, the speech identifying means for identifying the speech indicated by the instruction input unit, an instruction intention of the searcher related to the identified speech,
Intention extraction means for extracting using an attribute value determined in advance to extract an intention, and instructed by the instruction input means using information on the instruction intention of the searcher extracted by the intention extraction means A similar utterance detection unit for detecting a similar utterance candidate having an intention similar to the utterance; and a similar utterance candidate display unit for visualizing the similar utterance candidate detected by the similar utterance detection unit on a display device. A conference information recording / reproducing apparatus characterized by the above-mentioned. 3. The conference information recording / reproducing apparatus according to claim 2, wherein the instruction intention extracting means includes:
A conference information recording / reproducing apparatus for extracting a searcher's intention from values of four attributes of a speaker name, a speech time, a preceding speaker name, and a subsequent speaker name. 4. The conference information recording / reproducing apparatus according to claim 2, wherein the similar utterance detection means includes: an instruction input utterance intention extracted by the instruction intention extraction means; Statement similarity calculating means for calculating a similarity to the statement; and a statement similarity determining whether the similarity calculated by the statement similarity calculating means has a similarity greater than or equal to a predetermined value. A meeting information recording / reproducing apparatus, comprising: a determination unit; and detecting the similar utterance candidate based on a determination result of the utterance similarity determination unit. 5. The conference information recording / reproducing apparatus according to claim 2, wherein the searcher can give an instruction of a playback section by the instruction input means, and the intention extracting means performs a playback action of the searcher. Conference information recording, comprising: playback operation monitoring means for monitoring; and playback intention extraction means for extracting a search intention of a searcher from predetermined attribute values relating to a series of utterance groups in a reproduced voice data section. Playback device. 6. The conference information recording / reproducing apparatus according to claim 5, wherein the attribute value used by the reproduction intention extraction means is an instruction intention or a stop of a reproduction start utterance of a series of utterance groups of the reproduced audio data section. A conference information recording / reproducing apparatus characterized by a speaker name, a total number of speeches, a total speech time, a speaker set, and a speech transition matrix. 7. The conference information recording / reproducing apparatus according to claim 5, wherein the similar utterance detecting means uses the reproduction intention from the reproduction intention extracting means to output another series of utterances in the utterance structure table. For a group, a statement structure similarity calculating means for calculating a similarity of a statement structure, and whether the similarity of the statement structure calculated by the statement structure similarity calculating means has a similarity greater than or equal to a predetermined value. A meeting information recording / reproducing apparatus, comprising: statement structure similarity determining means for determining whether the similar statement structure candidate is detected based on the determination result of the statement structure similarity determining means. 8. The conference information recording / reproducing apparatus according to claim 5, wherein said similar utterance detecting means automatically selects a similar utterance detecting means and a similar utterance structure detecting means in accordance with the state of the reproduced utterance. A conference information recording / reproducing apparatus comprising a degree determination method selecting means. 9. The conference information recording / reproducing apparatus according to claim 2, wherein said similar utterance candidate display means includes: an entire conference time display area for visualizing chronological information of conference times; and a reduced view of a plurality of utterance structures. And a similar section reduced display area for displaying a similar section. In the total meeting time display area, a reproduction section determined by an input instruction of the searcher from the instruction input device and a similar candidate of the reproduction section are provided. Means for displaying the existing section as a partial display area on the time series; and the similar candidate reduced figure display area includes a similarity obtained by reducing the comment structure of the section of the partial display area displayed in the entire conference time display area. List display means for displaying a list of the reduced candidate figures as many as the number of the partial display areas, and one of the plurality of reduced similar candidate candidates displayed in the list is displayed by the searcher. By detecting that shown 択指, meeting information recording and reproducing apparatus characterized in that it comprises a means for reproducing the audio data of the selected designated interval.