JP2005345508A - Listening support program, listening support device, and listening support method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To inform a congress participant of the existence of a participant's interesting portion, if any, in the contents of the congress. <P>SOLUTION: A computer is functioned as a keyword memory means for memorizing keyword data indicating keyword and registrant data specifying a registrant registering the keyword data by making both the data correspondent to each other, a data acquisition section 91 for acquiring the speech data as a processing object, a speech recognition section 89 for recognizing the speech expressed by the speech data acquired from the processing object acquisition means as a character string, and a notification section 82 for performing prescribed notification processing to the registrant indicated by the registrant data memorized by the keyword memory means by being made correspondent to the keyword data when the keyword indicated by the keyword data memorized by the keyword memory means is included in the character string. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a listening support program, a listening support device, and a listening support method. For example, the present invention relates to a technique for notifying a participant of a conference when there is a part of interest in the content of the conference in a remote conference.

  Conventionally, in a remote conference where a conference is performed via a network, since a speaker and a listener are in a remote place, particularly in a multipoint conference, the main task is performed and the conference is performed as a subsidiary task. Many people can be seen.

  However, if you participate in a teleconference as a secondary task, you will tend to continue to circulate the teleconference pattern, and you may miss the interesting part or miss the opportunity to speak.

  The present invention has been made in view of the above problems, and one of its purposes is to make it possible to notify a conference participant when there is an interesting part of the conference in the content of the conference. To provide a listening support program, a listening support device, and a listening support method.

  A listening support program according to the present invention for solving the above-described problem is a keyword storage means for storing keyword data indicating a keyword and registrant data for specifying a registrant who has registered the keyword data in association with each other. Processing target acquisition means for acquiring voice data as a target, voice recognition means for recognizing a voice represented by the voice data acquired by the processing target acquisition means as a character string, and storing the character string in the keyword storage means When a keyword indicated by the keyword data is included, the computer functions as a notification means for performing a predetermined notification process for the registrant indicated by the registrant data stored in the keyword storage means in association with the keyword data. It is characterized by making it.

  By doing in this way, when there is a part in which the participant is interested in the content of the conference, the conference participant can be notified.

  Further, in the listening support program, speaker storage means for storing speaker data indicating a speaker and registrant data indicating a registrant who has registered the speaker data in association with each other, and the processing target acquisition means The computer further functions as speaker specifying means for specifying a speaker of the voice represented by the voice data acquired by the voice, and the notification means recognizes the voice recognized by the voice recognition means as the character string that is the keyword. When the speaker data indicating the speaker specified as the speaker is included in the speaker data stored by the speaker storage unit, the speaker data is stored in association with the speaker data by the speaker storage unit. The predetermined notification process for the registrant indicated by the registrant data may be performed.

  In this way, it is possible to specify the speaker as well as the keyword and perform the notification process.

  In the listening support program, the computer further functions as a speaker specifying unit that specifies a speaker of a voice represented by the voice data acquired by the processing target acquisition unit, and the keyword storage unit includes the keyword storage unit. Data, speaker data indicating a speaker, and the registrant data are stored in association with each other, and the notification unit stores speaker data indicating the specified speaker by the speaker storage unit. The predetermined notification process is performed when the character string includes a keyword indicated by the keyword data stored in association with the speaker data by the keyword storage means. It may be done.

  Even in this way, it is possible to specify the speaker together with the keyword and perform the notification process.

  In the listening support program, the audio data is acquired by the processing target acquisition unit along with an image including an image of a speaker of the voice represented by the audio data, and the speaker specifying unit The voice speaker may be specified by pattern recognition of the included video of the speaker.

  In the listening support program, the speaker specifying means may specify the speaker of the voice by pattern recognition of the voice print of the voice.

  Further, in the listening support program, when the voice signal storage means for storing the voice acquired by the processing target acquisition means, the character string includes a keyword indicated by the keyword data stored by the keyword storage means, Timing information storage means for storing information indicating the generation timing in the voice of the voice recognized by the voice recognition means as the character string as the keyword, and the voice stored by the voice signal storage means as the generation timing The computer may be caused to further function as audio signal reproduction means for reproduction according to information indicating the above.

  In this way, past processing target data can be reproduced.

  The listening support device according to the present invention includes keyword storage means for storing keyword data indicating a keyword and registrant data for specifying a registrant who has registered the keyword data in association with each other, and voice as a processing target. Processing target acquisition means for acquiring voice data to be represented, voice recognition means for recognizing a voice represented by the voice data acquired by the processing target acquisition means as a character string, and storing the character string in the keyword storage means Notification means for performing a predetermined notification process for the registrant indicated by the registrant data stored in the keyword storage means in association with the keyword data when the keyword indicated by the keyword data is included. It is characterized by.

  Further, the listening support method according to the present invention includes a keyword storage step for storing keyword data indicating a keyword and registrant data for specifying a registrant who has registered the keyword data in association with each other, and voice as a processing target. A processing target acquisition step of acquiring voice data to be represented, a voice recognition step of recognizing a voice represented by the voice data acquired in the processing target acquisition step as a character string, and storing the character string in the keyword storage step A notification step for performing a predetermined notification process for the registrant indicated by the registrant data stored in the keyword storage step in association with the keyword data when the keyword indicated by the keyword data is included. It is characterized by.

  Embodiments of the present invention will be described with reference to the drawings.

  As shown in FIG. 1, the listening support system 1 according to the present embodiment includes a plurality of communication devices 3, a server computer 4, a recording device 5, a display 6, and a client computer 7 that are connected to each other via a communication network 2. The camera 8 and the microphone 9 are included.

  For example, when the listening support system 1 is used as a part of a videophone conference system, the communication network 2 may be a multipoint videophone network or an IP telephone network, that is, the Internet. When the listening support system 1 is used as a part of a videophone system, it may be a telephone network or an IP telephone network, that is, the Internet. Further, when the listening support system 1 is used as a part of the telephone system, it may be a telephone network or an IP telephone network, that is, the Internet.

  The communication device 3 is a device that serves as an interface with the communication network 2. Each communication device 3 may be a part of another device described below, or may be an independent device. When the communication network 2 is a telephone network, the communication device 3 may have a dial function. When the communication network 2 is the Internet, the communication device 3 is, for example, a LAN card or a dial-up modem.

  As illustrated in FIG. 2, the server computer 4 includes a bus 40, a control unit 41, a main storage unit 42, an input / output control unit 43, a communication unit 44, a database 45, and a sub storage unit 46. The control unit 41, the main storage unit 42, and the input / output control unit 43 are connected to each other via the bus 40 and input / output data. The communication unit 44, the database 45, and the sub storage unit 46 are connected to the input / output control unit 43 and similarly perform data input / output. The server computer 4 can also be used as a multipoint video conference server.

  The control unit 41 controls each unit of the server computer 4. The main storage unit 42 operates as a work memory of the control unit 41 and holds programs and parameters related to various processes performed by the control unit 41. The input / output control unit 43 controls each unit connected to the input / output control unit 43 in accordance with an instruction from the control unit 41, and outputs data input from the respective unit to the control unit 41 or controls the control unit 41. The data input from is output to the respective units. The communication unit 44 transmits / receives data to / from the communication network 2. Then, the received data is output to the input / output control unit 43, and the data input from the input / output control unit 43 is transmitted to the communication network 2. As the database 45, for example, a conventionally known relational database can be used. The secondary storage unit 46 is a hard disk or an external storage medium, stores various data used by the server computer 4, and stores a program according to the present embodiment for operating the control unit 41. Then, the control unit 41 expands the program in the main storage unit 42 and executes it.

  When the server computer 4 is used as a multipoint video conference server, the control unit 41 synthesizes processing target data that is video data and audio data from a plurality of cameras 8 and microphones 9 received by the communication unit 44. And transmitted to the plurality of displays 6.

  The recording device 5 can use a conventionally known recording means such as a video deck or a DVD recorder. Moreover, it is good also as recording on the secondary memory | storage part 77 of the client computer 7 mentioned later. The recording device 5 is connected to the communication device 3, can record audio data and video data received by the communication device 3, and can output the recorded data to the display 6.

  The display 6 can use a conventionally known image display means such as a television or a CRT, and generally includes a speaker. The display 6 is connected to the communication device 3 and can output audio data and video data received by the communication device 3 and can also output recording data input from the recording device 5.

  As shown in FIG. 3, the client computer 7 includes a bus 70, a control unit 71, a main storage unit 72, an input / output control unit 73, a communication unit 74, an operation unit 75, a display unit 76, and a sub storage unit 77. Has been. The control unit 71, the main storage unit 72, and the input / output control unit 73 are connected to each other via the bus 70 and input / output data. The communication unit 74, the operation unit 75, the display unit 76, and the sub storage unit 77 are connected to the input / output control unit 73 and similarly perform data input / output.

  The control unit 71 controls each unit of the client computer 7. The main storage unit 72 operates as a work memory of the control unit 71 and holds programs and parameters related to various processes performed by the control unit 71. Also, processing such as display on the display 6 and change of volume is performed. The input / output control unit 73 controls each unit connected to the input / output control unit 73 in accordance with an instruction from the control unit 71, outputs data input from the respective unit to the control unit 71, and controls the control unit 71. The data input from is output to the respective units. The communication unit 74 transmits and receives data to and from the communication network 2. Then, the received data is output to the input / output control unit 73 and the data input from the input / output control unit 73 is transmitted to the communication network 2. The operation unit 75 includes a mouse and a keyboard, for example, and receives an input from a user of the client computer 7 and outputs the input to the input / output control unit 73. The display unit 76 includes, for example, a liquid crystal display device, and displays and outputs information according to a signal input from the input / output control unit 73. The display 6 may be the display unit 76. The secondary storage unit 77 is a hard disk or an external storage medium, and stores various data used by the client computer 7 or stores, for example, a program according to the present embodiment for operating the control unit 71. Then, the control unit 71 develops the program in the main storage unit 72 and executes it.

  Next, FIG. 4 is a functional block diagram when the listening support system 1 is used as a video conference system. As shown in FIG. 4, the listening support system 1 includes a communication start unit 80, a reproduction instruction unit 81, a notification unit 82, a speaker setting unit 83, a keyword setting unit 84, a reproduction operation setting unit 85, a notification setting unit 86, The reproduction unit 87, the verification unit 88, the voice recognition unit 89, the recording medium 90, the data acquisition unit 91, the camera 8, and the microphone 9 are configured. All these units may be in one client computer 7, or a part may be in the server computer 4. Here, it is assumed that all are present in one client computer 7, and the processing when moving to the server computer 4 will be described later.

  The communication start unit 80 starts communication between the client computer 7 and the communication network 2 and starts communication with the communication device 3 that is a partner of the video conference. When communication is started, the data acquisition unit 91 starts acquisition of processing target data that is video data and audio data from the camera 8 and the microphone 9 connected to the communication device that is the communication partner. An example of the video data in this case is shown in FIG. The video data includes the upper body image 20 of the other party of the video conference.

  The recording medium 90 is a recording medium used in the recording device 5 and sequentially records the processing target data acquired by the data acquisition unit.

  The voice recognition unit 89 recognizes the voice data acquired by the data acquisition unit 91 as a character string. Various types of voice recognition software that has been sold in the past can be used as the voice recognition means. Then, the video data and the audio data acquired as a character string are output to the matching unit 88.

  The matching unit 88 determines whether or not the keyword indicated by the keyword data set in the keyword setting unit 84 to be described later is included in the character string input from the voice recognition unit 89. Specifically, the character string may be divided into words every predetermined time, acquired as a word set, and it may be determined whether or not a keyword is included in the word set. It is also possible to sequentially extract words from the sequentially acquired audio data and determine whether or not the words are included in the keyword. Furthermore, the timing at which the character string that is the keyword is included in the voice data may be stored. Specifically, for example, the number of seconds after the start of the video conference may be stored, or the time may be stored.

  Further, the collation unit 88 determines whether or not a speaker indicated by speaker data set in a speaker setting unit 83 described later is included in the video data or audio data input from the voice recognition unit 89. It is good. That is, it is determined whether or not the speaker indicated by the speaker data has hit the keyword. Specifically, the video of the speaker is extracted by pattern recognition of the video data, and it is determined whether or not the video of the speaker can be determined as the speaker indicated by the speaker data stored by the neural network. It is also possible to extract the voiceprint of the speaker by pattern recognition of the voice data and determine whether the speaker's voiceprint can be determined as the speaker indicated by the speaker data stored by the neural network. It is good to do.

  As described above, the matching unit 88 can acquire speakers and keywords included in the processing target data. Note that either the speaker or keyword acquisition process may be performed first. For example, when the speaker is determined as the speaker indicated by the speaker data, it may be determined whether or not the keyword is included in the character string, or when the keyword is included in the character string. May be determined as the speaker indicated by the speaker data.

  Here, processing in the speaker setting unit 83, the keyword setting unit 84, the reproduction operation setting unit 85, and the notification setting unit 86 will be described in detail.

  The speaker setting unit 83, the keyword setting unit 84, the reproduction operation setting unit 85, and the notification setting unit 86 are function units for setting data described later by the user of the client computer 7 in the operation unit 75. Specifically, for example, by displaying a menu screen as shown in FIG. 6 on the display unit 76, the user may set each data using the GUI. The speaker setting unit 83, the keyword setting unit 84, the reproduction operation setting unit 85, and the notification setting unit 86 will be described with reference to FIG.

  The speaker setting unit 83 is a functional unit for setting speaker data in order to register speaker data. In FIG. 6, the speaker setting unit 83 is set by inputting speaker data in the input field 22 for inputting members. be able to. As the speaker data, the name of the speaker is used here. A plurality of speaker data may be set.

  The keyword setting unit 84 is a functional unit for setting keyword data in order to register keyword data. In FIG. 6, the keyword setting unit 84 can be set by inputting keyword data in the input field 23 for inputting a remark keyword. . Here, the character string itself used as the keyword is used as the keyword data. It is also possible to set a plurality of keyword data.

  The notification setting unit 86 is set by inputting a processing method when performing notification processing in the notification unit 82 described later. Here, “icon blinking” is set, and for example, the setting of performing notification processing to the user when the icon 21 in FIG. 5 blinks is set. In addition, for example, notification may be performed by increasing the volume, or notification may be performed by ringing a pager. Among these, it may be set to perform a plurality of notification processes.

  The reproduction operation setting unit 85 is set by inputting a method for reproducing the processing target data sequentially recorded on the recording medium 90 in the reproduction unit 87 described later. As the playback method, for example, “playing back from one minute before the timing when the keyword is present (at the time of speech)” can be set, and for example, “the meeting pattern is displayed on the display 6 in real time. It is also possible to set the display method, for example, “reproduce by sub-screen display on display 6 while continuing to flow”. Of course, the display method can also be set such as “do not play”.

  Note that the speaker setting unit 83, the keyword setting unit 84, the playback operation setting unit 85, and the notification setting unit 86 can perform setting processing for each unit when the icon 26 in FIG. 6 is pressed. In the setting process, each set data is stored in the main storage unit 72 so that the collation unit 88 can refer to the keyword data or the speaker data, or when the reproduction unit 87 reproduces the data, This is processing such that the unit 81 can refer to the playback method, or can refer to the notification method when the notification unit 82 performs notification processing. In this case, for example, a keyword and a speaker may be stored in association with each other. That is, when a plurality of speakers are set, different keywords may be stored for each speaker. In this way, the collation unit 88 can determine whether or not the keyword is included in the character string depending on whether or not the speaker is one of a plurality of speakers. That is, a keyword to be notified for each speaker can be set.

  Next, processing of the reproduction instruction unit 81 and the reproduction unit 87 will be described. The reproduction instruction unit 81 determines that the keyword is included in the collation unit 88 and the reproduction instruction set in the reproduction operation setting unit 85 when it is determined that the speaker is a speaker set as speaker data. The reproduction unit 87 is instructed to reproduce the processing target data recorded on the recording medium 90 according to the method. A specific example of this reproduction is shown in FIG. FIG. 7 shows the upper body image 27 of the other party of the video conference as a video image as in FIG. Further, an icon 28 is displayed. When the user clicks on the icon 28, the reproduction is terminated and the process returns to the normal video conference video. In this case, the processing target data acquired by the data acquisition unit 91 may be continuously recorded during reproduction, or the telephone is disconnected once during reproduction and reconnected when the icon 28 is clicked. Also good.

  The notification unit 82 performs a predetermined notification process when it is determined by the verification unit 88 that the keyword is included and the speaker is determined to be a speaker set as speaker data. As described above, the predetermined notification processing is performed by the notification method set in the notification setting unit 86. By the notification process, the user can know that the other party of the videophone has spoken the keyword. Furthermore, it is possible to know that the person who has spoken the keyword is a set person.

  In the above description, it has been described that each unit is present in the client computer 7. Next, processing when a part of processing is performed in the server computer 4 will be described.

  When processing is performed in the server computer 4, the data set by the speaker setting unit 83, the keyword setting unit 84, the reproduction operation setting unit 85, and the notification setting unit 86 of the plurality of client computers 7 are stored in the database 45. Remember. When storing, each data is stored in association with registrant data that identifies the registrant of each data who is the user of each client computer 7. As the registrant data, registrant data corresponding to the notification process in the notification unit 82 is stored. For example, when the notification process is blinking of an icon or increase in volume, the IP address of the client computer 7 can be assumed. If the notification process is ringing of the pager, it can be the address of the pager. Furthermore, when the server computer 4 is used as a multi-point video conference server, it can be data for synthesizing video for notification with video for each display 6. A specific example of the table stored in the database 45 is shown in FIG. As shown in FIG. 9, registrant data, keyword data, speaker data, a notification method, and a reproduction method can be stored in association with each other. And when these are stored in association with each other, when a specific speaker hits a specific keyword, it is possible to notify by a specific notification method and reproduce by a specific reproduction method. Become. However, all of these data may not be associated but may be stored individually associated with each other.

  The server computer 4 may also be provided with a data acquisition unit 91 to acquire processing target data, and the processing target data may be subjected to the above processing in the voice recognition unit 89 and the verification unit 88 that are also installed in the server computer 4. . Further, the secondary storage unit 46 of the server computer 4 may be used as the recording medium 90. Furthermore, the reproduction instruction unit 81, the reproduction unit 87, and the notification unit 82 may be installed in the server computer 4. In the following description, it is assumed that the voice recognition unit 89, the collation unit 88, the recording medium 90, the reproduction instruction unit 81, the reproduction unit 87, and the notification unit 82 are installed in the server computer 4. Furthermore, it demonstrates as what uses the server computer 4 as a multipoint video conference server.

  The voice recognition unit 89 performs processing for acquiring the processing target data acquired by the data acquisition unit 91 as the character string described above. Then, the collation unit 88 determines whether or not the keyword stored in the database 45 is included in the character string. If included, a speaker indicated by the speaker data stored in the database 45 in association with the registrant data stored in association with the keyword is input from the speech recognition unit 89 by the above processing. It may be determined whether it is included in video data or audio data.

  If it does in this way, a keyword can be detected for every registrant data, and the speaker of the keyword can be acquired. And the notification part 82 performs a notification process with respect to registrant data by the notification method memorize | stored in association with this registrant data. Similarly, when the reproduction instruction unit 81 receives an instruction from the registrant specified by the registrant data (for example, clicking on the icon 21) from the client computer 7 used by the registrant, the display corresponding to the client computer is displayed. 6 is combined with the video recorded on the recording medium 90 to be played back by the playback unit 87 according to the playback method stored in association with the registrant data.

  As described above, the server computer 4 is used to detect a keyword for each registrant data, acquire a speaker of the keyword, execute a notification process, and transmit a playback video. be able to.

  The case where the above processing is performed by one client computer 7 will be described with reference to a flowchart.

  FIG. 8 is a flowchart showing an example of processing performed in the client computer 7.

  First, processing target data that is audio data and video data is acquired (S100). Here, the description will be made assuming that the processing target data is acquired by dividing every time. Then, a speaker included therein is specified (S102), and voice data is converted into a character string by voice recognition processing (S104). Then, the character string is divided into words, and the words, speakers, and utterance times are stored in association with each other (S106). In addition, you may memorize | store the process target data divided | segmented for every time in correlation with the start time. In this case, however, it is desirable that the speaker is associated with a word. Note that phrases can be used even if they are not words. In the case of a phrase, it has a group of voice characteristics by itself, so it is considered more suitable for voice recognition.

  Then, it is determined whether or not each of the above words is included in the stored keyword (S108). When a phrase is stored in place of a word as described above, the keyword input as a word becomes a part of the phrase, so it is also determined whether or not the keyword is included in a part of the phrase. There is a need. If a keyword is included in a part of the phrase, it is determined that the phrase is included in the stored keyword. If it is determined that it is included, the process proceeds to the next process of S110, and if it is determined that it is not included, the process returns to S100 to acquire the next process target data.

  In the process of S110, it is determined whether or not the speaker stored in association with the word including the keyword is included in the stored speaker. That is, it is determined whether or not the person who spoke the keyword is a speaker registered by the registrant. If the speaker is stored, the process proceeds to the next process of S112. If it is determined that the speaker is not included, the process returns to S100 to acquire the next process target data. Of course, since there may be a plurality of keywords, processing is performed for each keyword, and if even one of the stored keywords is a spoken keyword, the process proceeds to the next step S112.

  In S112, a predetermined notification process for the registrant is performed. As described above, this is a process of blinking the icon or increasing the volume.

  Then, it is determined by monitoring whether or not replaying is requested by the registrant (S114). That is, it is determined whether or not reproduction is requested by a reproduction method set in advance by a registrant. When it is determined that the request has been made, the process proceeds to the next process of S116. When it is determined that the request has not been made (when time-out occurs), the process returns to S100 to acquire the next process target data.

  In S116, the calculation is performed from the utterance time stored in association with the word including the keyword spoken by the speaker, and the reproduction process is performed from the time one minute before. At this time, when new data to be processed is sequentially recorded by the recording medium 90, it is always read out until the registrant requests to stop reproduction by the next processing. It is also possible to continue playing the data to be processed.

  Next, it is monitored and judged whether the end of reproduction is requested or whether the reproduction is completed to the end (S118). If an affirmative determination is made, the reproduction ends and the normal video conference is resumed (S120). Then, the process returns to S100 to acquire the next processing target data. If a negative determination is made, the determination process is repeated until an affirmative determination or timeout occurs.

  By doing as described above, when there is a part in which the participant is interested in the content of the conference, the conference participant can be notified. Then, an interesting part can be specified by a keyword included in the speaker's speech. It is also possible to change the process depending on the speaker who spoke the keyword.

  The present invention is not limited to the above embodiment.

  For example, the present invention can be applied to other than video conference. That is, the present invention can be applied to cases where it is necessary to extract keywords from voice in real time. In addition, it is possible to specify only a speaker and perform a predetermined notification process without extracting a keyword. In this way, for example, when a specific speaker starts speaking in a video conference, a predetermined notification process can be performed.

It is a block diagram which shows the structure of the listening assistance system concerning embodiment of this invention. It is a block diagram which shows the hardware constitutions of the server computer concerning embodiment of this invention. It is a block diagram which shows the hardware constitutions of the client computer concerning embodiment of this invention. It is a functional block diagram of the listening assistance system concerning an embodiment of the invention. It is an example of a display in the display concerning embodiment of this invention. It is an example of a display in the display concerning embodiment of this invention. It is an example of a display in the display concerning embodiment of this invention. It is a flowchart of the process concerning embodiment of this invention. It is a table memorize | stored in the database concerning embodiment of this invention.

Explanation of symbols

1 listening support system, 2 communication network, 3 communication device, 4 server computer, 5 recording device, 6 display, 7 client computer, 8 camera, 9 microphone, 40 bus, 41, 71 control unit, 42, 72 main storage unit, 43, 73 I / O storage unit, 44, 74 communication unit, 45 database, 46, 77 secondary storage unit, 75 operation unit, 76 display unit, 80 communication start unit, 81 playback instruction unit, 82 notification unit, 83 speaker setting Unit, 84 keyword setting unit, 85 reproduction operation setting unit, 86 notification setting unit, 87 reproduction unit, 88 collation unit, 89 voice recognition unit, 90 recording medium, 91 data acquisition unit.

Claims (8)

Keyword storage means for storing keyword data indicating a keyword and registrant data specifying a registrant who registered the keyword data in association with each other,
Processing target acquisition means for acquiring audio data as a processing target;
A voice recognition unit for recognizing a voice represented by the voice data acquired by the processing target acquisition unit as a character string, and a keyword indicated by the keyword data stored by the keyword storage unit in the character string Notification means for performing a predetermined notification process for the registrant indicated by the registrant data stored in the keyword storage means in association with the keyword data;
Listening support program characterized by causing a computer to function as In the listening support program according to claim 1,
Speaker data indicating a speaker and registrant data indicating a registrant who registered the speaker data are stored in association with each other, and the voice data acquired by the processing target acquisition unit is stored. Speaker identification means for identifying the speaker of the voice to be represented,
Further functioning the computer as
The notification means includes speaker data indicating a speaker identified as a voice speaker recognized by the voice recognition means as a character string that is the keyword, in speaker data stored by the speaker storage means. If included, the predetermined notification process for the registrant indicated by the registrant data stored in the speaker storage means in association with the speaker data is performed.
Listening support program characterized by this. In the listening support program according to claim 1,
A speaker identification unit that identifies a speaker of the voice represented by the voice data acquired by the processing target acquisition unit;
Further functioning the computer as
The keyword storage means stores the keyword data, speaker data indicating a speaker, and the registrant data in association with each other,
The notification means is a case in which speaker data indicating the specified speaker is included in speaker data stored by the speaker storage means, and the character string is stored in the character string by the speaker storage means. When the keyword indicated by the keyword data stored in association with is included, the predetermined notification process is performed.
Listening support program characterized by this. In the listening support program according to claim 2 or 3,
The audio data is acquired by the processing target acquisition unit along with an image including an image of an audio speaker represented by the audio data,
The speaker specifying means specifies the speaker of the voice by pattern recognition of the video of the speaker included in the video.
Listening support program characterized by this. In the listening support program according to claim 2 or 3,
The speaker specifying means specifies the speaker of the voice by pattern recognition of the voice print of the voice;
Listening support program characterized by this. The listening support program according to any one of claims 1 to 5,
Audio signal storage means for storing the sound acquired by the processing object acquisition means;
When the character string includes the keyword indicated by the keyword data stored by the keyword storage unit, the generation timing in the voice of the voice recognized by the voice recognition unit as the character string that is the keyword is indicated. Timing information storage means for storing information,
Audio signal reproduction means for reproducing the audio stored by the audio signal storage means in accordance with the information indicating the generation timing;
As a listening support program, the computer further functions as: Keyword storage means for storing keyword data indicating a keyword and registrant data for specifying a registrant who registered the keyword data in association with each other;
Processing target acquisition means for acquiring voice data representing voice as a processing target;
Voice recognition means for recognizing a voice represented by the voice data acquired by the processing target acquisition means as a character string;
When the character string includes a keyword indicated by the keyword data stored by the keyword storage means, a predetermined notification to the registrant indicated by the registrant data stored by the keyword storage means in association with the keyword data Notification means for processing;
A listening support apparatus comprising: A keyword storage step for storing keyword data indicating a keyword and registrant data for identifying a registrant who has registered the keyword data in association with each other;
A processing target acquisition step for acquiring voice data representing a voice as a processing target;
A voice recognition step for recognizing the voice represented by the voice data acquired in the processing target acquisition step as a character string;
When the character string includes a keyword indicated by the keyword data stored in the keyword storage step, a predetermined notification to the registrant indicated by the registrant data stored in the keyword storage step in association with the keyword data A notification step for processing;
Listening support method characterized by including.

Images