JP2006157692A - Video reproducing method and device thereof, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To listen to user voice comments, without their overlapping, while viewing video. <P>SOLUTION: Video reproduction is started, presence/non-presence of user voice is determined, it is determined whether the voice is non-language; and when it is non-language, the non-language is reproduced, or if the voice is other than the non-language, a user utterance phase of highest appraise within an utterance phase is reproduced. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a video playback method, apparatus, and program, and more particularly, to a video playback method, apparatus, and program for playing back audio of a video viewer together with video.

  Nowadays, a variety of information is presented to users by presenting text and still images as additional information along with the video.

  In such a system, when a text or a still image is presented, even when a plurality of pieces of information are presented at the same time, the user can selectively read and view necessary information. However, when a plurality of voices are to be presented simultaneously as additional information, if the voices are played back at the same time, the user cannot understand the contents due to the duplication of the voices. There are several ways to do this.

  As a conventional first method, there is a method in which the voice of another speaker is cut according to an utterance request from a specific speaker and the utterance of the speaker is preferentially reproduced (see, for example, Patent Document 1). ).

Further, as a second conventional method, there is a method of facilitating the discrimination of the partner who is the center of the conversation by controlling the gain of each user based on the user's utterance history (see, for example, Patent Document 2). .
Japanese Patent Laid-Open No. 4-191350 JP 2000-49948 A

  However, in the first conventional method, when another speaker speaks in duplicate with the voice of the specific speaker, the other speaker is uttered at the start and end of the speech of the specific speaker. There is a problem that the utterance of is interrupted or reproduced from the middle. For example, when a specific speaker starts speaking while a certain speaker is speaking, the utterance of the speaker originally speaking is interrupted in the middle, which may hinder understanding of the content.

  Further, the second conventional method has a problem that it cannot be used for the purpose of facilitating discrimination of a person who is not the center of conversation. For example, the user may want to hear the voice of a person who is not the center of the conversation, but cannot be used for such a purpose.

  The present invention has been made in view of the above points, and an object of the present invention is to provide a video reproduction method, apparatus, and program capable of listening to a user's voice comment without overlapping.

  FIG. 1 is an explanatory diagram of the principle of the present invention.

The present invention (Claim 1) is a video playback method for playing back a group of voices of a plurality of users who are viewing the same video together with the video.
A playback start step (step 1) for reading video and starting playback;
Refer to the audio section information file that stores the audio section consisting of the start time and end time of the utterance for the video provided for each user, and based on whether the start time and the end time are set, An audio presence determination step (step 2) for determining whether audio is stored in a viewer audio file provided for each user;
A non-language determining step (step 3) of determining whether or not the sound is non-language when the sound is stored in the viewer's sound file, and reproducing the sound if the sound is non-language;
In the non-language determination step (step 3), when it is determined that the language is not non-language, the user's voice corresponding to the voice section stored in the voice section information file is predetermined for each user. A sound reproduction step (step 4) of referring to the user evaluation information file in which the value is stored based on the user ID and reproducing the sound of the user having the highest evaluation value;
For the next audio segment that has been reproduced in the audio reproduction step (step 4), the audio presence determination step (step 2), the non-language determination step (step 3), and the audio reproduction step (step 4) are performed in the video file. And repeat step (step 5) until it is finished.

The present invention relates to a video playback method for playing back a group of voices of a plurality of users who are viewing the same video together with the video.
A playback start step for reading video and starting playback;
Refer to the audio section information file that stores the audio section consisting of the start time and end time of the utterance for the video provided for each user, and based on whether the start time and the end time are set, An audio presence determination step for determining whether audio is stored in a viewer audio file provided for each user;
A non-language determination step of determining whether or not the sound is non-language when the sound is stored in the viewer sound file, and reproducing the sound if the sound is non-language;
If it is determined in the non-language determination step that the language is not non-language, an evaluation value predetermined for each user is stored for the user's voice corresponding to the voice section stored in the voice section information file. A sound reproduction step of referring to the user evaluation information file being determined based on the user ID, determining a volume to be reproduced based on the evaluation value, and reproducing sound;
A voice presence determination step, a non-language determination step, and a repetition step of repeating the voice playback step until the video of the video file is completed are performed on the next voice section that has been played back by the voice playback step.

The present invention (Claim 3) is a video playback method for playing back a group of voices of a plurality of users who are viewing the same video together with the video.
A playback start step for reading video and starting playback;
Refer to the audio section information file that stores the audio section consisting of the start time and end time of the utterance for the video provided for each user, and based on whether the start time and the end time are set, An audio presence determination step for determining whether audio is stored in a viewer audio file provided for each user;
When audio is stored in the viewer audio file, it is determined whether or not the audio is non-language, and a group of users other than non-language who does not overlap utterances is obtained by searching all audio section information files A first user group search step to:
A first audio reproduction step of reproducing audio by reading out audio of all users belonging to the group from a viewer audio file based on the group of users acquired by the first user group search step;
A second user group search step of searching for a group of users whose utterances are not duplicated among the users not selected in the first user group search step by searching the voice section information file of the user;
A second audio reproduction step of reading out audio of all users of the group searched in the second user group search step from the viewer audio file and reproducing the audio retroactively together with the video;
A repetition step of repeating the second user group search step and the second sound reproduction step is performed until the reproduction of all the user's sounds is completed.

The present invention (Claim 4) is a video playback method for playing back a group of voices of a plurality of users who are viewing the same video together with the video.
A playback start step for reading video and starting playback;
Among the viewer audio files provided for each user, with respect to unreproduced user audio, the evaluation value is maximized by referring to a user evaluation information file in which an evaluation value predetermined for each user is stored. A first audio reproduction step of reading out and reproducing the user's audio from the user's viewer audio file;
All audio section information for storing the audio section consisting of the start time and end time of the utterance for the video provided for each user, with the voice of another user falling within the time interval until the next utterance of the user reproduced last time An utterance candidate search step obtained by searching a file;
A second audio reproduction step of reading out and reproducing the user's voice having the next highest evaluation value from the user's voice reproduced in the utterance candidate search step from the user's viewer voice file;
Repeating the utterance candidate search step and the second sound reproduction step until all user sounds are reproduced.

According to the present invention (Claim 5), in the sound reproduction step of Claim 1,
When the user's voice having the highest evaluation value in the voice section is read from the viewer voice file and played back,
Overlapping reproduction of audio sections with a predetermined time margin T is allowed.

The present invention (Claim 6) is a video playback method for playing back a plurality of audio groups of users who are viewing the same video together with the video.
A playback start step for reading video and starting playback;
Refer to the audio section information file that stores the audio section consisting of the start time and end time of the utterance for the video provided for each user, and based on whether the start time and the end time are set, An audio presence determination step for determining whether audio is stored in a viewer audio file provided for each user;
A non-language determination step of reproducing the sound if the sound is stored in the viewer's sound file, if the sound is non-language;
In the non-language determination step, for the first sound determined not to be non-language, the start time of the first sound and the start of the second sound of another user uttered next to the first sound If the difference from the time is shorter than the section length of the first sound, a sound playback step for shortening and playing back the first sound;
A repeat step of repeating the voice presence determination step, the non-language determination step, and the voice playback step is performed on all the voice segments stored in the voice segment information file.

  FIG. 2 is a principle configuration diagram of the present invention.

The present invention (Claim 7) is a video playback device that plays back a group of voices of a plurality of users who are viewing the same video together with the video,
A video file 101 storing video;
A plurality of audio section information files 210 for storing audio sections including start time and end time of utterance for video provided for each user;
A plurality of viewer audio files 10 each storing audio of the user provided for each user;
A user evaluation information file 108 storing evaluation values predetermined for each user;
Playback means 110 for playing back video and audio;
Reproduction control means 120 for controlling the reproduction of sound,
The reproduction means 110
Video playback start means for reading video from the video file 101 and starting playback;
Based on an instruction from the reproduction control unit 120, and reproduces the audio in synchronization with the video,
The reproduction control means 120 is
Audio presence determination means for referring to the audio section information file 20 and determining whether audio is stored in the viewer audio file 10 based on whether the start time and the end time are set;
Non-language determination means for determining whether or not the sound is non-language when the sound is stored in the viewer sound file 10;
When the non-language determining unit determines that the language is not non-language, the user evaluation information file 108 is referred to based on the user ID for the user's voice corresponding to the voice section stored in the voice section information file 20. A voice reproduction control unit that causes the reproduction unit to reproduce the voice of the user having the highest evaluation value, and causes the reproduction unit to reproduce the non-language voice when the non-language determination unit determines the non-language.
Repeating means for repeating the sound presence determining means, the non-language determining means, and the sound playback control means until the video of the video file is completed for the next voice section played back by the playback means 110;
Have

The present invention (Claim 8) is a video reproduction for reproducing a group of users of a plurality of users who are viewing the same video together with the video,
A video file containing the video,
A plurality of audio section information files for storing audio sections consisting of start time and end time of utterance for video provided for each user;
A plurality of viewer audio files storing the user's audio provided for each user;
A user evaluation information file storing evaluation values predetermined for each user;
Playback means for playing back video and audio;
Playback control means for controlling the playback of audio,
Reproduction means
Video playback start means for reading video from a video file and starting playback;
Based on an instruction from the reproduction control means, and has a means for reproducing the sound in synchronization with the video,
The playback control means
Audio presence determination means for referring to an audio section information file and determining whether audio is stored in the viewer audio file based on whether the start time and the end time are set;
Non-language determination means for determining whether or not the sound is non-language when the sound is stored in the viewer sound file;
When the non-language determination unit determines that the language is not non-language, the user evaluation information file is stored based on the user ID of the user for the user's voice corresponding to the voice section stored in the voice section information file. The sound volume to be played back by the playback means is determined based on the evaluation value, and the sound for each user is played back by the playback means at the volume, and if the non-language determination means determines that the language is non-language, Audio reproduction control means for causing the reproduction means to reproduce the sound of
Repeating means for repeating the sound presence determining means, the non-language determining means, the volume control means, and the sound reproduction control means until the video of the video file is finished for the next voice section that has been played back,
Have

The present invention (Claim 9) is a video playback device that plays back a group of voices of a plurality of users who are viewing the same video together with the video,
A video file containing the video,
A plurality of audio section information files for storing audio sections consisting of start time and end time of utterance for video provided for each user;
A plurality of viewer audio files storing the user's audio provided for each user;
Playback means for playing back video and audio;
Playback control means for controlling the playback of audio,
Reproduction means
Video playback start means for reading video from a video file and starting playback;
Based on an instruction from the reproduction control means, and has a means for reproducing the sound in synchronization with the video,
The playback control means
Audio presence determination means for referring to an audio section information file and determining whether audio is stored in the viewer audio file based on whether the start time and the end time are set;
Non-language determination means for determining whether or not the sound is non-language when the sound is stored in the viewer sound file;
First user group search means for acquiring a group of users whose utterances other than non-languages are not duplicated by searching all voice section information files;
First audio reproduction control means for reading out voices of all users belonging to the group from the viewer audio file based on the user group acquired by the first user group search means and causing the reproduction means to reproduce the audio; ,
A second user group search means for searching for a group of users whose utterances are not duplicated among the users not selected by the first user group search means by searching the voice section information file;
Second audio reproduction control means for reading out voices of all users belonging to the group searched by the second user group search means from the viewer audio file and causing the reproduction means to play back together with the video;
And repeating means for repeating the second user group search means and the second sound reproduction control means until the reproduction of the sounds of all the users is completed.

The present invention (Claim 10) is a video playback device that plays back a group of voices of a plurality of users who are viewing the same video together with the video,
A video file containing the video,
A plurality of audio section information files for storing audio sections consisting of start time and end time of utterance for video provided for each user;
A plurality of viewer audio files storing the user's audio provided for each user;
A user evaluation information file storing evaluation values predetermined for each user;
Playback means for playing back video and audio;
Audio reproduction control means for controlling audio reproduction,
Reproduction means
Video playback start means for reading video from a video file and starting playback;
Based on an instruction from the reproduction control means, and has a means for reproducing the sound in synchronization with the video,
The sound reproduction control means
First audio reproduction for reading out the user's voice having the maximum evaluation value from the viewer's voice file and playing it back to the playback means with reference to the user evaluation information file for the voice of the unplayed user in the viewer's voice file Control means;
Utterance candidate search means for acquiring other user's voice that fits in the time interval until the next utterance of the user reproduced last time by searching all voice section information files;
Of the user's voices acquired by the utterance candidate search means, the user evaluation information file is referred to, and the voice of the user having the second highest evaluation value after the previous playback is read from the user's viewer voice file and played back. Second sound reproduction control means for reproducing the means;
And repeating means for repeating the utterance candidate search means and the second sound reproduction control means until all user sounds are reproduced.

According to the present invention (claim 11), in the sound reproduction control means of claim 7,
When the voice of the user having the highest evaluation value in the voice section is read from the viewer voice file and played back by the playback means,
Means for allowing reproduction of audio sections of a predetermined time margin T in duplicate.

The present invention (Claim 12) is a video playback device that plays back a group of voices of a plurality of users who are viewing the same video together with the video,
A video file containing the video,
A plurality of audio section information files for storing audio sections consisting of start time and end time of utterance for video provided for each user;
A plurality of viewer audio files storing the user's audio provided for each user;
Playback means for playing back video and audio;
Audio reproduction control means for controlling audio reproduction,
Reproduction means
Video playback start means for reading video from a video file and starting playback;
Based on an instruction from the reproduction control means, and has a means for reproducing the sound in synchronization with the video,
The sound reproduction control means
Audio presence determination means for referring to an audio section information file and determining whether audio is stored in the viewer audio file based on whether the start time and the end time are set;
Non-language determination means for determining whether or not the sound is non-language when the sound is stored in the viewer sound file;
The non-language determining means causes the playback means to reproduce the sound determined to be non-language, and for the first sound determined not to be non-language, the start time of the first sound and the first sound If the difference from the start time of the second voice of another user uttered next is shorter than the section length of the first voice, the first voice is shortened and played back on the playback means. Voice reproduction control means for
Repeating means for repeating the voice presence determining means, the non-language determining means, and the voice reproduction control means for all the voice sections stored in the voice section information file.

The present invention (Claim 13) is a video playback program for playing back a group of voices of a plurality of users who are viewing the same video together with the video,
A program for causing a computer to execute processing for realizing the video playback method according to claim 1.

  As described above, according to the present invention, it is possible to listen to the voices of a plurality of users without hindering listening when reproducing the voice of the video viewer together with the video.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 3 shows a system configuration according to an embodiment of the present invention. The system shown in FIG. 1 includes a plurality of files stored in a storage medium, a computer 100 used as a video playback device, and a monitor 200 including a speaker that outputs a processing result of the computer 100.

  The files include a video file 101, a viewer audio file A102, an audio segment file A103, a viewer audio file B104, an audio segment file B105, a viewer audio file C106, an audio segment file C107, and a user evaluation information file 108. In the figure, the number of groups of the viewer audio file and the audio section file is three, but this number may be arbitrary as long as it is one or more. For example, when listening to the sound of two people who are viewing a video, the viewer audio file C106 and the audio section file 107 are not necessary, and N viewer audio files and audio section files are provided for N viewers. If this is prepared, the present invention can be realized.

  The video file 101 is a file that the user views.

  The viewer audio file A102 is an audio file obtained by recording a voice uttered by a viewer A while watching the video file 101 with a microphone or the like (not shown). Here, it is assumed that the time in the video file 101 and the time in the viewer audio file A102 are synchronized. That is, the sound uttered at a certain time in the video file 101 is recorded in association with the time in the viewer sound file A102.

  The audio section file A103 is a file having the start time and end time of the section group in which the voice is uttered in the viewer audio file A102. An example of the speech segment file is shown in FIG. The voice section file stores the start time / end time of utterance associated with the voice ID. Further, the computer 100 may generate information corresponding to the voice section file A103 without the voice section file A103.

  The viewer audio file B104 is an audio file issued while viewing the video file 101 by a user other than the user who generated the audio of the viewer audio file A102. Here, it is assumed that the video file 101 is synchronized with the viewer audio file A102.

  The audio section file B105 is a file having the start time / end time of the section group in which the voice is uttered in the viewer audio file B104.

  The viewer audio file C106 is an audio file generated while a user other than the user who generated the audio of the viewer audio file A102 and the user who generated the audio of the viewer audio file B104 is viewing the video file 101. . Here, the video file 101 is synchronized in the same manner as the viewer audio file A102.

  The audio section file C107 is a file having the start time / end time of the section group in which the voice is uttered in the viewer audio file C106.

  The viewer audio files 102, 104, and 106 have a video ID, a scene ID to which audio is input, an audio ID, and audio, and are synchronized with the video file 101 by the scene ID.

  The user evaluation information file 108 is uttered by the user (user A) who utters the viewer audio file A102, the user (user B) who utters the viewer audio file B104, and the user (user C) who utters the viewer audio file C104. Information on the importance of For example, as shown in FIG. 5, the importance of the utterance is designated by a numerical value from 0 to 100 for each user. Note that the user evaluation information file 108 is generated, for example, by performing voting in advance, performing evaluation for each user based on the number of votes, and assigning and storing an evaluation value.

  The computer 100 performs various processes such as input of a video file 101, a viewer audio file A102, an audio section file C106, an audio section file C107, and a user evaluation information file 108, transmission of video and audio to the monitor 200, and the like.

  The monitor 200 displays the video sent from the computer 100 and presents it to the user.

  In the system shown in FIG. 3, the video and the audio are presented to an arbitrary user as voices uttered while a plurality of users are viewing the same video in advance. For example, as the voice uttered by the user, “great”, “good”, “yahoo”, and the like can be considered.

[First Embodiment]
In the present embodiment, it is determined whether or not the accumulated voice is a non-language / language, and if the voice is a language, the voice to be reproduced is determined based on a predetermined evaluation value. is there.

  FIG. 6 shows the configuration of the video reproduction apparatus according to the first embodiment of the present invention.

  The video playback apparatus shown in the figure includes the video file 101, viewer audio file A102, audio segment file A103, viewer audio file B104, audio segment file B105, viewer audio file C106, audio segment file C107, described above. In this configuration, the user evaluation information file 108 and the monitor 200 are connected to a computer 100 used as a video playback device.

  The video playback device 100 includes a playback unit 110 and an audio playback control unit 120.

  The monitor 200 has a built-in speaker, and audio is output from the playback unit 110.

  Hereinafter, the operation of the video reproduction apparatus 100 will be described.

  FIG. 7 is a flowchart of the operation in the first embodiment of the present invention.

  Step 101) The playback unit 110 reads the video file 101, starts playback, and starts sending video and audio to the monitor 200.

  Step 102) The audio reproduction control unit 120 determines whether or not audio is present in any of the viewer audio file A102, the viewer audio file B104, and the viewer audio file C106 at present, Judgment is made with reference to B105 and the voice section file C107. As a determination method, if the start time and end time fields of each audio section file are other than NULL, it is determined that there is audio. If there is sound, the process proceeds to step 103, and if not, the step is repeated.

  Step 103) If it is determined in step 102 that the sound is present, it is determined whether or not the sound is a non-language sound for the viewer sound file in which the sound is present. Here, the non-verbal speech refers to speech such as “e” or “a”. For example, “Hiroshi Kanazawa, Chris Maeda, Yoichi Takebayashi,“ Recognition and synthesis of non-verbal speech for interaction with a computer ”can be used as a process for determining whether or not a speech signal is non-verbal speech. Journal, D-II, Vol.J77-D-II No.8, pp.1512-1421, 1994 ”can be used. Further, a method of requesting the center to make a non-language / language determination is also conceivable.

  Here, an example is given in which the determination is performed during the video playback process, but before the video playback process, it is determined whether each audio file (102, 104, 106) is non-verbal audio and an audio section file ( 103, 105, 107) as additional information, and in this step, it may be determined whether or not it is non-verbal speech by referring to the additional information in the speech segment file. If it is a non-language, the process proceeds to step 105. If it is not a non-language, the process proceeds to step 104. This is because non-linguistic voices are all non-linguistic voices such as a, b, and c in FIG. 8 in consideration of the fact that, when overlapped with the voices of a language, the influence of hindering the understanding of the language is small. It is for reproduction. For example, if you hear “Thank you” and “Ojamamasu” at the same time, they will be difficult to hear, but “Ah” and “Ojamamasu” will be played at the same time. Even so, listening to the voice of “Ojamasashi” is not much disturbed.

  It is also possible to perform processing for determining non-voice on the center side. In this case, it is not necessary to have a viewer audio file and a video file in the apparatus.

  Step 104) The voice reproduction control unit 120 identifies the voice utterance section with respect to the voice recognized in Step 102 with reference to the voice section files 103, 105, and 107, and within the section from the user. It is determined with reference to the user evaluation information file 108 and the voice section files 103, 105, and 107 whether there is a voice of a user with a high evaluation. If there is sound, the process proceeds to step 102, and if there is no sound, the process proceeds to step 105. For example, in FIG. 8, since there is no user's voice with a higher evaluation than user A in the voice section with respect to the voice d of user A, the process moves to step 105 and the voice is reproduced. The On the other hand, in FIG. 8, since there is a voice of user A having a higher evaluation than the user B in the voice section with respect to the voice e of user B, the process proceeds to step 102 and the voice is not reproduced. .

  This is because the voice of a user with a high evaluation and the voice of a user with a low evaluation overlap with the voice of a user with a high evaluation in consideration of the necessity of listening to the voice of the user with a high evaluation. In this case, only the voice of a highly evaluated user is reproduced, thereby eliminating the influence of hindering the voice listening due to the voice overlap. For example, in the portion (d, e) where the utterance of user A and the utterance of user B overlap in FIG. 8, the utterance d of user A is reproduced, but the utterance e of user B is not reproduced.

  Step 105) The reproduction unit 110 reproduces the sound determined to exist at step 102.

  In this way, by reproducing the video and the user's voice in synchronization, it is possible to enjoy the video while listening to the voices of other users in each scene of the video.

  Step 106) It is determined whether or not the end of the video has been reproduced. If it is the end of the video, the process proceeds to Step 107. If it is not the end of the video, the process proceeds to step 102.

  Step 107) The reproduction unit 110 stops the reproduction of the video and audio.

  As described above, in this embodiment, it is possible to listen to the voice of a user who is highly evaluated among a plurality of voices without omission.

[Second Embodiment]
In this embodiment, an example will be described in which the playback volume is controlled based on a predetermined evaluation value.

  The configuration of the video playback apparatus 100 in the present embodiment is the same as the configuration of FIG. 6 in the first embodiment described above, but the operations of the audio playback control unit 120 and the playback unit 110 are different.

  In the present embodiment, as in the first embodiment described above, instead of determining whether or not there is a utterance section of a person with a higher evaluation than that in the utterance section, the user evaluation information file 108 is stored. The playback volume is controlled in accordance with a predetermined user evaluation value.

  FIG. 9 is a flowchart of the operation in the second embodiment of the present invention. In the figure, the same operations as those in FIG. 7 in the first embodiment are denoted by the same step numbers as those in FIG. 7, and the description thereof is omitted.

  Step 204) The sound reproduction control unit 120 refers to the user evaluation information file 108 for the sound determined to be present in Step 102, and determines the sound volume at the time of reproduction according to the magnitude of the evaluation. For example, in the example of FIG. 10, the user A, the user B, and the user C are increased in this order. In FIG. 10, the volume level is schematically shown by the length of the vertical side of the rectangle.

  When determining the volume, for example, it is determined by multiplying the original speech waveform by the c value calculated by the following equation. Here, ε is an evaluation value, and takes a value from 0 to 100 as shown in FIG. 5. The larger the value, the higher the evaluation.

c = ε / 100
This is because, by considering that the voice of a high-rated user is higher in necessity than the voice of a low-rated user, the volume of the high-rated user's voice is reproduced relatively large. This is to reduce the influence of the hindrance to the voice listening due to the overlapping of voices. For example, in the portion where user A's utterance f and user B's utterance g overlap in FIG. 10, user A's utterance is reproduced at a louder volume than user B's utterance. Listening to is not hindered.

  As described above, in the present embodiment, it is possible to hear a user's voice with a high evaluation among a plurality of voices with a loud sound and a user's voice with a low evaluation with a small sound.

[Third Embodiment]
In the present embodiment, the video is reproduced a plurality of times for each user group whose voice sections do not overlap.

  The configuration of the video playback apparatus 100 in the present embodiment is the same as the configuration of FIG. 6 in the first embodiment described above, but the operations of the audio playback control unit 120 and the playback unit 110 are different.

  FIG. 11 is a flowchart of the operation in the third embodiment of the present invention. In the figure, the same operations as those in the flowchart of FIG. 7 of the first embodiment are denoted by the same reference numerals, and the description thereof is omitted.

  Step 502) The voice reproduction control unit 120 searches the voice section files 103, 105, and 107 for a group of users who have utterance patterns that are not non-language and have no duplication. This is because even if the voices of a plurality of users whose voice sections do not overlap are reproduced at the same time, the voices are not overlapped, so that it is possible to prevent the user's voice from being disturbed. For example, in the case of the group consisting of the user A and the user C shown in FIG. 12h, there is no duplication other than non-language, and even when played at the same time, there is no hindrance to listening (duplication between the user A and the user C). Part is a non-language part and will not be disturbed).

  Step 503) The reproduction unit 110 reproduces the voice of the user (user A, user C) in the user group obtained in step 502.

  Step 504) The voice reproduction control unit 120 searches for a group of users having an utterance pattern without duplication other than the user's non-language other than the group of users obtained in Step 502 in the same manner as in Step 502. For example, in the case of FIG. 12, the user B who is the remaining user is searched, and only the user B is searched as a group of users having an utterance pattern without duplication. This is done in step 503 to hear the voice of the user whose voice was not played (not grouped in group h).

  Step 505) The reproduction unit 110 reproduces the video and the audio of the user by going back to the time of the first audio among the users included in the user group obtained in Step 504.

  Step 506) It is determined whether or not the voices of all users have been reproduced in Step 506. If all the user's voices have been reproduced, the process proceeds to step 106, and if not all the user's voices have been reproduced, the process proceeds to step 504.

  This is because, as shown in FIG. 12, in order to reproduce the voices of all the users without overlapping the utterance sections, the video is reproduced a plurality of times in time, and the user's voice is reproduced in synchronization therewith.

  Thus, in this Embodiment, all the voices of all the users can be heard without omission without duplication of voices.

[Fourth Embodiment]
In the present embodiment, the voice of a user with a relatively low evaluation is reproduced so that the utterance sections do not overlap by shifting the synchronization with the video.

  The configuration of the video playback device 100 in the present embodiment is the same as the configuration of FIG. 6 in the first embodiment described above, but the operations of the playback unit 110 and the audio playback control unit 120 are different.

  FIG. 13 is a flowchart of the operation in the fourth embodiment of the present invention. In the figure, the same step numbers are assigned to the same operations as those in FIGS. 9 and 11, and the description thereof is omitted.

  Step 702) The audio reproduction control unit 120 identifies the user who has the highest evaluation among the users who have not yet reproduced the audio with reference to the user evaluation information file 108, and the reproduction unit 110 determines the first user of the user. Play audio.

  The process for determining whether or not the sound has been reproduced can be realized by assigning a flag for each sound section of the sound section file and determining whether the flag is on or off before the process. Further, any method other than this may be used as long as it can be determined whether or not the reproduction has been completed.

  Step 703) The voice reproduction control unit 120 calculates a time interval until the next utterance of the user who reproduced the voice in Step 702 with reference to the voice section files 103, 105, and 107, and other users who fall within the time interval The utterance is retrieved and reproduced by the reproduction unit 110. Here, when there are a plurality of utterances that fit within the time, the utterance closest to the time when the reproduction with respect to the start time of the utterance ends is adopted. For example, as shown in FIG. 14, the utterance n of user B that falls within the interval T of the first utterance j of user A is reproduced. Detailed processing will be described later with reference to FIG.

  This is because the voice of a user with a high evaluation is synchronized with the video, considering that the voice of a user with a high evaluation is more likely to be played back in synchronization with the video than the voice of a user with a low evaluation. By playing back relatively low-rated user's audio out of sync with the video, the video that hinders voice listening due to the overlap of audio is reduced and the total playback time is shortened. Because.

  Step 704) The playback unit 110 plays back the voice of the next user played back in Step 702.

  Step 705) It is determined whether or not all of the user's voice reproduced in Step 702 has been reproduced, and if all have been reproduced, the process proceeds to Step 506, and if not all, the process proceeds to Step 703.

  Next, the processing in step 703 will be described in detail.

  FIG. 15 is a flowchart of the operation of reproducing the voice of a user with a low evaluation value according to the fourth embodiment of the present invention. The following description will be made using the example of FIG.

  Step 7031) The end time of the current utterance j is subtracted from the start time of the next utterance k of the user A to obtain a free time T.

  Step 7032) It is determined whether or not there is another user's utterance within the vacant time T. If there is not, the process proceeds to Step 704. If there is, the process proceeds to Step 7033.

  Step 7033) The utterance n of another user (user B) having the start time closest to the end time of the utterance j of user A who has just uttered is picked up and reproduced, and the process proceeds to step 7031.

  According to the present embodiment, it is possible to hear all voices of all users without omissions and to hear all voices in a shorter time compared to the third embodiment. .

[Fifth Embodiment]
In the present embodiment, in the determination of whether or not there is an utterance section of a person with a higher evaluation than that in the utterance section in the first embodiment described above (step 104), an overlap of a certain time t is allowed. Is.

  The configuration of the video playback device 100 in the present embodiment is the same as the configuration of FIG. 6 in the first embodiment described above, but the operation of the audio playback control unit 120 is different.

  FIG. 16 is a flowchart of the operation in the fifth embodiment of the present invention. In the figure, the same operation as the operation of FIG. 9 in the first embodiment is given the same step number, and the description thereof is omitted.

  Step 904) The voice reproduction control unit 120 determines whether there is a utterance section of a person with a higher evaluation than itself in the utterance section excluding the margin T seconds with reference to the user evaluation information file 108. If there is the utterance section, the process proceeds to step 102. If there is no utterance section, the process proceeds to step 105. The value of T is, for example, 1 second as a default value. Here, in the case of FIG. 17, since the overlap time q of the utterance section of the utterance o of the user A and the utterance section of the utterance p of the user B is shorter than the margin T, these utterances are reproduced simultaneously.

  This takes into account the human perceptual characteristic that if the overlap between the end of a certain utterance and the beginning of another utterance is short, it will not hinder listening.

  In the present embodiment, since the sound can be played back densely in time, more sounds can be played back.

  As described above, according to the present embodiment, it is possible to hear the voice of a user who is highly evaluated among a plurality of voices without omission, and it is possible to hear more voices compared to the first embodiment.

[Sixth Embodiment]
In the present embodiment, when the user's utterance section overlaps, the voice is fast-forwarded to shorten the reproduction time, thereby eliminating the hindrance to hearing due to the overlap of the voice.

  The configuration of the video playback apparatus 100 in the present embodiment is the same except that the user evaluation information file 108 included in the configuration of the configuration of FIG. 6 in the first embodiment is not used, but the audio playback control unit 120 operations are different.

  FIG. 18 is a flowchart of the operation in the sixth embodiment of the present invention. In the figure, the same operations as those in the flowchart of FIG. 7 in the first embodiment are denoted by the same step numbers, and the description thereof is omitted.

  Step 1104) The reproduction control unit 120 determines the time difference between the start time of the sound determined to be present in Step 102 and the next start time in the utterances of all users as the sound section files 103, 105, Calculate with reference to 107. It is determined whether or not the obtained time difference is shorter than the utterance interval length recognized in step 102. If it is shorter, the process proceeds to step 1105. If it is not shorter, the process proceeds to step 105.

  Step 1105) The playback unit 110 plays back the utterance determined to be present in step 102, with the time difference calculated in step 1104 reduced as shown in FIG. For example, in the example of FIG. 10, since the time difference u from the start time of the utterance r of the user A is shorter than the section length v of the utterance of the utterance r of the user A, the utterance 1201 of the user A is shortened to the time difference u. To play.

  For example, a software sampler can be used as a method for reducing the playback time.

  As shown in FIG. 19, when the utterance section overlaps with another user's utterance, the utterance time is shortened and reproduced to prevent the hearing from being disturbed, and the video is traced back in time. Listen to the voices of all users without playing them.

  As described above, according to the present embodiment, it is possible to listen to all the sounds of all users without any overlap without reproducing the video retroactively.

  In the above-described first to sixth embodiments, the operations shown in FIGS. 7, 9, 11, 13, 15, 15, and 18 are constructed as a program, and the computer used as a video reproduction apparatus is used. It can be installed and executed, or distributed via a network.

  The present invention is not limited to the above-described embodiment, and various modifications and applications can be made within the scope of the claims.

  The present invention is applicable to a technique for reproducing the audio of a video viewer together with video.

It is a principle explanatory view of the present invention. It is a principle block diagram of this invention. 1 is a system configuration diagram according to an embodiment of the present invention. It is an example of the audio | voice area file in one embodiment of this invention. It is an example of the user evaluation information file in one embodiment of the present invention. 1 is a configuration diagram of a video reproduction device according to a first embodiment of the present invention. It is a flowchart of the operation | movement in the 1st Embodiment of this invention. It is a figure which shows the user's utterance and evaluation with respect to utterance in the 1st Embodiment of this invention. It is a flowchart of the operation | movement in the 2nd Embodiment of this invention. It is a figure which shows the relationship between the user's utterance and the reproduction | regeneration sound volume by evaluation value in the 2nd Embodiment of this invention. It is a flowchart of the operation | movement in the 3rd Embodiment of this invention. It is a figure which shows the relationship between a user's utterance and reproduction | regeneration order in the 3rd Embodiment of this invention. It is a flowchart of the operation | movement in the 4th Embodiment of this invention. It is a figure which shows the relationship between utterance and reproduction | regeneration in the 4th Embodiment of this invention. It is a flowchart of the operation | movement which reproduces | regenerates the user's audio | voice with a low evaluation value in the 4th Embodiment of this invention. It is a flowchart of the operation | movement in the 5th Embodiment of this invention. It is a figure which shows the relationship between utterance and reproduction | regeneration in the 5th Embodiment of this invention. It is a flowchart of the operation | movement in the 6th Embodiment of this invention. It is a figure which shows the relationship between utterance and reproduction | regeneration in the 6th Embodiment of this invention.

Explanation of symbols

10 viewer audio file 20 audio segment file 100 computer, video playback device 101 video file 102 viewer audio file A
103 Voice section information file A
104 Viewer audio file B
105 Voice section information file B
106 Viewer audio file C
107 Voice section information file C
108 user evaluation information file 110 playback means, playback section 120 playback control means, playback control section 200 monitor

Claims (13)

In a video playback method for playing back audio groups of a plurality of users viewing the same video together with the video,
A playback start step for reading video and starting playback;
Based on whether or not the start time and the end time are set with reference to an audio section information file that is provided for each user and stores an audio section consisting of a start time and an end time of utterance for the video. An audio presence determination step for determining whether audio is stored in a viewer audio file provided for each user;
A non-language determination step of determining whether or not the sound is non-language when sound is stored in the viewer sound file, and reproducing the sound if non-language;
In the non-language determination step, when it is determined that the language is not non-language, an evaluation value predetermined for each user is obtained for the user's voice corresponding to the voice section stored in the voice section information file. Referring to the stored user evaluation information file based on the user ID, and reproducing the user's voice having the highest evaluation value;
Repeating the audio presence determination step, the non-language determination step, and the audio reproduction step until the video of the video file is completed for the next audio section that is audio-reproduced in the audio reproduction step;
A video playback method characterized in that In a video playback method for playing back audio groups of a plurality of users viewing the same video together with the video,
A playback start step for reading video and starting playback;
Based on whether or not the start time and the end time are set with reference to an audio section information file that is provided for each user and stores an audio section consisting of a start time and an end time of utterance for the video. An audio presence determination step for determining whether audio is stored in a viewer audio file provided for each user;
A non-language determination step of determining whether or not the sound is non-language when sound is stored in the viewer sound file, and reproducing the sound if non-language;
In the non-language determination step, when it is determined that the language is not non-language, an evaluation value predetermined for each user is obtained for the user's voice corresponding to the voice section stored in the voice section information file. Referring to the stored user evaluation information file based on the user ID, determining a volume to be played based on the evaluation value, and playing back the sound;
Repeating the audio presence determination step, the non-language determination step, and the audio reproduction step until the video of the video file is completed for the next audio section that is audio-reproduced in the audio reproduction step;
A video playback method characterized in that In a video playback method for playing back audio groups of a plurality of users viewing the same video together with the video,
A playback start step for reading video and starting playback;
Based on whether or not the start time and the end time are set with reference to an audio section information file that is provided for each user and stores an audio section consisting of a start time and an end time of utterance for the video. An audio presence determination step for determining whether audio is stored in a viewer audio file provided for each user;
When audio is stored in the viewer audio file, it is determined whether or not the audio is non-language, and a search is made for all the audio section information files for a group of users whose non-speech sounds other than non-language. A first user group search step acquired by:
Based on the group of users acquired in the first user group search step, a first audio reproduction step of reproducing audio by reading out audio of all users belonging to the group from the viewer audio file;
A second user group search step of searching for a group of users whose utterances are not duplicated among the users not selected in the first user group search step by searching the voice section information file of the user;
A second audio reproduction step of reading out audio of all users in the group searched in the second user group search step from the viewer audio file and reproducing the audio retroactively together with the video;
Repeating the second user group search step and the second voice playback step until the playback of the voices of all users is completed;
A video playback method characterized in that In a video playback method for playing back audio groups of a plurality of users viewing the same video together with the video,
A playback start step for reading video and starting playback;
Among the viewer audio files provided for each user, with respect to unreproduced user audio, the evaluation value is maximized by referring to a user evaluation information file in which an evaluation value predetermined for each user is stored. A first audio reproduction step of reading out and reproducing the user's audio from the user's viewer audio file;
All audio segments that store the audio segment consisting of the start time and the end time of the utterance for the video provided for each user for the other user's speech that fits in the time interval until the next utterance of the user reproduced last time An utterance candidate search step acquired by searching an information file;
A second audio reproduction step of reading out and reproducing the user's voice having the next highest evaluation value from the user's voice reproduced in the utterance candidate search step from the user's viewer voice file;
Repeating the utterance candidate search step and the second audio reproduction step until all the user's audio is reproduced; and
A video playback method characterized in that In the audio reproduction step,
When reading and reproducing the user's voice having the highest evaluation value in the voice section from the viewer voice file,
2. The video playback method according to claim 1, wherein duplication of playback of an audio section having a predetermined time margin T is allowed. In a video playback method for playing back audio groups of a plurality of users viewing the same video together with the video,
A playback start step for reading video and starting playback;
Based on whether or not the start time and the end time are set with reference to an audio section information file that is provided for each user and stores an audio section consisting of a start time and an end time of utterance for the video. An audio presence determination step for determining whether audio is stored in a viewer audio file provided for each user;
A non-language determination step of determining whether or not the sound is non-language when sound is stored in the viewer sound file, and reproducing the sound if non-language;
In the non-language determination step, for the first sound determined not to be non-language, the start time of the first sound and the second sound of another user uttered next to the first sound If the difference from the start time is shorter than the section length of the first sound, a sound playback step for shortening and playing back the first sound;
Repeating the voice presence determination step, the non-language determination step, and the voice reproduction step for all the voice segments stored in the voice segment information file;
A video playback method characterized in that A video playback device that plays back a group of users who are watching the same video together with the video,
A video file containing the video,
A plurality of audio section information files that are provided for each user and store an audio section including a start time and an end time of utterance for the video;
A plurality of viewer audio files storing the user's audio provided for each user;
A user evaluation information file storing evaluation values predetermined for each user;
Playback means for playing back video and audio;
Playback control means for controlling the playback of audio,
The reproducing means includes
Video playback start means for reading video from the video file and starting playback;
Based on an instruction from the reproduction control means, and a means for reproducing audio in synchronization with the video,
The reproduction control means includes
Sound presence determination means for referring to the sound section information file and determining whether sound is stored in the viewer sound file based on whether the start time and the end time are set;
Non-language determination means for determining whether or not the sound is non-language when sound is stored in the viewer sound file;
If the non-language determining means determines that the language is not non-language, the user evaluation information file is determined based on the user ID for the user's voice corresponding to the voice section stored in the voice section information file. A sound reproduction control unit that refers to and reproduces the voice of the user having the highest evaluation value on the reproduction unit and causes the reproduction unit to reproduce the non-language voice when the non-language determination unit determines that the voice is non-language. When,
Repeating means for repeating the sound presence determination means, the non-language determination means, and the sound reproduction control means until the end of the video of the video file, for the next voice section played back by the playback means,
A video playback apparatus comprising: A video playback for playing back a group of voices of a plurality of users who are viewing the same video together with the video,
A video file containing the video,
A plurality of audio section information files that are provided for each user and store an audio section including a start time and an end time of utterance for the video;
A plurality of viewer audio files storing the user's audio provided for each user;
A user evaluation information file storing evaluation values predetermined for each user;
Playback means for playing back video and audio;
Playback control means for controlling the playback of audio,
The reproducing means includes
Video playback start means for reading video from the video file and starting playback;
Based on an instruction from the reproduction control means, and a means for reproducing audio in synchronization with the video,
The reproduction control means includes
Sound presence determination means for referring to the sound section information file and determining whether sound is stored in the viewer sound file based on whether the start time and the end time are set;
Non-language determination means for determining whether or not the sound is non-language when sound is stored in the viewer sound file;
When the non-language determining means determines that the language is not non-language, the user evaluation information file is stored in the user evaluation information file for the user's voice corresponding to the voice section stored in the voice section information file. Is determined based on the evaluation value, the sound volume to be played back by the playback means is determined based on the evaluation value, the sound for each user is played back by the playback means at the volume, and the non-language determination means determines that the sound is non-language. Includes a sound reproduction control means for causing the reproduction means to reproduce non-language sound;
Repeating means for repeating the sound presence determination means, the non-language determination means, the volume control means, and the sound reproduction control means until the video of the video file ends for the next voice section that has been played back,
A video playback apparatus comprising: A video playback device that plays back a group of users who are watching the same video together with the video,
A video file containing the video,
A plurality of audio section information files that are provided for each user and store an audio section including a start time and an end time of utterance for the video;
A plurality of viewer audio files storing the user's audio provided for each user;
Playback means for playing back video and audio;
Playback control means for controlling the playback of audio,
The reproducing means includes
Video playback start means for reading video from the video file and starting playback;
Based on an instruction from the reproduction control means, and a means for reproducing audio in synchronization with the video,
The reproduction control means includes
Sound presence determination means for referring to the sound section information file and determining whether sound is stored in the viewer sound file based on whether the start time and the end time are set;
Non-language determination means for determining whether or not the sound is non-language when sound is stored in the viewer sound file;
First user group search means for acquiring a group of users whose utterances other than non-languages are not duplicated by searching all the speech section information files;
Based on the group of users acquired by the first user group search means, the first voice for reading the voices of all users belonging to the group from the viewer voice file and causing the playback means to play back the voices Playback control means;
A second user group search means for searching for a group of users whose utterances are not duplicated among the users not selected in the first user group search means by searching the voice section information file;
A second audio reproduction control unit that reads out audio of all users belonging to the group searched by the second user group search unit from the viewer audio file and causes the reproduction unit to reproduce the audio retroactively;
Repeating means for repeating the second user group search means and the second sound reproduction control means until the reproduction of all the user's sounds is completed;
A video playback apparatus comprising: A video playback device that plays back a group of users who are watching the same video together with the video,
A video file containing the video,
A plurality of audio section information files that are provided for each user and store an audio section including a start time and an end time of utterance for the video;
A plurality of viewer audio files storing the user's audio provided for each user;
A user evaluation information file storing evaluation values predetermined for each user;
Playback means for playing back video and audio;
Audio reproduction control means for controlling audio reproduction,
The reproducing means includes
Video playback start means for reading video from the video file and starting playback;
Based on an instruction from the reproduction control means, and a means for reproducing audio in synchronization with the video,
The sound reproduction control means includes
Among the viewer audio files, the user's voice that has not been played is referred to the user evaluation information file, and the user's voice with the highest evaluation value is read from the viewer voice file and played back by the playback means. 1 audio reproduction control means;
Utterance candidate search means for acquiring other user's voice that fits in the time interval until the next utterance of the user reproduced last time by searching all the voice section information files;
Of the user's voice acquired by the utterance candidate search means, referring to the user evaluation information file, the voice of the user having the second highest evaluation value after the previous played user is read out from the user's viewer voice file and played. Second sound reproduction control means for reproducing the means;
Repeating means for repeating the utterance candidate search means and the second sound reproduction control means until all user sounds are reproduced;
A video playback apparatus comprising: The sound reproduction control means includes
When the user's voice having the highest evaluation value in the voice section is read from the viewer voice file and played back by the playback means,
8. The video reproduction apparatus according to claim 7, further comprising means for allowing reproduction of audio sections having a predetermined time margin T. A video playback device that plays back a group of users who are watching the same video together with the video,
A video file containing the video,
A plurality of audio section information files that are provided for each user and store an audio section including a start time and an end time of utterance for the video;
A plurality of viewer audio files storing the user's audio provided for each user;
Playback means for playing back video and audio;
Audio reproduction control means for controlling audio reproduction,
The reproducing means includes
Video playback start means for reading video from the video file and starting playback;
Based on an instruction from the reproduction control means, and a means for reproducing audio in synchronization with the video,
The sound reproduction control means includes
Sound presence determination means for referring to the sound section information file and determining whether sound is stored in the viewer sound file based on whether the start time and the end time are set;
Non-language determination means for determining whether or not the sound is non-language when sound is stored in the viewer sound file;
The non-language determination means causes the reproduction means to reproduce the sound determined to be non-language, and for the first sound determined not to be non-language, the first sound start time and the first sound If the difference from the start time of the second voice of another user uttered next to the voice of the first voice is shorter than the section length of the first voice, the first voice is shortened and the playback means Voice playback control means for playing back,
Repeating means for repeating the voice presence determination means, the non-language determination means, and the voice reproduction control means for all the voice sections stored in the voice section information file;
A video playback apparatus comprising: A video playback program that plays back a group of voices of a plurality of users who are viewing the same video together with the video,
7. A video playback program for causing a computer to execute processing for realizing the video playback method according to claim 1.

Images