JP4594908B2 - Explanation additional voice generation device and explanation additional voice generation program - Google Patents

Description

  The present invention relates to a commentary-added sound generation apparatus and a commentary-added sound generation program for adding commentary sound indicating the contents of a video to a video.

  Conventionally, in the production of commentary broadcast programs on television broadcasts, after completing the program in the normal procedure, a commentary manuscript about the scene description and caption content for visually impaired persons is written separately from the script and screenplay of the program. Created by experts. Then, in the studio, under the direction of the director for the commentary broadcast program, the next utterance is made from this utterance section so that it does not overlap the utterance section (speaking section) containing speech sounds such as dialogue or narration in the video and audio. A commentary broadcast program was produced by the narrator reading the commentary at an exquisite timing and adding commentary audio to the silent or background-only segment (pause segment) up to the segment.

  In this explanation broadcast program, using the “main audio” and “sub audio” modes of the TV audio, the video and audio are added to the “main audio” channel, and the audio with the explanation audio added to the video and audio is set to the “sub audio” channel. It is flowing. In analog broadcasting, both main audio and sub audio are monaural audio, but in digital broadcasting, both main audio and sub audio can be stereo audio in both terrestrial broadcasting and satellite broadcasting.

There is also a commentary broadcast that uses a wireless earphone receiver to provide audiences with the background of performing arts and theatrical performances and supplementation of difficult lines in stage performances such as classical performing arts and foreign language theaters. A technique for supporting the explanation broadcasting work is disclosed (see Patent Document 1 and Patent Document 2). In this technology, commentary audio is recorded in advance, and an identification number is assigned to each commentary unit. Based on this identification number, the playback order and playback timing are set in advance or designated by the operator. This will be a commentary broadcast as the project progresses.
JP 2002-26830 A (paragraph numbers 0011 to 0042) JP 2002-26840 A (paragraph numbers 0010 to 0033)

  However, in the production of commentary broadcast programs in conventional television broadcasts, it is effective for the commentary voice manuscript creator (scenario writer) to insert it within a limited time in the pause section, and does not break the atmosphere of the program. Knowledge and proficiency such as selection of terms were necessary. In addition, in order to record commentary audio, not only a specialized narrator was required, but also a studio for recording had to be secured. Furthermore, when recording commentary speech, it was necessary to adjust the voice start timing and voice rate, which required a lot of time and money including rehearsals. Therefore, the penetration rate of commentary broadcast programs is not high, and according to the press release of the Ministry of Internal Affairs and Communications in August 2005, the proportion of commentary broadcasts in the total broadcast time in 2004 is 3.2 for NHK (General). 7.9% for NHK (education) and 0.5% for 5 commercial broadcasting stations.

  In addition, the technique described in Patent Document 1 is specialized for work related to stage performances and the theater environment, and it is necessary to add commentary audio so that it does not overlap the beaten section in a pause section with a limited time. It cannot be applied to commentary broadcasting such as TV broadcasting.

  The present invention has been made to solve the above-described problems of the prior art, and provides a commentary-added sound generation apparatus and a commentary-added sound generation program that can produce sound of a commentary broadcast program in a short time and at a low cost. The purpose is to do.

  In order to solve the above-described problem, the commentary-added audio generation device according to claim 1 inputs video audio, which is audio of video, and text data related to the content of the video from the outside, A comment-added speech generation apparatus that generates comment-added speech to which commentary speech obtained by converting the text data into speech is added, and includes speech synthesis means, section detection means, speech speed conversion means, and speech addition means. The configuration.

According to such a configuration, the commentary-added sound generation device generates a commentary sound by synthesizing the text data from the voice data by the voice synthesizing unit, and on the time axis of the reproduction time of the video / audio from the video / sound by the section detection unit. , The utterance sound section that is the voice section of the utterance sound and the pause section that is the voice section of the silence or the background sound only are detected. The commentary-added sound generation device converts the commentary speech to speech speed based on the section length of the pause section by the speech speed conversion means, and the speech addition means applies the detected pause section to the detected pause section. The commentary speech converted by the speech rate conversion means is added to generate the commentary-added speech.

  As a result, the commentary-added sound generation apparatus can generate commentary sound from the text data and add it to the pause section of the video and sound to generate commentary-added sound.

Further, the commentary-added audio generation apparatus according to claim 2 inputs video audio, which is audio of video, and text data related to the content of the video from the outside, and converts the text data to the video audio. A comment-added speech generation apparatus that generates comment-added speech to which comment speech converted into a voice is added, and is configured to include speech synthesis means, section detection means, speech speed conversion means, and speech addition means .
According to such a configuration, the commentary-added sound generation device generates commentary sound by speech synthesis from the text data by the voice synthesis unit, and from the video and audio by the section detection unit on the time axis of the playback time of the video / sound. , A utterance sound section that is a voice section of a uttered sound, and a pause section that is a voice section of silence or background sound only. Then, the speech speed is converted by the speech speed conversion means, and the commentary sound is converted based on the length of the speech sound section expanded and contracted by the speech speed conversion and the length of the pause section. The speech speed is converted by the speech speed conversion means, and the speech speed conversion means converts the pause period, which is a voice section of silence or background sound only, from the audio / video converted by the speech speed conversion means. Add commentary audio.

  As a result, the comment-added sound generation device can generate comment-added sound by adding the comment sound to the image sound by converting the speech speed of both the image sound and the comment sound in the utterance sound section.

  According to a third aspect of the present invention, there is provided the commentary-added speech generation apparatus according to the second aspect, wherein the speech speed in the speech sound section of the video and audio converted by the speech speed conversion means is the speech speed conversion unit. On the basis of information indicating expansion / contraction due to conversion, video speed conversion means for expanding / contracting the section length of the section corresponding to the uttered sound section of the video input from the outside is provided.

  As a result, the commentary-added audio generation device generates a video synchronized with the commentary-added audio together with the commentary-added voice by expanding / contracting the video corresponding to the video-sound according to the expansion / contraction of the video / sound by the speech speed conversion. Can do.

  The comment added audio generation program according to claim 4 inputs video and audio, which is video audio, and text data related to the content of the video from outside, and converts the text data into audio and video. The computer is caused to function as speech synthesis means, section detection means, speech speed conversion means, and speech addition means in order to generate commentary-added voice to which the commentary voice converted into is added.

According to such a configuration, the commentary additional sound generation program generates commentary sound by voice synthesis from the text data by the voice synthesizer, and on the time axis of the reproduction time of the video and sound from the video sound by the section detection unit. , A utterance sound section that is a voice section of a uttered sound, and a pause section that is a voice section of silence or background sound only. The commentary-added sound generation program converts the commentary speech based on the section length of the pause section by the speech speed conversion means, and for the pause section detected in the video and sound by the sound addition means, The commentary speech converted by the speech rate conversion means is added to generate commentary added speech.

As a result, the commentary-added sound generation program can generate commentary sound from the text data and add it to the pause section of the video and sound to generate commentary-added sound.
The comment added audio generation program according to claim 5 inputs video and audio as video audio and text data related to the content of the video from outside, and outputs the text data to the video and audio. The computer is caused to function as speech synthesis means, section detection means, speech speed conversion means, and speech addition means in order to generate commentary-added voice to which the commentary voice converted into is added.
According to such a configuration, the commentary additional sound generation program generates commentary sound by voice synthesis from the text data by the voice synthesizer, and on the time axis of the reproduction time of the video and sound from the video sound by the section detection unit. , A utterance sound section that is a voice section of a uttered sound, and a pause section that is a voice section of silence or background sound only. Then, the speech speed is converted by the speech speed conversion means, and the commentary sound is converted based on the length of the speech sound section expanded and contracted by the speech speed conversion and the length of the pause section. The speech speed is converted by the speech speed conversion means, and the speech speed conversion means converts the pause period, which is a voice section of silence or background sound only, from the audio / video converted by the speech speed conversion means. Add commentary audio.

  The commentary-added sound generation apparatus and commentary-added sound generation program according to the present invention have the following excellent effects. According to the first and fourth aspects of the present invention, since commentary speech is generated from text data by speech synthesis, specialized narrators, recording studios, recording rehearsals, and the like are not required, reducing costs and production time. Can be shortened.

  In addition, since the explanation voice is converted to the speech speed according to the length of the pause section and the time scale is adjusted, even the redundant text can be converted to the explanation voice according to the length of the pause section, which is conventionally used to create a manuscript It is possible to generate commentary-added speech using commentary speech generated by speech synthesis. In addition, it is easy to prepare manuscripts and it is easy to secure human resources for scenario writers, and it is possible to reduce the cost of production of commentary-added audio and shorten the production time. Can contribute to the spread of

According to the second and fifth aspects of the present invention, even if the length of the pause section is shorter than that of the commentary voice, the voice of the commentary voice can be spoken by converting the voice speed of the video sound to make the pause section longer. It is possible to prevent the speed from becoming extremely fast, and to adjust both the speed of the audio and video speech and the speed of the commentary audio. This makes it possible to generate commentary-added speech that is easy to hear.

  According to the third aspect of the present invention, since the video synchronized with the commentary additional sound can be generated, the commentary broadcast program that can be used not only for the visually impaired but also for the visually impaired and the sighted is generated. be able to.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[Configuration of commentary broadcast program generator]
First, with reference to FIG. 1, the structure of the explanation broadcast program production | generation apparatus 1 in this invention is demonstrated. FIG. 1 is a schematic diagram schematically showing a configuration of an explanation broadcast program generating apparatus according to the present invention.

  An explanation broadcast program generation device (explanation additional audio generation device) 1 inputs a video, video audio that is the audio of this video, and an explanation manuscript that is text data that describes the contents of each scene of this video. A sub-audio (commentary additional audio) in which a commentary voice obtained by synthesizing a commentary manuscript is added to a pause period, which is a period of only silence or background sound in the video and audio, and a sub-audio synchronized video that is a video corresponding to the sub-audio. Is generated. Here, the explanation broadcast program generating apparatus 1 includes an explanation manuscript input means 11, a video speed conversion means 12, a voice analysis means 13, a voice synthesis means 14, a speech speed conversion means 15, a voice connection means 16, A video / audio output unit 17, an original video output unit 18, a sub audio synchronized video output unit 19, a main audio output unit 20, and a sub audio output unit 21 are provided.

  Here, a program video storage device 3 and a comment broadcast program storage device 7 are externally connected to the comment broadcast program generation device 1. The program video storage apparatus 3 stores videos of a plurality of television programs and video / audio of these videos, and is configured by general storage means such as a hard disk. In this program video storage device 3, for example, video from a VTR (Video Tape Recorder) or an optical disk (not shown) in which video and audio of a television program produced by a normal method are recorded by input means (not shown). And video and audio are input. At this time, if the video and audio are mixed and compressed and encoded, the video and audio are separated and stored in the program video storage device 3. Here, the program video storage device 3 stores the video and video / audio of the TV program, but may store video and video / audio other than the TV program.

  The explanation broadcast program storage device 7 stores video (original video), video / audio (main audio), sub-audio, and sub-audio synchronized video, and is constituted by general storage means such as a hard disk. The Then, the explanation broadcast program generating device 1 inputs video and video / audio from the program video storage device 3, generates sub audio and sub audio synchronized video, and reads the video (original video) read from the program video storage device 3. In addition, it is stored in the explanation broadcast program storage device 7 together with the video and audio (main audio). For the original video, main audio, sub audio, and sub audio synchronized video stored in the explanation broadcast program storage device 7, a necessary combination is selected by the operator, for example, explanation broadcast to a VTR, an optical disc or the like (not shown). Recorded as a program.

  Furthermore, an output device 5 is connected to the commentary broadcast program generating device 1 externally. The output device 5 displays an image or video output from the commentary broadcast program generation device 1 and outputs sound, and includes, for example, a display such as a liquid crystal display panel and a speaker. . Hereinafter, the configuration of the explanation broadcast program generation device 1 will be described in detail.

  The comment manuscript input means 11 is for inputting a comment manuscript that is a manuscript audio manuscript to be added to the video and audio inputted from the program video storage device 3 from the outside. The commentary manuscript input here is output to the speech synthesizer 14. It is assumed that the text of the commentary manuscript input here is associated in advance with each pause section of the inserted video and audio by, for example, a time code. Here, for example, the operator who produces the commentary broadcast refers to the detection results of the audio / voice beat section (voiced sound section) and the pause section analyzed by the voice analysis means 13 described later via the output device 5. However, the explanation manuscript may be input via a keyboard or an automatic voice recognition device (not shown). Here, commentary speech uttered by an operator who is familiar with commentary speech may be input, and this speech may be used in place of the commentary speech generated by the speech synthesis means 14 described later.

  The video speed conversion means 12 expands / contracts the video input from the program video storage device 3 based on the expansion / contraction information of the video / audio and commentary voice input from the speech speed conversion means 15 described later. Here, the video speed conversion means 12 is input with information on the expansion and contraction of the length of the talk section and the pause section when the video speed and the commentary voice are converted by the speech speed conversion section 15. In accordance with the expansion / contraction of the section length, the image 12 expands / contracts the image by thinning or repeating the image of the section corresponding to the beat section and the pause section in units of frames. By expanding and contracting the video in this manner, the video speed converting unit 12 can generate a video (sub audio synchronized video) synchronized with the sub audio generated by the audio connecting unit 16. The sub-audio synchronized video generated here is output to the video / audio output means 17 and the sub-audio synchronized video output means 19. Note that the expansion / contraction information of the video / audio and the commentary audio is, for example, the start time and end time or time code of the speech section and the speech section converted to the speech section and pause section corresponding to the playback time of the video and video / audio. Information, and information indicating the expansion / contraction rate of the section length of this section.

  The audio analysis means (section detection means) 13 analyzes the video and audio input from the program video storage device 3, and includes a squealing section that is a section such as speech and narration, and a pause section that is a section of silence or background sound only. Is detected. The voice analysis means 13 can be realized by various voice analysis techniques as described in Japanese Patent No. 3160228, for example. Here, the audio analysis means 13 records the start time and end time of each detected section in association with the video / audio reproduction time of the video / audio. The section information and video / audio that have been analyzed here are output to the speech speed conversion means 15 and the video / audio output means 17.

  The voice synthesizing unit 14 synthesizes the commentary manuscript input from the commentary manuscript input unit 11 and generates a commentary voice. The commentary speech synthesized here is output to the speech speed conversion means 15 and the video / audio output means 17.

  Here, the worker can listen to the commentary sound output from the output device 5 via the video / audio output means 17 described later. Here, the speech synthesizer 14 has a reading and accent correcting function, and based on a command input by an operator via an input unit (not shown), the speech synthesizer 14 corrects a reading or accent error included in the commentary speech. It can be corrected. Furthermore, here, the speech synthesizer 14 can synthesize speech with the voice quality of a plurality of speakers including men and women based on a command input by an operator via an input unit (not shown). It also has the function of adjusting the height of the lens and enhancing or suppressing intonation. Then, the operator can select the voice quality appropriate for the video, set the voice pitch and inflection according to the video scene, and input the command to change the speaker in the middle of the video, thereby synthesizing the voice The means 14 can synthesize the commentary speech corresponding to those commands. The voice synthesizing means 14 can be realized by various voice synthesis techniques as described in, for example, Japanese Patent Application Laid-Open No. 2004-139033.

  The speaking speed conversion means 15 performs speaking speed conversion between the video and audio input from the sound analysis means 13 and the commentary sound input from the sound synthesis means 14. Here, the speech speed conversion is to extend / contract the voice time length while maintaining the voice pitch and quality. This speech speed conversion means 15 can be realized by various speech speed conversion techniques as described in, for example, Japanese Patent No. 2955247 and Japanese Patent No. 3220043. Here, the speech speed conversion means 15 performs the speech speed conversion for only the commentary sound or both the video sound and the commentary sound based on the instruction from the operator input from the input means (not shown). The audio / video and commentary audio converted at the speech speed are output to the audio connection means 16. Further, the information about the expansion / contraction of the video / audio and the commentary audio by the speech speed conversion of the speech speed conversion means 15 is output to the video speed conversion means 12 and the video / audio output means 17.

  Here, the speech speed conversion by the speech speed converting means 15 will be described using a specific example with reference to FIG. 2 (refer to FIG. 1 as appropriate). FIG. 2 is an explanatory diagram for explaining the speech speed conversion by the speech speed conversion means. FIG. 2A is a schematic diagram schematically showing examples of a talk section and a pause section detected by the speech analysis means. ) Is a schematic diagram schematically showing an example of a speech speed conversion pattern by the speech speed conversion means, and (c) is a schematic diagram schematically showing another pattern example of the speech speed conversion by the speech speed conversion means. FIG. 4D is a schematic diagram schematically showing another example of the speech speed conversion pattern by the speech speed conversion means. In FIG. 2, the broken lines indicate the start times and end times of the beat sections A1, A2, A3,... And the pause sections B1, B2,. The time at the center of B2, ... is schematically shown.

  Here, as shown in FIG. 2A, it is assumed that the speech analysis means 13 detects the beat sections A1, A2, A3,... And the pause sections B1, B2,. FIG. 2 (a) shows some of the detected sections (turning sections A1, A2, A3 and pause sections B1, B2). Then, the speech speed conversion means 15 is adapted to provide only the commentary audio or the video / audio so that the time scale of the video / audio corresponding to one whole video matches the sum of the converted video / audio, the commentary audio and the time scale. Talk speed conversion for both voice and commentary. Here, it is assumed that three conversion patterns from pattern 1 to pattern 3 shown in FIGS. 2B to 2D are set in advance in the speech speed conversion means 15.

  Here, an example of the pattern set in the speech speed converting means 15 will be described. In the pattern 1 shown in FIG. 2 (b), the speech speed conversion means 15 does not convert the speech speed of the audio / video C1, C2, C3,... In the talk sections A1, A2, A3,. The commentary voice input from the voice synthesizer 14 is converted to the speech speed in accordance with the section length of ... to obtain commentary voices D1, D2,. At this time, the speech speed is converted so that the commentary voices D1, D2,... The commentary voices D1, D2,... Can be inserted into the pause section while leaving the section, and sub-voices that are easy to hear for the viewer can be generated.

  In addition, there is a report that visually impaired persons can fully understand even a voice that is 2 to 3 times faster than a sighted person (speaking speed of about 800 to 1200 characters per minute in kana) (“visual impairments”). Optimum / Maximum Speed for Presenting Speech to the Person ”, Journal of Human Interface Society, Vol.7, No.1, pp.105-111, 2005), pause section B1, B2,. Even if D1, D2,... In this pattern 1, since the audio speed of the video / audio is not converted, the sub-audio generated by the audio connection means 16 described later is the same as the audio in the talk section, which is viewed by a low-sighted person or a sighted person. However, the sub-audio is not shifted from the video. Here, when the section length of the pause sections B1, B2,... Is longer than the commentary speech before the speech speed conversion, the speech speed conversion means 15 does not convert the commentary voice input from the speech synthesis means 14 to the speech speed.

  Further, in the pattern 2 shown in FIG. 2C, the speech speed converting means 15 does not change the start time of the video and audio in each speaking section A1, A2, A3,. The speech speed of the video and audio input from the voice analyzing means 13 and the commentary voice input from the voice synthesizing means 14 is set to the same speed, and the video and audio C1 ′, C2 ′, C3 ′,. Explanation voices D1 ′, D2 ′,... In this pattern 2, since the voice speeds of video audio C1 ′, C2 ′, C3 ′,... And commentary audio D1 ′, D2 ′,... Match, the sub audio generated by the audio connection means 16 described later is It becomes easier to hear than a combination of voices with different speaking speeds. In addition, since the start time of the program performers in the video coincides with the start times of the video and audio C1 ′, C2 ′, C3 ′,... The uncomfortable feeling due to the movement of the lips of the program performers in the video and the shift of the sub-audio is reduced.

  Further, in the pattern 3 shown in FIG. 2 (d), the speech speed conversion means 15 causes the center time of each of the talk sections A1, A2, A3,... And the pause sections B1, B2,. The speech speeds of the video audio and the commentary audio are set to the same level so as to coincide with the center time of the video audio C1 ″, C2 ″, C3 ″,... And the commentary audio D1 ″, D2 ″,. The voice speed is converted between the video and audio and the explanation voice.In this pattern 3, as in the case of pattern 2, the voice speed is changed between the video and voice C1 ″, C2 ″, C3 ″,... And the explanation voice D1 ″, D2 ″,. For this reason, the sub-voice generated by the voice connection unit 16 described later is easier to hear than a combination of voices having different speech speeds. In addition, since the explanation voices D1 ″, D2 ″,... Overlap with the talk sections A1, A2, A3,... This is suitable for programs that are narrated and rarely appear.

  Then, the speech speed conversion means 15, based on the section information input from the sound analysis means 13, for each of the set patterns 1 to 3, the video and audio corresponding to each talk section and pause section, and explanations Set the expansion / contraction and speech speed of the section with the voice. Further, the speech speed conversion means 15 outputs information indicating the expansion / contraction of the video / audio and the explanation voice of the patterns 1 to 3 to the video / audio output means 17 described later. This information is further output to the output device 5, and when a command for designating a pattern is input via the output device 5 by an operator who refers to this information, the speech speed conversion means 15 performs an explanation voice based on this command. Or the speech speed is converted for both the video and audio and the commentary audio, and the video and speech that have been converted to the speech speed are output to the audio connection means 16. Further, the speech speed conversion means 15 outputs the information about the expansion and contraction of the video and audio and the commentary voice converted at the speech speed to the video speed conversion means 12.

  Note that the speech speed conversion pattern is not limited to the above example. In addition, although the speech speed conversion pattern described here is set in advance, the operator may appropriately modify or add the speech speed conversion pattern of the speech speed conversion means 15.

  Returning to FIG. 1, the description will be continued. The audio connection means (audio adding means) 16 connects the video and audio converted from the speech speed by the speech speed conversion means 15 and the explanation audio to generate sub audio. The sub audio generated here is output to the video / audio output means 17 and the sub audio output means 21.

  The video / audio output unit 17 receives the video / audio input from the audio analysis unit 13, the commentary audio input from the audio synthesis unit 14, the video input from the program video storage device 3, and the video speed conversion unit 12. The input sub-audio synchronized video is converted into a format that can be output based on a command from an operator input from an input means (not shown) and output to the output device 5. The video / audio output means 17 receives the detection results of the beat and pause sections input from the audio analysis means 13 and the speech speed conversion pattern input from the speech speed conversion means 15 from an input means (not shown). Based on the input command from the operator, the output format is also converted to a displayable output format and output to the output device 5. Further, the video / audio output means 17 converts the sub-audio input from the audio connection means 16 into a displayable output format on the basis of a command from an operator input from an input means (not shown) and outputs the output device 5. Also output to

  Here, an example of an image generated by the video / audio output means 17 and displayed on the display screen of the output device 5 will be described with reference to FIG. FIG. 3 is a schematic diagram showing an example of an image generated by the video / audio output means.

  As shown in FIG. 3, the image W mainly includes an audio analysis result presentation area E0, pattern presentation areas E1, E2,..., A video reproduction area F, and a comment document presentation area G.

  The audio analysis result presentation area E0 is an area for visualizing and presenting video and audio sections on the time axis of the video playback time. In this case, the video / audio output unit 17 enters the sections A1, A2,... And the pause sections B1, B2 in the voice analysis result presentation area E0 based on the information on the beat sections and pause sections input from the voice analysis means 13. , ... and present.

  The pattern presentation areas E1, E2,... Are areas for presenting speech speed conversion patterns of video and audio. Here, the video / audio output means 17 performs video playback time in each of the pattern presentation areas E1, E2,... Based on the information indicating the expansion and contraction of the video and audio input from the speech speed conversion means 15. On the time axis, a section between the video and audio of each pattern of speech speed conversion and commentary audio is visualized and presented. For example, in the pattern presentation area E1, video and audio C1, C2,... Are explained in the same section as the pause sections B1, B2,... In the same section as the beat sections A1, A2,. Voices D1, D2,... Are presented.

  The video reproduction area F is an area for presenting video based on a command input from an operator by an input means (not shown). Here, when the operator drags and selects an arbitrary area in the voice analysis result presentation area E0 and the pattern presentation areas E1, E2,... With a mouse or the like (not shown), the video / audio output means 17 displays the video in that range. And the sub audio synchronized video is presented in the video reproduction area F. At the same time, the video / audio output means 17 outputs the selected range of video / audio and commentary audio to a speaker (not shown) provided in the output device 5. Thereby, the operator can select the most appropriate one from the patterns of speech speed conversion shown in the pattern presentation areas E1, E2,.

  The comment manuscript presentation area G is an area for inputting and presenting comment manuscripts corresponding to pause sections B1, B2,. Here, the comment document presentation area G has a comment document input area G1, and when the operator designates a pause section by input means (not shown) and inputs the text of the comment document, the video / audio output means 17 The explanation manuscript is displayed in the explanation manuscript input area G1. Then, when the operator confirms the comment document, the comment broadcast program generating apparatus 1 can input the comment document corresponding to each pause section B1, B2,... From the comment document input means 11. In addition, the video / audio output unit 17 causes the voice synthesis unit 14 to synthesize the pose section in which the operator has designated and reproduced a pause section in which the commentary document has already been input using an input unit (not shown). Is output to a speaker (not shown) provided in the output device 5.

  Returning to FIG. 1, the description will be continued. The original video output means 18 outputs the video (original video) input from the program video storage device 3. This original video is stored in the program video storage device 3 and is a video produced by the producer. The original video output here is stored in the explanation broadcast program storage device 7.

  The sub audio synchronized video output means 19 outputs the sub audio synchronized video input from the video speed converting means 12. The sub-audio synchronized video output here is stored in the explanation broadcast program storage device 7 in association with the original video output from the original video output means 18.

  The main audio output means 20 outputs video audio (main audio) input from the program video storage device 3. This main audio is the audio of the video stored in the program video storage device 3 and produced by the producer. The main audio output here is stored in the explanation broadcast program storage device 7 in association with the original video output from the original video output means 18.

  The secondary audio output unit 21 outputs the secondary audio input from the audio connection unit 16. The sub audio output here is stored in the explanation broadcast program storage device 7 in association with the sub audio synchronized video output from the sub audio synchronized video output means 19.

  Note that the explanation broadcast program generating apparatus 1 can also realize each means as a function program in a computer, and the function programs are combined to operate as an explanation broadcast program generation program (explanation additional sound generation program). It is also possible.

[Operation of commentary broadcast program generator]
Next, with reference to FIG. 4, the operation of the explanation broadcast program generating apparatus 1 will be described. FIG. 4 is a flowchart showing an operation in which the explanation broadcast program generating device generates an explanation broadcast program.

  The explanation broadcast program generating apparatus 1 performs audio analysis on the video and audio stored in the program video storage apparatus 3 connected to the outside by the audio analyzing means 13, and detects a talk section and a pause section (step S11). Here, the explanation broadcast program generating apparatus 1 outputs the result of the audio analysis to the output apparatus 5 connected to the outside by the video / audio output means 17.

  Then, the explanation broadcast program generating apparatus 1 inputs the explanation manuscript created by the operator through the explanation manuscript input means 11 (step S12). The commentary manuscript input here is associated in advance with each of the pause sections detected in step S11 by the operator. Further, the commentary broadcast program generating device 1 generates a commentary voice by synthesizing the commentary document input in step S12 by the voice synthesizing unit 14 (step S13).

  Subsequently, the commentary broadcast program generating device 1 sets the expansion / contraction and the speech speed of the section between the video and audio and the commentary sound by the speech speed conversion means 15 (step S14). Here, a plurality of speech speed conversion patterns are set in advance, and the commentary broadcast program generating apparatus 1 sets the expansion / contraction of the section between the video and audio and the commentary sound and the speech speed for each pattern by the speech speed conversion means 15. Thus, the video / audio output means 17 outputs an image showing expansion / contraction of the video / audio and the commentary audio to the output device 5 connected to the outside.

  Then, the commentary broadcast program generating apparatus 1 converts the video and audio and the commentary voice to the speech speed set in step S14 by the speech speed conversion means 15 (step S15). Here, the explanation broadcast program generating apparatus 1 converts the audio and video and the explanation voice into the designated pattern speaking speed based on a command for designating the speaking speed conversion pattern of the worker input from the outside.

  Furthermore, the commentary broadcast program generating device 1 connects the video and audio converted to the speech speed in step S15 by the audio connecting unit 16 to generate sub audio (step S16). The commentary broadcast program generating device 1 corresponds to the video and audio stored in the program video storage device 3 in accordance with the expansion and contraction of the video and audio and the commentary audio converted at step S15 by the video speed conversion means 12. The sub-audio synchronized video is generated by expanding and contracting the video to be played (step S17).

  Further, the commentary broadcast program generating device 1 outputs the video input from the program video storage device 3 as an original video from the original video output means 18 to the commentary broadcast program storage device 7, and the sub audio synchronized video output means 19 performs step S17. Is output to the explanation broadcast program storage device 7. Also, the commentary broadcast program generating device 1 inputs the video and audio from the program video storage device 3 by the main audio output means 20 and outputs it as the main sound to the commentary broadcast program storage device 7, and the sub audio output means 21 performs the step. The sub audio generated in S16 is output to the explanation broadcast program storage device 7 (step S18), and the operation is terminated. The original video, sub audio synchronized video, main audio, and sub audio output here are associated with each other and stored in the explanation broadcast program storage device 7.

  Through the above operation, the explanation broadcast program generation device 1 generates sub-audio and sub-audio synchronized video of a plurality of videos stored in the program video storage device 3, and the explanation broadcast program storage device together with the original video and the main audio. 7 can be accumulated. The operator can store the necessary video and sub-audio on an optical disc such as a DVD (Digital Versatile Disc) or a VTR.

It is the schematic diagram which showed typically the structure of the description broadcast program production | generation apparatus in this invention. Explanatory drawing for demonstrating the speech speed conversion by the speech speed conversion means of the commentary broadcast program production | generation apparatus in this invention, (a) showed typically the example of the talk section and pause section detected by the audio | voice analysis means. Schematic diagram, (b) is a schematic diagram schematically showing an example of a speech speed conversion pattern by the speech speed conversion means, and (c) is a schematic example of another pattern of speech speed conversion by the speech speed conversion means. (D) is a schematic diagram schematically showing another pattern example of speech speed conversion by the speech speed conversion means. It is a schematic diagram which shows an example of the image produced | generated by the video / audio output means of the commentary broadcast program production | generation apparatus in this invention. It is the flowchart which showed the operation | movement which the comment broadcast program production | generation apparatus in this invention produces | generates a comment broadcast program.

Explanation of symbols

1 Commentary broadcast program generation device (explanatory additional sound generation device)
12 Video speed conversion means 13 Audio analysis means (section detection means)
14 speech synthesis means 15 speech speed conversion means 16 voice connection means (speech addition means)
3 Program Video Storage Device 5 Output Device 7 Explanation Broadcast Program Storage Device

Claims (5)

A commentary that generates a commentary supplementary sound by adding a commentary sound in which the text data is converted into a voice to the video and voice, and the video and voice that is the voice of the video and text data related to the content of the video are input from the outside. An additional sound generation device,
Speech synthesis means for generating the commentary speech by speech synthesis from the text data;
On the time axis of the playback time of the video and audio from the video and audio, a section detecting means for detecting a voicing sound section that is a voice section of a uttered sound and a pause section that is a voice section of silence or background sound only;
A speech speed converting means for converting the commentary speech based on a section length of the pause section;
Voice adding means for adding the commentary voice converted by the voice speed conversion means to the detected pause section of the video and audio to generate the commentary additional voice;
A commentary-added voice generation device characterized by comprising: A commentary that generates a commentary supplementary sound by adding a commentary sound in which the text data is converted into a voice to the video and voice, and the video and voice that is the voice of the video and text data related to the content of the video are input from the outside. An additional sound generation device,
Speech synthesis means for generating the commentary speech by speech synthesis from the text data;
On the time axis of the playback time of the video and audio from the video and audio, a zone detection means for detecting a speech zone that is a voice zone of a uttered sound and a pause zone that is a voice zone of only silence or background sound;
The video and audio corresponding to the utterance sound section is converted to speech speed, and the commentary voice is converted to speech speed based on the length of the utterance sound section expanded and contracted by the speech speed conversion and the section length of the pause section. Speaking speed conversion means,
Relative silence or pause section is a speech section of only the background sound in the video voice converted speech speed by the front Kihanashi speed conversion means, audio additional adding a commentary voice converted speech speed by the speech speed converting means Means,
Description additional voice generating device wherein Rukoto equipped with.   A section of a section corresponding to the uttered sound section of the video input from the outside based on information indicating expansion / contraction by the speaking speed conversion in the utterance sound section of the video and audio converted by the speech speed converting means. The comment-added audio generation apparatus according to claim 2, further comprising video speed conversion means for expanding and contracting the length. In order to generate a commentary-added voice in which a video voice that is a video voice and text data related to the content of the video are input from the outside and a commentary voice obtained by converting the text data into a voice is added to the video voice. Computer
Speech synthesis means for generating the commentary speech by speech synthesis from the text data;
On the time axis of the playback time of the video and audio, a section detection means for detecting a voicing sound section that is a voice section of a uttered sound and a pause section that is a voice section of silence or only background sound from the video and audio,
A speech speed converting means for converting the commentary speech based on a section length of the pause section;
Voice adding means for adding the commentary voice converted by the voice speed conversion means to the detected pause section of the video and audio to generate the commentary additional voice;
A commentary-added voice generation program characterized by functioning as   In order to generate a commentary-added voice in which a video voice that is a video voice and text data related to the content of the video are input from the outside and a commentary voice obtained by converting the text data into a voice is added to the video voice. Computer
  Speech synthesis means for generating the commentary speech by speech synthesis from the text data;
  On the time axis of the playback time of the video and audio, a section detection means for detecting a voicing sound section that is a voice section of a uttered sound and a pause section that is a voice section of silence or only background sound from the video and audio,
  The video and audio corresponding to the utterance sound section is converted to speech speed, and the commentary voice is converted to speech speed based on the length of the utterance sound section expanded and contracted by the speech speed conversion and the length of the pause section. Speaking speed conversion means,
  Audio adding means for adding commentary speech converted by the speech speed conversion means to a pause section that is a speech section of only silence or background sound in the video and audio converted by the speech speed conversion means. ,
A commentary-added voice generation program characterized by functioning as

Images