JPH11308565A - Method and system for displaying voice recording aid into video image and recording medium recording this method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an interface by which an editor easily instructs when and how a talker makes a speech to the talker and the talker easily makes a speech by the instruction in the case of recording a voice in a video image. SOLUTION: First a video image block segmentation means 10 segments a video block to which a voice is inserted. A text addition means 11 adds text information of a voice desired to be recorded to the segmented video block. A voice synthesis means 12 synthesizes voice from the added text information. A synthesized voice processing means 13 processes length, height, intonation, voice tone and voice timing or the like of the synthesized voice information. A text and voice instruction means 14 reproduces the processed and synthesized voice and displays a voice talking timing. A voice recording means 15 records the voice talked by the talker by this instruction, a comparison means 16 compares a degree of coincidence between the recorded voice and the processed and synthesized voice to decide adoption of the recorded voice and the recorded voice whose adoption is decided is combined with the video block.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a technology relating to an interface for instructing a timing of inputting audio when adding audio information to a video. The present invention relates to a voice recording support display method and apparatus useful for support.

[0002]

2. Description of the Related Art With the rapid development of image digital processing technology, high quality images can be easily edited with cheap equipment.
It is possible to create.

[0003] A prominent example of the use of video by digital technology is a non-linear video editing device. By expanding the contents of the video, which is time-series information, spatially and enabling random access to each video section, it is possible to freely edit the video without being bound by the time series .

[0004] However, audio information called one-dimensional time-series information has a problem in that it is difficult to spatially expand the information, so that the problem is large when editing.

[0005] For example, in operations such as narration and serif insertion, which are important processes of video editing, an operation is performed in which an audio section is cut out from a recorded original audio, and the time for arranging it in a video is appropriately changed and then arranged. Was. If the time and timing of the audio are not compatible with the video, an operation of re-recording the adapted audio many times is required.

Therefore, instead of recording such voices, there has been proposed a method of inputting text to be spoken and adding voices by voice synthesis. For example, "Speech creation tool capable of realizing various speech expressions-Speed
97- "(Speech and Language Information Processing 17-12 1997/7
/ 19, Masanobu Abe, Hideyuki Mizuno, Shinya Nakatsuta).

[0007]

However, in the above conventional method of inputting text instead of voice and adding voice by voice synthesis, the quality of voice to be synthesized is low.
There was a problem that personality was not reflected.

[0008] Therefore, in order to edit and produce a video work using high-quality sound, it is better to record and use raw original sounds, including humans, in terms of quality as well as personal information. It was also desired from the point.

However, as described above, the recording of the original sound
There is a problem that the input is difficult.

Therefore, in order to solve the above-mentioned problems, the present invention requires the editor to have the utterer utter a sound to be recorded in video editing at any time. It is an object of the present invention to provide an interface that can easily give an instruction such that the speaker can easily perform an operation of making the user understand the instruction and make it easier to speak.

[0011]

SUMMARY OF THE INVENTION The present invention solves the above-mentioned problems by the inventions listed below.

[0012] The invention of (1) provides a method for supporting voice recording in a video, which displays a requirement from an editor to a speaker when voice is inserted when voice information is inserted into the video. The video section is cut out by limiting the video section that the user wants to perform, a text section with a text is generated by adding text information corresponding to the sound to be inserted to the cut video section, and the text of the video section with the text is generated. A synthesized speech is generated by synthesizing speech from information, and a processed synthesized speech is generated for the synthesized speech by processing the length, height, intonation, timbre, timing of speech, etc. of the speech information, and the speech to be inserted. Is given a specific instruction on what kind of sound should be added to the video section by using the processed synthesized speech and the text information. The recorded content is recorded, a recorded voice is obtained, the recorded voice is compared with the processed synthetic voice, and the result of the comparison is used to determine a video section with sound from the recorded voice and the video section. This is a method for supporting and displaying audio recording in a video, which is characterized in that it is generated.

[0013] The invention of (2) provides a method for supporting voice recording in a video, which displays a request condition from an editor to a speaker when voice is to be inserted when voice information is inserted into the video. The video section is cut out by limiting the video section that the user wants to perform, a text section with a text is generated by adding text information corresponding to the sound to be inserted to the cut video section, and the text of the video section with the text is generated. Synthesizes a voice from the information to generate a synthesized voice, modifies the synthesized voice to generate a processed synthesized voice, synchronizes the video section with text information with the processed synthesized voice, and edits as necessary To the speaker, which extracts three or some of the synchronized video section, processed / synthesized voice, and text information, reproduces them synchronously, and utters the voice to be inserted. Depart for Voice to be instructed, required conditions such as timing of utterance, etc., record the content uttered by the speaker, obtain a recorded voice, compare the recorded voice with the processed synthesized voice, A method for supporting and displaying audio recording in a video, wherein a video section with audio is generated from the recorded voice and the video section based on the result of comparison.

[0014] The invention of (3) provides a voice recording support display apparatus for displaying a requirement from an editor to a speaker when voice is to be inserted when voice information is inserted into a video. A video section cutout section that cuts out a video section that the user wants to perform, and a text addition section that generates text-added video section by adding text information of audio to be recorded to the video for the cutout video section, A voice synthesis unit that generates a synthesized voice from the text information of the video section with text; a synthesized voice processing unit that processes the synthesized voice to generate a processed synthesized voice; A text / audio / video synchronizing unit for synchronizing the synthesized voice and displaying it to the editor as necessary, and three or more of the synchronized video section, processed synthesized voice, and text information A text-synchronous playback unit that extracts and sorts the types of text and reproduces them synchronously, and instructs the speaker on required conditions such as the voice to be uttered, the timing of the utterance, and voice recording that records the voice uttered by the speaker Department and
A video, comprising: a comparing unit that compares the degree of coincidence between the recorded voice and the processed synthesized voice; and a video / audio adding unit that adds the recorded voice to the video section based on the comparison result. It is a voice recording support display device for inside.

According to the invention of (4), when audio information is inserted into a video, a video section in which audio input is desired to be performed when displaying a requirement from an editor to a speaker when uttering audio is displayed. Extracting a video section, adding text information corresponding to the sound to be inserted to the extracted video section to generate a video section with text, and synthesizing audio from the text information of the video section with text. A step of generating a synthesized voice, a step of generating a processed synthesized voice by processing the synthesized voice, and a speaker who utters the voice to be inserted, using the processed synthesized voice and the text information, A procedure for giving specific instructions on what kind of sound should be added to the video section, a procedure for recording the content uttered by the speaker and obtaining a recorded voice, Processing synthesis The computer reads a program for causing the computer to execute a procedure of comparing the degree of matching of voices, and a procedure of generating a video section with sound from the recorded voice and the video section based on the comparison result. A recording medium for recording a method for supporting and displaying audio recording in a video, characterized by being recorded on a possible recording medium.

According to the invention of (5), when audio information is inserted into a video, a video section in which audio input is desired to be performed when displaying a requirement from an editor to a speaker when uttering a voice is displayed. A function of adding text information of a sound to be recorded to a video to generate a video section with text information for the extracted video section, and synthesizing the text information of the video section with text information. A function of generating a voice, a function of processing the synthesized voice to generate a processed synthesized voice, and synchronizing the video section with text information and the processed synthesized voice, and displaying the processed synthesized voice to an editor as necessary And three or some of the synchronized video section, processed / synthesized voice, and text information are extracted and played back in synchronization, and the voice to be uttered to the speaker, Taimi A function for instructing the requirements such as grayed, a function to record a voice uttered by the speaker,
A program for realizing a function of comparing the degree of coincidence between the recorded voice and the processed synthetic voice and a function of adding the recorded voice to the video section based on the result of the comparison by the computer. Is a recording medium for recording a method for supporting and displaying audio recording in a video, the method being recorded on a readable recording medium.

According to the present invention, the editor inputs text to be uttered by the speaker as text, and performs a synthetic voice and an operation on the synthesized voice to determine in advance when and how to utter text information. It can be easily expressed. On the other hand, referring to the example of the text information and the synthetic voice information, the speaker can intuitively understand the utterance information when and at what timing and in what atmosphere. Furthermore, by having the function of comparing the synthesized voice with the voice of the speaker, the speaker can easily utter a voice approaching the ideal voice considered by the editor, and the editor can select from among the uttered voices This makes it easier to select and use, and makes it possible to add audio to video.

[0018]

Embodiments of the present invention will be described below in detail with reference to the drawings.

FIG. 1 shows a processing flow of a method for supporting audio recording into a video image according to an embodiment of the present invention.

First, input video data 0 as a material for editing is provided. From there, the video section 1 to which audio is to be added is cut out using the video section cutout means 10.
The editor adds text information 2 corresponding to the audio information to be added by the text adding means 11 to the clipped video section 1. It is desirable that the text information 2 at this time is represented by phonetic symbols as much as possible.

The video section 3 with the text thus obtained
Then, the synthesized voice 4 is generated from the text portion by the voice synthesis means 12. The generated speech synthesis 4 is
The synthesized voice processing unit 13 further processes the synthesized voice to obtain the processed synthesized voice 5. The synthetic voice processing means 13 may be manually operated or fully automatic.

From the generated synthesized speech 5, an insertion speech 6 for inserting speech is obtained by the text and speech instruction means 14. The inserted voice 6 is recorded by the voice recording means 15 to obtain a recorded voice 7.

The recording sound 7 is compared with the processed synthesized sound 5 by the comparing means 16, and if it is determined that the recorded sound 7 is to be used, the recorded sound 7 is adopted and combined with the video section to produce a video with sound. Section 8 can be generated.

FIG. 2 is a block diagram showing an example of the configuration of an apparatus for supporting audio recording in a video according to an embodiment of the present invention.

There is provided a recording medium 100 for storing a result of processing a video and a target video, and an input video 20 is first stored in the recording medium 100. The input video 20 only needs to be digitized, and any video data may be used. For example, it may be uncompressed video data, may be compressed by JPEG or the like, or may be MPEG encoded. Various file formats such as AVI and QuickTime may be used. The recording medium 100 for recording the input image and various processing results may be a tape, a CDR, a DVD, a memory (Memory), or the like using magnetism, light, or the like, but is a medium that can read and write at sufficiently high speed. Desirably.

From the input video 20, the editor 30 gives an instruction to the video section cutout section 40 via the editor instructing section 50 to cut out an arbitrary video section 21. At this time, the video section cutout unit 40 may cut out completely manually or may cut out automatically. For this purpose, a conventional technique such as Japanese Unexamined Patent Publication No. Hei 5-37853, “Automatic Video Cut Automatic Division Method” can be used.

The editor 30 similarly adds text information 31 describing the text content of the audio information to be added to the video via the editor instructing section 50. This text information 31
Is preferably a phonetic symbol, but any ideographic symbol or the like may be used as long as its pronunciation is uniquely determined. If it is a symbol that can be uniquely determined as to how to pronounce, it is not necessarily required to be a letter. The text information adding section 41 adds the text information 31 to the video section 21.

The text information 31 is not only added to the video but also is subjected to voice synthesis in a voice synthesis section 51 to generate a synthesized voice 32. The length of the voice, the pitch of the voice, the intonation, the timbre, etc. may be arbitrary. Synthetic speech 3
2 is generally mechanical, and the intonation and the like may not be in an ideal state. Therefore, the synthesized voice processing unit 52 specifies the intonation of the synthesized voice 32 and information such as the timing of utterance, Process. Since it is assumed that the synthesized speech 32 is added to the video section 21, the synthesized speech processing unit 52 refers to the video section with text information 22 so as to match the video section 21.
Processing can be performed.

That is, in the processed / synthesized speech 33, where the synthesized speech is uttered in accordance with the time series of the video,
Processing up to the end of the process is performed.
The processed and synthesized voice 33 thus obtained is converted by the text / audio / video synchronizing unit 42 into the video 22 with text information,
Synchronization is performed, and reproduction can be performed in a state where the video, the synthesized voice, and the text are completely synchronized by the video and audio corresponding text 23. The text here is text information from which the synthesized speech is based, and the condition is that the pronunciation content is uniquely determined. When video and audio are reproduced, which part of the text information is explicitly reproduced. You can see if it is done. Details of how this is specified will be described later.

When the obtained video and audio corresponding text 23 is reproduced by the text synchronous reproduction section 44, a voice is actually inputted and a speaker 35 who utters the voice to be added is given a voice message during reproduction of the video. It is possible to express explicitly what kind of sound should be produced at what timing.

The utterer 35 sends a message to the utterance recorder 45
A voice is uttered, and a recorded voice 36 is obtained. The voice comparison unit 46 compares the obtained recorded voice 36 with the processed synthesized voice 33 to determine whether it matches the processed synthesized voice 33. The recorded voice 37 is selected and extracted. By combining the obtained use recorded voice 37 and the video and voice corresponding text 23 in the video / audio adding unit 47, a video with voice 38 by the speaker is generated. Obtained video 3 with audio
8 is stored in the recording medium 100.

In each block, a video section 21, a video section with text information 22,
Text information 31, synthesized speech 32, processed synthesized speech 33,
The recorded voice 36, the video / audio corresponding text 23, the used recorded voice 37, and the like are appropriately stored in the recording medium 100. In the block diagram of FIG. 2, only a part is explicitly stored, but expansion such as storing all information can be easily analogized, and is within the scope of the embodiment of the present invention. It is.

Hereinafter, some blocks in FIG. 2 will be described.

The speech synthesizer 51 generates a synthesized speech 32 based on text information 31 composed of phonetic symbols whose pronunciation is uniquely determined. Various synthetic speech generation tools have already been proposed, including the examples introduced in the conventional method. Similarly, in the speech synthesis processing section 52, the synthesized speech 32
There have been proposed various means for arbitrarily changing the starting point, intonation, and length of pronunciation of a sound, such as Speed97, introduced by a conventional method. That is, generation and processing of synthesized speech can be realized by using an existing technology.

Next, in the text-synchronous reproduction section 44, the editor 30 can easily understand the specific example of what timing, what to pronounce, and where to end it for the speaker 35 who actually speaks. It is desirable. Both audio and video information are time-series information, and it is most effective to display text information consisting of phonetic symbols while displaying the time-series. This text synchronous playback unit 44-2
FIG. 3 shows two specific images.

FIG. 3A shows a first image,
This is an example in which time-series information is expanded and shown in a space, in which a part or all of an image is displayed. On the other hand, some or all information is displayed in the audio track,
The corresponding text information is also displayed. Among them, while the displayed frame and the audio track are switched in chronological order when the video is reproduced, the corresponding text information is also switched.

FIG. 3B shows a second image, in which an image is displayed on a video display means such as a monitor, and audio information is reproduced by a voice display means such as a speaker while a part of the monitor or a dedicated monitor is used. Display text information in the text display section. Part of the text information is synchronized with the transition of the synchronized video, and the reproduced characters will change color, etc.
Express the change transition of text information.

Next, the utterance recording section 45 records the voice of the utterer 35. The recorded voice 36 may be a monaural recording, a stereo recording, a coded recording, or may use any bandwidth. Of the recorded voices 36, the determination as to whether or not to be adopted as the use recorded voice 37 is made by a voice comparison unit 36 for comparison with the processed and synthesized voice 33, and from the result, automatic determination and determination by the editor are performed. Is done. The voice comparison unit 46 has a function of comparing whether or not the time-series information based on the synthesized voice matches the uttered information. The method can be realized, for example, by comparing the level of the synthesized voice with the level of the recorded voice, and comparing the relative changes in the voice levels.

In addition to such a sound level comparison method, other operation methods such as a comparison based on a change in a waveform or a frequency distribution can be considered. Some comparison methods are shown in FIG. This is a comparison of the sound volume. The sound volume change of the processed speech synthesis is compared with the recorded sound. When the difference in which the difference result is close to 0 and the difference is small, there is no change in the sound volume. It is assumed that the voice has been played. On the other hand, where the difference is large, there is a change in the sound volume, so it is assumed that they are different. At the time of comparison, a process of normalizing the volume is performed in advance. Furthermore, in the comparison in the frequency band as well as in the comparison based on the volume, it is also possible to extract each piece of pitch information and compare the frequency components to perform evaluation.

It should be noted that the comparison result obtained by the audio comparing section 46 may be used as reference information by the editor 30 to finally make a judgment by adding a qualitative evaluation, or the comparison result may be quantitatively determined. May be evaluated.

The method and apparatus for supporting audio recording in a video have been described above. It is easy to guess, by adding various arrangements to the above-described embodiment, and they are also within the scope of the present invention.

It should be noted that a computer program describing the processing procedure described with reference to FIG. 1 or a recording medium recording a computer program for realizing the function of each block described with reference to FIG. The present invention can be implemented as long as there is a computer that can read and execute a program from a recording medium. The recording medium may be, for example, a ROM, FD, memory card, MO, CD, DVD, or the like, or any other computer-readable medium.

[0043]

According to the present invention, when audio information is added to a video, the editor can easily instruct the person who actually utters the voice to be recorded, and the speaker can determine when, when, and when. It is possible to provide an interface that enables specific instructions from an editor to be received as to what kind of intonation should be pronounced. This makes it possible to reduce the burden on the speaker of the sound to be recorded, and to easily produce a video work with sound. Therefore, there is an effect of supporting video production.

[Brief description of the drawings]

FIG. 1 is a process flowchart illustrating an embodiment of a method for supporting audio recording in a video according to the present invention.

FIG. 2 is a block diagram illustrating an embodiment of an apparatus for supporting audio recording in a video according to the present invention.

FIGS. 3A and 3B are diagrams showing an image of a text synchronous reproduction unit in the audio recording support apparatus for video in the embodiment of the present invention.

FIG. 4 is a diagram illustrating an example of a sound comparison method in a sound comparison unit in the embodiment of the sound recording support apparatus for a video.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 10 ... Video section cutout means 11 ... Text addition means 12 ... Speech synthesis means 13 ... Synthesis voice processing means 14 ... Text and voice instruction means 15 ... Generation recording means 16 ... Comparison means 40 ... Video section cutout part 41 ... Text information addition part Reference numeral 42: Text / sound synchronization unit 44: Text synchronization reproduction unit 45: Voice recording unit 46: Voice comparison unit 47: Video / audio addition unit 50: Editor instruction unit 51: Voice synthesis unit 52: Synthetic voice processing unit 100: Recording media

Claims (5)

[Claims] 1. A video section in which voice input is to be performed in a method of supporting voice recording in a video, which displays a request condition from an editor to a voice speaker when voice is inserted when voice information is inserted into the video. To cut out the video section, add text information corresponding to the sound to be inserted to the cut-out video section to generate a video section with text, and synthesize audio from the text information of the video section with text Then, a synthesized voice is generated, and a processed synthesized voice is generated by processing the synthesized voice. A speaker who utters the voice to be inserted is provided with the processed synthesized voice and the text information by using the processed synthesized voice and the text information. A specific instruction is given as to what kind of sound should be added to the video section, the contents spoken by the speaker are recorded, a recorded sound is obtained, and the recorded sound and the processed synthesized sound are obtained. Matching tool Comparing, in result of the comparison, and a 該録 sound audio and the video section, and generates an image with sound interval, sound recording support display method for the video, characterized in that. 2. A method according to claim 1, wherein when inserting audio information into a video, a method of supporting voice recording in a video, which displays a requirement from an editor to a speaker when the voice is emitted, is used in a video section in which a voice is to be input. To cut out the video section, add text information corresponding to the sound to be inserted to the cut-out video section to generate a video section with text, and synthesize audio from the text information of the video section with text Process the synthesized voice to generate a processed synthesized voice, synchronize the video section with text information with the processed synthesized voice, and display it to the editor as necessary. Then, three or some of the synchronized video section, processed synthetic voice, and text information are extracted, reproduced in synchronization, and should be uttered to the speaker who utters the voice to be inserted. voice , Requesting conditions such as utterance timing, recording the contents uttered by the speaker, acquiring a recorded voice, comparing the recorded voice with the processed synthetic voice, and comparing the results. And generating a video section with sound from the recorded voice and the video section. 3. A video section in which audio input is to be performed in a display apparatus for supporting audio recording in a video, which displays a request condition from an editor to a speaker when voice is inserted when audio information is inserted into the video. A video section clipping section for limiting the clipping of a video section, a text adding section for adding text information of a sound to be recorded to a video to generate a video section with text for the clipped video section, A speech synthesis unit that generates a synthesized voice from the text information of the above, a synthesized voice processing unit that processes the synthesized voice to generate a processed synthesized voice, and synchronizes the video section with text information with the processed synthesized voice. A text / audio / video synchronizing unit to be displayed to the editor as required, and three or some types of the synchronized video section, processed / synthesized audio, and text information A text-synchronous playback unit that outputs and reproduces in synchronism and instructs a speaker on required conditions such as a voice to be uttered, timing of utterance, and the like; a voice recording unit that records a voice uttered by the speaker; A comparison unit that compares the degree of coincidence between the recorded voice and the processed synthesized voice; and a video / audio addition unit that adds the recorded voice to the video section based on the comparison result. Voice recording support display device. 4. A procedure for inserting an audio information into a video and cutting out a video section in which an audio input is to be performed in a case where a requirement from an editor to a speaker at the time of uttering a voice is displayed. Adding a text information corresponding to a sound to be inserted to the cut-out video section to generate a video section with a text; and synthesizing a voice from the text information of the video section with a text to generate a synthesized voice. A procedure for processing the synthesized voice to generate a processed synthesized voice; and providing a speaker who utters the voice to be inserted to the video section using the processed synthesized voice and the text information. A procedure for giving specific instructions on what kind of voice should be added; a procedure for recording the content uttered by the speaker to obtain a recorded voice; and matching the recorded voice with the processed synthetic voice. A program for causing a computer to execute a procedure for comparing the combination, a procedure for generating a video section with sound from the recorded voice and the video section based on the comparison result, and A recording medium recording a method for supporting and displaying audio recording in a video, characterized by being recorded on the medium. 5. A function for cutting out a video section in which audio input is to be performed when inserting audio information into a video and displaying a request condition from an editor to a speaker when uttering a voice, A function of adding text information of a sound to be recorded to a video to the extracted video section to generate a video section with text information; and a function of generating a synthesized voice from the text information of the video section with text information. A function of processing the synthesized voice to generate a processed synthesized voice, a function of synchronizing the video section with text information and the processed synthesized voice, and displaying the processed synthesized voice to an editor as necessary, Synchronized video section, processed / synthesized voice, and three or more types of text information are extracted and reproduced in synchronization with each other, and the voice to be voiced, the timing of voice generation, etc. are requested from the speaker. A function of instructing conditions, a function of recording a voice uttered by the speaker, a function of comparing the degree of coincidence between the recorded voice and the processed synthetic voice, and A recording medium recording a method for supporting and displaying audio recording in a video, wherein a program for realizing the function added to a video section and the program for realizing the following by a computer is recorded on a recording medium readable by the computer.