JP2003223176A - Data preparation/editing support system for superimposed character using speech approximate data - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a system for supporting data preparation/editing for super imposed characters by using speech approximate data as a guideline of speech block designation. <P>SOLUTION: The specific frequency range (5-7 Hz, for example), component of a sound power value is sliced at a suitable level and binarized data are prepared. In such a case, since a high level range presents the block of speech and a low level range presents the block of non-speech (pause), a timing to start/end the speech in sounds can be grasped in a certain degree from these data. Thus, by effectively utilizing the speech approximate data special processing sound data as a guideline of the speech block designation, a user can be concentrated on the comprehension/text of speech contents during writing-up and editing and such work can be effectively supported. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a semi-automatic caption program production system that effectively combines a manual caption production process and an automatic caption production process, by using speech approximation data to specify a speech section. By using it as a guideline, it relates to the technology that supports the creation and editing of subtitle data.

SUMMARY OF THE INVENTION The present invention is designed to display a speech approximation data obtained by specially processing voice data on a timeline so as to serve as a guideline for designating a speech section.

By facilitating the designation of the speech section, it is possible to understand the speech contents in transcription work and concentrate on the conversion to text, and it is possible to effectively support the creation and editing of subtitle data. Becomes

Therefore, it is possible to easily and efficiently create and edit subtitle data for various programs such as a program without an electronic manuscript and a program with a high background sound level, thereby improving the efficiency of subtitle program production. Can greatly contribute to

[0005]

2. Description of the Related Art As a technique for automatically producing a subtitle program offline, a news program or a narration-oriented documentary program is targeted, and when an electronic manuscript exists, "automatic summary", "automatic synchronization", and " There is a technology that has been constructed as an "automatic closed caption program production system" by aggregating research results such as "automatic closed caption screen creation technology".

These techniques have already been disclosed in Japanese Patent Application No. 11-72.
Although a patent application has been filed in Japanese Patent No. 671 etc., the range of programs to which such technology can be applied is limited, and automatic functions are provided for programs without electronic manuscripts, programs with a high background sound level such as dramas and varieties. However, there is a limit, so the part beyond the limit must be covered in the range of manual subtitle production, previewing and correction.

[0007]

[Problems to be Solved by the Invention] In the actual subtitle production field, many professionals with advanced specialized skills and knowledge are involved, and subtitle production is mostly dependent on such human ability. . On the other hand, under the situation where rapid expansion of subtitle programs is demanded like today, subtitle production is
Even if you are not an expert, it is desirable to have a system that allows part-time work to be shared by part-timers who can handle word-processing work. Therefore, it is necessary to consider the subtitle production efficiency not only as a subtitle production system that is premised on automatic processing, but also as a total system that includes the work of creating electronic text for subtitles, including manual work, and previewing and editing subtitle screens. There is.

[0008] Therefore, the inventors of the present invention, based on the knowledge obtained from the system evaluation of the automatic caption production system performed so far, will focus on a new automatic caption system which is a high performance of each automation element technology. , By applying the newly developed efficient manual subtitle production subsystem, we have developed a more practical semi-automatic subtitle production system that can support a wide program range.

This semi-automatic subtitle program production system has been separately filed, but in this semi-automatic subtitle program production system, the subtitle text creation function and the subtitle program data editing / preview function are to be performed manually. .

That is, simple and reliable text conversion of voice is an extremely important subject, but an error occurs in the current voice recognition technology. Therefore, in the semi-automatic subtitle program production system, the subtitle text creation function is performed by a human. It is supposed to be done by "transcription work".

This "transcription work" requires a high ability and a lot of time because it depends on human's advanced voice recognition ability and language judgment ability. In addition, the start and end timings of the speech are investigated and recorded, which increases the burden on the operator.

[0012] Here, if there is a system that supports this manual work, it is possible to reduce the ability, work time, and degree of tension required of the worker, and it is possible to perform more efficient "transcription work". Become.

Further, in the semi-automatic subtitle program production system, the operation of the subtitle program data editing / preview function is performed by an operator having specialized knowledge to preview the completed subtitle program data and correct it if necessary. In this work, if there is a system that assists the operator in making corrections to the subtitle content / timing, etc., the ability, working time, and degree of tension required of the operator can be reduced, and the efficiency can be improved. Edit work becomes possible.

The present invention has been made in view of the above problems, and in a semi-automatic subtitle program production system, assists in grasping the start and end timings of speech and performs transcription work and subtitle program data editing work. The purpose is to reduce the ability and tension required by the person.

[0015]

A subtitle data creation / editing support system based on speech approximate data according to the present invention comprises a subtitle text transcription function, an automatic subtitle program production function, and a subtitle program editing / preview function. A subtitle data creation / editing support system applied to a semi-automatic subtitle program production system that combines a subtitle production function with an automatic subtitle production function. And a means for displaying the speech approximation data on the timeline showing the time lapse of the program to be the caption program production target as a time axis. .

By specially processing the voice data, the data in which the speech component is emphasized is displayed on the timeline, so that the accuracy of grasping the timing of the start / end of the speech in the voice can be improved.

Further, according to the present invention, the speech approximate data creating means uses a cflx analysis value of the voice, a specific frequency range (for example, 5 to 7 Hz) component extraction value of the voice Power value, or a combination thereof. It has a feature.

The method using the cflx analysis value of the voice is a block cepstrum flux method (cflx method) which utilizes the feature of the voice level for detecting the speech section of the voice. The block cepstrum flux method is used. Is a method of more stably detecting the switching point of the audio data content by comparing a plurality of LPC cepstrum vectors in the audio data with each other from the reference frame.

The method using the extracted value of the specific frequency range component of the audio power value is a method performed by paying attention to the variation characteristic of the program audio power in the time axis direction. That is, when comparing the variation characteristic in the time-axis direction regarding speech with the pronunciation symbol string of speech, the speech power corresponding to the pronunciation symbol of the vowel tends to be larger than the others, and this variation characteristic in normal speed speech is The frequency range is approximately 4 to 7 Hz. This method basically extracts this frequency component and detects a range in which the component is equal to or more than a predetermined threshold value as a speech section.

By using these values, data in which the speech component is emphasized can be obtained.

[0021]

BEST MODE FOR CARRYING OUT THE INVENTION First, a semi-automatic caption program production system to which the present invention is applied will be described with reference to FIG.

The main function of the semi-automatic caption program production system is, as shown in FIG. 4, a caption text transcription function 1
, An automatic subtitle program data production function 2, a subtitle program data editing / preview function 3, and a basic GUI system 4 for overall control.

Here, the subtitle text transcription function 1 is a function of listening to the sound of the material program and inputting additional data such as transcription of the subtitle text and start / end timing, and the like. The sound is compressed and recorded on the disc of the personal computer, and the operation keys for playing the recorded video and audio and special playback operations are provided.
In order to support the "disk recording / playback control function" that performs the corresponding operation, and the manual work of inputting transcription and additional information data, various information related to the video / audio of the material program, transcription text, etc. is displayed on the timeline. "Information display function" that is displayed visually, "text creation control function" that has operation keys for inputting transcribed text and time data of speech pause, and performs corresponding operations, and data creation screen and main image display It consists of a screen and.

Further, the automatic subtitle program data production function 2 means that from among the subtitle texts arranged in the order of presentation time,
By forming a display unit subtitle sentence by appropriate line breaks and page breaks, and applying synchronization detection technology including voice recognition processing, the start point and end point are detected as synchronization points for each display unit subtitle sentence, and the start point / It is a function that automatically performs a series of operations that adds end point timing information to each display unit subtitle sentence,
It consists of an "automatic text summarization function" that summarizes the subtitle text if necessary, a "display unit subtitle creation function", and a "timing detection / addition function".

Further, the subtitle program data editing / preview function 3 manually edits / previews the subtitle program data automatically produced by the automatic subtitle program data production section based on the transcribed text for subtitles and additional information data. It has a dedicated operation key for special display operation for editing / previewing work regarding start and end time, subtitle page break, line feed, etc. "Control function", various information is visually displayed on the timeline, and especially for closed caption program data, the "information display / caption timing control function" that displays the timing change support screen and performs the corresponding operation, the page of closed caption data "Subtitle data page editing operation key and function" that has a dedicated operation key for unit editing and performs corresponding operation, finger superimposed on video Equipped with operation keys for displaying subtitle data, the corresponding operation "subtitle data / video display function" and operation keys necessary for selecting preview formats such as partial preview, continuous preview, etc. are provided. "Preview key and its function".

The subtitle data creation / editing support system using speech approximate data according to the present invention is applied to the subtitle text transcription function 1 and the subtitle program editing / preview function 3 in the semi-automatic subtitle program production system, and these functions are provided. It is a system for supporting.

First, regarding the subtitle data creation / editing support system based on the speech approximate data according to the present invention,
The basic principle will be described with reference to FIGS. 5 and 6, and then an embodiment in which this system is applied to a subtitle text transcription function 1 of a semi-automatic subtitle program production system will be described with reference to FIGS.
7 and 8, an embodiment in which this system is applied to the subtitle program editing / previewing function of the semi-automatic subtitle program production system will be described.

In the subtitle data creation / editing support system based on the speech approximate data according to the present invention, it is important to grasp the start and end timings of the speech in the transcribing work and the editing work of the subtitle text. Create speech approximation data and use it
By making it possible to easily grasp the speech section, it helps to grasp the start and end timings of speech.

Generally, in transcribing speech recorded on a tape, a method of slowing the reproduction speed of the tape to make it easier to listen to is used, and its effect is known.

However, in a documentary television program or the like, there are many cases where a relatively long non-speech (pause) section is interposed as compared with the case where the speech is continuous. In such a case, the tape operation is slowed down such that the tape playback speed is slowed down and the speech section is transcribed, and then the pause section is sent and then the next speech section tape is replayed at low speed. Transcription work is required, and a cueing operation performed while listening to the voice may be required in each speech section, which complicates the work.

Here, if the speech section can be easily grasped by creating and utilizing the speech approximate data including the cue for transcription, the transcription work can be effectively supported.

Similarly, in the editing work of a subtitle program,
If it is easy to understand the speech section by creating and utilizing the approximate speech data, it is possible to support the editing work in the production of subtitle programs.

FIG. 5 is an example in which a voice data waveform 51 is displayed as the speech approximate data.

The horizontal axis is a timeline showing the passage of time of the program, and when the sound is reproduced, the cursor is displayed at a position corresponding to this elapsed time and moves with the passage of time. Therefore, the reproduced voice and the voice waveform at each position of the cursor can be associated with each other.

The speech timing can be grasped to some extent from the audio waveform data depending on the background sound in the audio being sufficiently small or the experience of the waveform. However, in the normal program audio, various background sounds exist and their levels are also different. Because of the variety, it is generally difficult to accurately grasp the start and end timings of speech from this voice waveform data.

Here, if the speech approximation data in which the speech component is emphasized is used, it is possible to increase the accuracy of grasping the taming.

FIG. 6 shows an example using speech approximation data obtained by specially processing voice data. In FIG. 6, waveform 6
Reference numeral 1 is a cflx analysis value of the voice, waveform 62 is a component extraction value of a specific frequency range (for example, 5 to 7 Hz) of the voice power value, and waveform 63 is data obtained by slicing the waveform 62 at an appropriate level and binarizing it.

In the waveform 63, the high level range represents a speech section and the low level range represents a non-speech (pause) section, which almost coincides with the actually measured timing in this example. Therefore, the start and end timings of speech in the voice can be grasped to some extent accurately from the waveform 63.

As described above, by utilizing the speech approximation data obtained by specially processing the voice data as a guideline for the speech section designation, in the transcription and editing work,
Enables the understanding of speech contents and dedication to text,
These tasks can be effectively supported.

Next, an embodiment in which the subtitle data creation / editing support system using speech approximate data according to the present invention is applied to the subtitle text transcription function of a semi-automatic subtitle program production system will be described with reference to FIGS. 1 to 3. To do.

The subtitle text transcription function is a function for listening to the sound of the material program and inputting the subtitle text transcription and additional data. As described above, the "disc recording / reproduction control function" and the "information display" are displayed. Function ”,“ data creation control function ”, and data creation screen and main video display screen.

In the subtitle data creation / editing support system based on the speech approximate data according to the present invention, in the "information display function" which is a part of the subtitle text transcription function, the voice data is specially processed on the timeline. By displaying the speech approximate data, the manual work of transcribing and inputting additional information data is supported.

FIG. 1 shows a main screen for transcribing / editing to which the subtitle data creation / editing support system using speech approximate data according to the present invention is applied.

The main screen is a menu area 11 for calling each function, and an MPEG / AVI video display control area 1
2, an edit area 13 for subtitle text, and a list area 14 for images and subtitle text.

FIG. 2 shows a screen in which only 14 parts of the list area are taken out. In the list area 14, the image 21,
The subtitle text 22 and the waveform 23 are displayed. List area 1
The speech approximation data obtained by specially processing the voice data is displayed in the waveform 23 column of No. 4. In the waveform 23 column, a timeline 16 showing the passage of time is displayed as the horizontal axis. In the present embodiment, the specific frequency range (for example, 5 to 7 Hz) component of the voice power value is sliced at an appropriate level, and the binarized data is displayed as the speech approximate data. The cursor 15 moves over time and is displayed across the image, the subtitle text, and the waveform area, and the current mutual relationship can be grasped.

Next, FIG. 3 shows an example of a specific processing procedure for writing subtitle text and inputting additional information data.

As shown in FIG. 3, first, press [PLAY] to start image reproduction. Find the utterance timing (step S
1). Next, at the confirmation point of the utterance, press "start writing" (step S2). This point becomes the start point of the speech section. Then, it rewinds for a certain period of time and starts slow reproduction (step S3). Next, the transcription work is performed while listening to the reproduced sound (step S4). Next, when it is recognized that the speech has ended, the speech is appropriately rewound to search for the utterance ending point (step S5), and "end of writing" is pressed at the utterance ending point (step S6). The operations from steps S2 to S6 are repeated until the program ends. After a series of transcribing work is completed, scripts and terms are checked, summary support is executed (step S7), and then background sound information is registered (step S).
8). When the text creation is completed, the process proceeds to the automatic subtitle program data production process (step S9).

These operations are performed as shown in FIG.
Do it while looking at the main editing screen. In the list area 14 at the bottom of the main screen, as shown in FIG. 2, in the waveform 23 column below the column in which the subtitle text to be transcribed should be displayed, the specific frequency range of the audio power value recorded in the video file is displayed. Since the component (for example, 5 to 7 Hz) is sliced at an appropriate level and the binarized data is displayed, it is easy to confirm the utterance and find the speech end point.

Next, referring to FIG. 7 and FIG. 8, an embodiment in which the subtitle data creation / editing support system based on the speech approximate data according to the present invention is applied to the subtitle program edit / preview function of the semi-automatic subtitle program production system will be described. Explain.

As described above, the subtitle program data editing / previewing function 3 manually edits / previews the subtitle program data automatically produced by the automatic subtitle program data production section based on the produced subtitle text and additional information data. Is to
"Disc recording / playback and subtitle data control function", "Information display / subtitle timing control function", "Subtitle data page editing operation key and function", "Subtitle data / video display function", and "Preview key and its function" Consists of.

The subtitle data creation / editing support system based on the speech approximate data according to the present invention is in the "subtitle data / video display function" which is a part of the subtitle program data editing / preview function. By visually displaying speech approximation data obtained by specially processing audio data on the timeline, the subtitle program data editing work is supported.

FIG. 7 is a main screen for editing subtitle material to which the subtitle data creation / editing support system based on speech approximate data according to the present invention is applied. The main screen is
It is divided into a menu area 71 for calling various functions, an editing area 72 for inputting a subtitle text, and a list area 73 for MPEG / AVI images and subtitle text.

In the subtitle text list area 73, as shown in FIG. 8, an image 81, a subtitle text 82 and a waveform 83 are displayed. Here, as a waveform, a specific frequency range (for example, 5 to 7 Hz) component of the audio power value recorded in the video file is sliced at an appropriate level, and the binarized data is the speech approximate data, which indicates the passage of time. Since it is displayed on the timeline 75, the position of the speech or pose can be visually judged.

That is, in the subtitle text list area, since the subtitle text is displayed with the speaker and the text contents enclosed in a frame, the data displayed by binarizing as a waveform with reference to the cursor 74 is displayed. By comparing, you can change the time width of subtitle display and display timing while confirming the position of speech and pause.

As described above, since the position of the speech or the pose can be visually judged, it is possible to
Editing of subtitle display timing becomes easy.

[0056]

As described above, according to the present invention, it is possible to grasp the start and end timings of speech in voice to some extent accurately. Therefore, the present invention can be applied to a semi-automatic caption program production system. With this, it is possible to support the subtitle text transcription work and the editing work. Further, since the start and end timings of the speech can be visually grasped, it is possible to reduce the ability and the degree of tension required for the person who performs the transcription work or the closed caption program data editing work.

[Brief description of drawings]

FIG. 1 is a main screen for transcribing / editing to which a subtitle data creation / editing support system using speech approximate data according to the present invention is applied.

FIG. 2 is a screen in which only a list area portion is extracted from FIG.

FIG. 3 is a diagram showing a processing procedure of subtitle text transcription and additional information data input.

FIG. 4 is a functional configuration diagram of a semi-automatic caption program production system to which the present invention is applied.

FIG. 5 is a voice data waveform 51 as speech approximation data.
It is the figure which displayed.

FIG. 6 is a diagram showing speech approximation data obtained by specially processing voice data.

FIG. 7 is a main screen for editing subtitle material, to which the present invention is applied.

FIG. 8 is a screen in which only the list area portion is extracted from FIG.

[Explanation of symbols]

1 Subtitle text transcription function 2 Automatic subtitle program data production function 3 Subtitle program editing / preview function 11 Menu area 12 Control area 13 Editing area 14 List area 51 Audio data waveform 61 Audio cflx analysis value 62 Specific frequency range of audio power value Component extraction value 63 Specified frequency range of voice power value Binary data of component extraction value

─────────────────────────────────────────────────── ─── Continuation of front page (51) Int.Cl. ⁷ Identification code FI theme code (reference) H04N 5/278 H04N 7/08 A // H04N 7/025 7/03 7/035 (71) Applicant 000004352 Japan Broadcasting Corporation 2-2-1 Jinnan, Shibuya-ku, Tokyo (72) Inventor Eiji Sawamura 2-31-19, Shiba, Minato-ku, Tokyo (72) Inventor, Takao Kadoma, 2-shiba, Minato-ku, Tokyo 31-19 Communication / Broadcasting Organization (72) Inventor Noriyoshi Uratani 2-31-19 Shiba, Minato-ku, Tokyo Communications and Broadcasting Organization (72) Inventor Katsuhiko Shirai 2-31-19 Shiba, Minato-ku, Tokyo Communications F-term in broadcasting organization (reference) 5C023 AA18 BA01 BA11 CA03 CA04 CA05 5C063 AA01 AB05 AC01 AC05 AC10 CA29 DA03 DA07 DB10 5D015 DD03 KK02 5D045 AB30

Claims (2)

[Claims] 1. A semi-automatic subtitle program production system comprising a subtitle text transcription function, an automatic subtitle program production function, and a subtitle program editing / screening function, which is a combination of a manual subtitle production function and an automatic subtitle production function. A subtitle data creation / editing support system to be applied, wherein a speech approximation data creating unit for emphasizing a speech component in audio data recorded for a program to be a subtitle program production target, the speech approximation data, A subtitle data creation / editing support system based on speech approximate data, comprising: a display means for displaying the time lapse of a program to be produced as a timeline on a time axis. 2. The speech according to claim 1, wherein the speech approximate data creating means uses a cflx analysis value of a voice, a specific frequency range component extraction value of a voice Power value, or a combination thereof. Subtitle data creation / editing support system using approximate data.