JP2002344805A - Method for controlling subtitles display for open caption - Google Patents

Abstract

PROBLEM TO BE SOLVED: To automatically control the display position and the display contents of display schedule subtitles so as not to cause superimposition when an open caption exists at the display schedule position of the display schedule subtitles which are produced in an automatic subtitles program production system. SOLUTION: An open caption judging part 39 detects the open caption comprised in a video signal to be inputted from a D-VTR 23 through the use of the feature of the open caption which is displayed in a video by superimposition. A character recognizing part 41 converts the open caption detected by the open caption judging part 39 into a text. A screen change judging part 43 detects the changeover point of a screen concerning the video signal to be inputted from the D-VTR 23. A subtitles display position control part 45 performs control to permit the display position of the display schedule subtitles from an integrating device 17 and the display contents not to be superimposed on the open caption.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for automatically controlling the display position and timing of a subtitle to be displayed produced in an automatic subtitle program production system with respect to an open caption.

[0002]

2. Description of the Related Art Today, it is generally referred to as an advanced information society, but hearing impaired persons are in a situation where it is difficult to obtain information as compared with healthy persons. That is, for example, if a TV broadcast program widely used as an information medium is exemplified,
The ratio of subtitle programs to TV broadcast programs is 33
Up to 70%, compared to just 10% in Japan.
At present, it is extremely low at about%.

As described above, the reason why the ratio of subtitle programs to all TV broadcast programs in Japan is lower than that in Europe and the United States is mainly due to the lack of technology for producing subtitle programs. Specifically, there is also a problem unique to Japanese, and most of the subtitle program production processes are manually performed, requiring a great deal of labor, time, and cost.

[0004] The inventors of the present invention have conducted a survey on the current state of subtitle program production in an attempt to find out the cause of hindrance to the development of subtitle program production technology. The left side of FIG. 10 shows a subtitle program production flow that is currently generally performed. On the right side of FIG. 10, an improved current subtitle production flow is shown.

[0005] On the left side of FIG.
Then, a subtitle program maker receives three subtitle manuscript creation materials from a broadcasting station: program data in which a time code is superimposed on a video, a program tape in which the time code is recorded on an audio channel, and a program script. Note that in the drawings, “time code” may be abbreviated as “TC”.

In step S103, an expert, such as a person who has experience in broadcasting, transcribes (1) a summary of a program announcement on the basis of the subtitle manuscript preparation material received in step S101, and (2) presents a separately specified subtitle presentation. The subtitle presentation image is created in accordance with the standard manuscript creation procedure, and (3) the start and end time codes are entered in order to create a subtitle manuscript.

In step S105, the input operator creates digitized caption data based on the caption document created in step S103. In step S107, the digitized subtitle data created in step S105 is previewed and corrected in the presence of the person in charge of the subtitle production cost manager, the manuscript creator, and the input operator, to obtain completed subtitles.

By the way, recently, by training a human resource called a caption operator who has both been able to summarize and transcribe program announcements and digitize subtitles, part of the improved current subtitle production flow shown on the right side of FIG. It has been implemented.

That is, in step S111, a subtitle program creator receives two subtitle manuscript creation materials, a program tape having a time code recorded on an audio channel and a program script, from a broadcasting station.

In step S113, the caption operator reproduces the program tape having the time code recorded on the audio channel. At this time, by clicking a mouse button at the start point of the dialog, the start time code is extracted from the audio channel at that point and recorded. Furthermore, the user listens to the dialogue and inputs it as summary electronic data.
Similarly, when the mouse button is clicked again at the serif point corresponding to the break point based on the subtitle manuscript creation procedure, the end point time code is extracted from the audio channel at that point and recorded. These operations are repeated until the end of the program, and the subtitles of the entire program are digitized.

In step S117, step S105
The subtitle data created in step (1) is previewed and modified in the presence of the caption production charge manager in charge and the caption operator to obtain the completed subtitles.

In the latter improved current subtitle production flow, the caption operator summarizes the dialogue and converts it into electronic data using only the program tape in which the time code is recorded on the audio channel, and divides the dialog into presentation units. By clicking the mouse button at the timing of the dialog corresponding to the start point / end point of the subtitle, each time code of the audio channel is taken out and recorded, which can be said to be an effective subtitle production flow with considerably reduced labor.

In the above-mentioned series of processing in the current subtitle production flow, a particularly large number of man-hours are required because, in step S103 or S105 or step S113, (1) summary transcription of the program announcement,
(2) Subtitle presentation imaging, and (3) start / end time code entry. These work processes largely depend on the knowledge and experience of skilled workers.

[0014] However, among the currently broadcast subtitle programs,
There are some programs in which an announcement manuscript is created in advance, and the manuscript is supposed to be an actual broadcast subtitle with little modification. For example, when actually examining an information program with subtitles called "Travels of the Earth", the announcement sound and subtitle content are almost the same, and the common Hara Hanawa is used for both announcement and subtitle. I can guess.

Therefore, the present inventors have developed an automatic subtitle program production system capable of automating the production of subtitle programs, assuming a form in which a substantially common digitized manuscript is used for both announcements and subtitles. (For example, Japanese Patent Application Laid-Open
-270623).

[0016]

However, when a character displayed superimposed on a television image, that is, a so-called open caption is displayed, a subtitle of a subtitle broadcast to be displayed on the image overlaps with the open caption. , Both are very hard to see.

Therefore, the open caption of the video is checked, and a display position and timing that do not overlap with the open caption are specified at the stage of the subtitle manuscript, or the display position and timing are corrected so as not to overlap at the preview / correction stage. Work must be done to ensure that subtitles do not overlap for open captions. All of these tasks were performed manually by a person with specialized knowledge, and took a lot of time.

The present invention has been made in view of such circumstances, and has been developed in consideration of a form in which a substantially common digitized manuscript is used for both announcements and subtitles. Provided is a subtitle display control method for an open caption that can automatically control a display position and a timing of a display subtitle to be displayed so as not to be duplicated when an open caption is present at a display position of a display subtitle produced by a production system. It is intended to be.

[0019]

According to one aspect of the present invention, there is provided a subtitle display control method for an open caption according to the first aspect of the present invention. When the open caption exists at the display position of the display subtitle produced by the production system, the display position of the display subtitle is determined to be a predetermined position that does not overlap with the open caption in the same frame, or The display position of the subtitle is determined at a predetermined position in a frame where the open caption does not exist.

According to this method, as a part of subtitle production in an automatic subtitle program production system for producing a subtitle program on the assumption that a substantially common digitized manuscript is used for both an announcement and a subtitle, The presence / absence of an open caption that is super-displayed is automatically detected, and the display position and the display timing of the subtitle can be automatically controlled in accordance with the open caption so that the subtitle to be displayed does not overlap with the detected open caption.

According to a second aspect of the present invention, there is provided a subtitle display control method for an open caption, wherein the presence / absence of an open caption superimposed on a television image and the content of the open caption are detected, and a display schedule produced by an automatic subtitle program production system is detected. When the open caption is present at the display position of the subtitle, the similarity between the content of the open caption and the content of the subtitle to be displayed is investigated, and the display of the subtitle to be displayed is performed according to the degree of approximation that has been investigated. / Non-display or correction / non-correction.

According to this method, the open caption is automatically detected as text, and the display / non-display of the subtitle, the correction / non-correction of the subtitle content, and the like are automatically determined by comparing with the subtitle content to be displayed. Can be performed.

According to a third aspect of the present invention, there is provided the subtitle display control method for an open caption according to the first or second aspect, wherein the scene change point of the television image is detected and detected. It is characterized in that the scene change point is compared with the subtitle end timing, and the subtitle end timing is changed based on the comparison result.

According to this method, for example, it is possible to control the start / end timing of the subtitle so as to avoid the cut timing, and to eliminate inappropriate subtitle display such that the video / audio does not match the subtitle. it can.

[0025]

Embodiments of the present invention will be described below in detail with reference to the accompanying drawings.

FIG. 1 is a functional block diagram of an automatic subtitle program production system that implements a subtitle display control method for open captions according to the present invention. FIG. 2 is an explanatory diagram showing a subtitle production flow in the automatic subtitle program production system in comparison with an improved current subtitle production flow. FIG. 3 is a diagram for explaining a division rule applied when dividing a unit subtitle sentence for each display unit subtitle sentence. FIGS. 4 and 5 are diagrams for explaining the technique for detecting the synchronization of the caption transmission timing for the announcement sound.
FIG. 6 is a flowchart illustrating a subtitle display control method for open captions according to Embodiment 1 of the present invention. FIG. 7 is a diagram showing an example of changing the display position of captions and the start / end timing according to the first embodiment.
FIG. 8 is a flowchart illustrating a subtitle display control method for open captions according to Embodiment 2 of the present invention. FIG. 9 is a diagram showing an example of a subtitle display suitability processing according to the second embodiment.

As shown in FIG. 1, the automatic subtitle program production system 11 includes an electronic document recording medium 13, a synchronization detection device 15, an integration device 17, a morphological analysis unit 19, a division rule storage unit 21, , A program material VTR such as a digital video tape recorder (hereinafter, referred to as “D-VTR”) 23, an open caption judging unit 39, a character recognizing unit 41, a screen change judging unit 43, subtitle display position control And a part 45. Among these, the open caption determining unit 39, the character recognizing unit 41, the screen change determining unit 43, and the subtitle display position control unit 45 are functional units that perform subtitle display control according to the present embodiment.

First, an outline of caption production in the automatic caption program production system will be described with reference to FIGS. After that, the subtitle display control according to the present embodiment will be described as the first and second embodiments.

As described above, among the subtitle programs currently being broadcast, the announcement Hara Hanawa is created in advance, and the program whose original is presumed to be the actual broadcast subtitles with little modification is provided. There are several. For example, when actually examining an information program with subtitles called "Travel of the Earth", the announcement sound and subtitle content are almost the same, and it is estimated that almost the same manuscript is used for both the announcement and subtitle it can.

The automatic subtitle program production system proposed by the present inventors is to provide an announcement proposed by the present inventors in an electronic manuscript of a program in which a common manuscript for both announcement and subtitle is digitized. By applying a synchronous detection technology of audio and subtitle text and a text segmentation technology using a Japanese feature analysis method, the display unit subtitle text is synchronized with the progress of the announcement reproduced from the D-VTR 23. Creation and provision of timing information corresponding to each of the start point / end point thereof are automated, so that the production of a subtitle program can be automated without human intervention.

Here, prior to a specific description of caption production, terms used in the description are defined. That is,
The entire set of subtitles to be displayed is called “subtitle sentence text”. In the caption text, a subset of text units delimited by punctuation is called a “unit caption text”. The entire set of display unit subtitles on the display screen of the display is called a “display unit subtitle group”. An arbitrary one-line subtitle in the display unit subtitle group is called a “display unit subtitle sentence”. When expressing an arbitrary character in the display unit subtitle text, it is referred to as “subtitle character”.

In FIG. 1, the digitized original recording medium 13
Is composed of, for example, a hard disk storage device, a floppy (registered trademark) disk device, or the like, and stores a caption sentence text representing an entire set of captions to be displayed. In this case, since it is assumed that almost common digitized original is used for both announcement and subtitle, the contents of the subtitle sentence text stored in the digitized original recording medium 13 are the display target subtitles. Not only D-VTR23
It is assumed that the announcement voice also matches.

The synchronization detecting device 15 includes a display unit subtitle sentence,
It has a function of assisting time synchronization with the announcement voice read out. Specifically, it has a safety verification function, a verification result response function, and a timing information detection function. The safety verification function is a synchronization detection device 1
5 is a function for verifying the validity of the display unit subtitle array every time the display unit subtitle array determined by the integration device 17 is sent. The verification result reply function is a function of replying the verification result to the integrated device 17 when the verification result obtained by performing the validity verification function is invalid. The timing information detection function means that when the verification result obtained by performing the validity verification function is valid,
-With reference to the announcement sound and the time code corresponding to the display unit subtitle array fetched from the VTR 23, the timing information for each corresponding display unit subtitle sentence, that is, the start point / end point time code is detected, and the detected start point / end point time code is detected. This is a function of sending the end point time code to the integrating device 17.

The morphological analysis unit 19 divides the unit subtitle sentence described in the kanji kana mixed sentence into morphemes and divides the morpheme by the morpheme into an expression form and a part of speech. It has an additional information adding function of adding additional information such as reading, standard expression, and the like, and an information element sequence obtaining function of grouping morphemes into units of clauses or clauses and obtaining some information element strings. Thereby, the unit caption sentence is expressed as a surface element sequence, a symbol element sequence (part of speech), a standard element sequence, and an information element sequence.

As shown in FIG. 3, the division rule storage unit 21 stores division rules which are referred to when optimizing a line feed / page break position for a unit caption sentence. DV
TR23 is a program material VTR in which the program material is recorded.
It has a function of reproducing and outputting video, audio, and their time codes from a tape.

The integrating device 17 includes a unit subtitle sentence extracting unit 33
, Display unit subtitle conversion unit 35, timing information addition unit 3
7 are provided. The unit subtitle sentence extraction unit 33 sequentially extracts unit subtitle sentences from, for example, about 40 to 50 subtitle characters from the subtitle sentence text read from the digitized original recording medium 13. Specifically, the unit subtitle sentence extraction unit 3
Reference numeral 3 has a function of sequentially extracting unit subtitle sentences to be displayed which have at least a larger number of characters than the display unit subtitle sentences in the order of display time by utilizing the delimitable portion information and the like as necessary. Note that examples of the delimitable portion information include morphological analysis data with phrase data obtained by the morphological analysis unit 19 and division rules (line feed / page break data) stored in the division rule storage unit 21.

The display unit subtitle unit 35 converts the unit subtitle sentences extracted by the unit subtitle sentence extraction unit 33 into display unit subtitle sentences according to a desired display format. Specifically, the display unit subtitle conversion unit 35 includes the unit subtitle sentence extracted by the unit subtitle sentence extraction unit 33, delimitable portion information added to the unit subtitle sentence, information from the synchronization detection device 15, and the like. The unit subtitle sentence extracted by the unit subtitle sentence extraction unit 33 based on
It has a function of converting into at least one or more display unit subtitle sentences according to a desired display format.

The timing information adding unit 37 converts the display unit subtitle text converted by the display unit subtitle conversion unit 35 into a start point / end point, which is timing information for each display unit subtitle sent from the synchronization detection device 15. Each time code is assigned. Specifically, the timing information adding unit 37
Is timing information for giving each start / end point time code, which is timing information for each display unit subtitle sent from the synchronization detection device 15, to the display unit subtitle sentence converted by the display unit subtitle generator 35. It has an application function.

Next, the subtitle production operation in the automatic subtitle program production system 11 will be described according to the subtitle production flow shown on the right side of FIG. 2 and compared with the improved current subtitle production flow shown on the left side of FIG.

First, the improved current subtitle production flow shown on the left side of FIG. 2 will be described again. Step S111
In, the subtitle program creator receives two subtitle manuscript creation materials, a program tape on which a time code is recorded in an audio channel and a program script, from a broadcasting station. Note that in the drawings, “time code” may be abbreviated as “TC”.

In step S113, the caption operator plays the program tape on which the time code is recorded on another audio channel of the VTR (Rch when the line is Lch), and clicks a mouse button at the start point of the line. The start time code is extracted from the audio channel at that point and recorded. In addition to listening to the dialogue and inputting it as summary electronic data, by clicking the mouse button again at the dialogue point corresponding to the break point based on the subtitle manuscript creation procedure, the end point time code from the audio channel at that point is Remove and record. These operations are repeated until the end of the program, and the subtitles of the entire program are digitized.

In step S117, step S1
The digitized subtitle data created in step 05 is previewed and modified in the presence of the caption production manager in charge and the caption operator to obtain completed subtitles.

As described above, in the improved current subtitle production flow, the caption operator sets the time code to V
Using only the program tape recorded on another audio channel of TR, the dialogue is summarized and converted into electronic data, and the mouse button is clicked at the timing of the dialogue corresponding to the start / end points of the subtitle divided into display units By doing so, each time code of the audio channel is taken out and recorded, thereby realizing a considerably labor-saving and effective subtitle production.

However, according to the subtitle production method according to the present invention, further labor saving is achieved as can be understood from the subtitle production flow shown on the right side of FIG. The processing here is the same as the summary manuscript of step S113 shown on the left side of FIG.
This corresponds to electronic data creation processing.

That is, in step S1, the unit subtitle sentence extraction unit 33 has at least more than the display unit subtitle sentences from the subtitle sentence text read out from the digitized original recording medium 13, using about 40 to 50 characters as a guide. The unit subtitle sentences exhibiting the number of characters are sequentially extracted by utilizing the delimitable portion information and the like. Note that the captions to be produced are usually in the form of subtitles in which two lines of display unit subtitles are sequentially replaced, with a limit of 15 characters per line. Is used as a guide to extract a unit subtitle sentence (this also takes into account the processing amount of 15 characters).

In steps S2 to S5, the display unit subtitle conversion unit 35 determines the unit subtitle based on the unit subtitle sentence extracted by the unit subtitle sentence extraction unit 33 and the delimitable location information added to the unit subtitle sentence. The unit subtitle sentence extracted by the sentence extraction unit 33 is converted into at least one or more display unit subtitle sentences according to a desired display format.

More specifically, the unit subtitle sentence extracted by the unit subtitle sentence extraction unit 33 is converted into a display unit that is a two-line display unit subtitle group with, for example, 13 subtitle characters per line according to the above-described subtitle display format. Create a caption arrangement plan (step S
2). On the other hand, morphological analysis is performed on the unit caption sentence extracted by the unit caption sentence extraction unit 33 to obtain morphological analysis data (step S3). The morphological analysis data also includes data representing a phrase. Then, the line break / page break point of the display unit subtitle arrangement plan is optimized with reference to the morphological analysis data for the display unit subtitle arrangement plan created as described above (step S4), and the display relating to the first unit subtitle sentence is performed. The unit subtitle arrangement is determined (step S5). As a result, it is possible to realize display unit subtitles optimized with high precision in accordance with the actual situation.

In optimizing the display unit subtitle arrangement plan in step S4, a separately prepared division rule (line feed / page break data) is also applied. Specifically, as shown in FIG. 3, the recommended line feed / page break defined by the division rule (line feed / page break data) is first after a period, second after a reading point, and thirdly. Between clauses, fourth
Contains between morpheme parts of speech,
When applying (page break data), the application is preferentially applied from the head of the description order described above. By doing so, it is possible to realize display unit subtitles optimized with high precision in accordance with the actual situation.

In particular, when the fourth morpheme part-of-speech is applied as a division rule (line feed / page break data), the chart shown in FIG. The example of the morpheme part-of-speech and its frequency are shown, but line breaks and page breaks may be performed immediately after the frequent morpheme part-of-speech in the chart of FIG. If you do this,
It is possible to realize display unit subtitles optimized with high precision in accordance with the actual situation.

In step S 6 or S 7, the timing information adding unit 37 compares the timing information for each display unit subtitle sent from the synchronization detection device 15 with the display unit subtitle sentence converted by the display unit subtitle unit 35. The start / end time codes are given.

Specifically, the integrating device 17 gives the display unit subtitle sentence determined in step S5 to the synchronization detecting device 15. On the other hand, in step S6, the synchronization detection device 15
When the announcement sound and its time code are fetched from the VTR 23, the display unit subtitle array determined in step S5, that is, a non-speech section exceeding a predetermined time of, for example, 2 seconds or more in the announcement sound corresponding to the display unit subtitle sentence, that is, the pause The presence or absence is checked (step S7). As a result of this investigation, when the presence of a pause is detected in the announcement sound, the corresponding display unit subtitle sentence is regarded as invalid, and the process returns to the display unit subtitle arrangement determination process in step S5, and the unit subtitle corresponding to before this pause is detected. The display unit subtitle array is reconverted from the sentence.

On the other hand, when the synchronization detecting device 15 detects that there is no pause exceeding a predetermined time as a result of the above-described investigation,
The corresponding display unit subtitle sentence is regarded as valid, and its start / end time code is detected (step S7), and the detected start / end time codes are added to the corresponding display unit subtitle sentence (step S8). ), The process of creating the display unit subtitle sentence relating to the first unit subtitle sentence ends.

Here, the purpose of investigating the presence or absence of a pause in the announcement sound corresponding to the display unit subtitle sentence in step S7 is as follows. That is, the presence of a pause exceeding a predetermined time in the display unit subtitle sentences means that the display unit subtitle sentences are temporally separated and include at least subtitle sentences corresponding to a plurality of different scenes. This is because such caption sentences may not be preferable if these caption sentences are regarded as one display unit caption sentence. As a result, the validity of the display unit caption sentence once determined in step S5 can be re-verified from the viewpoint of the corresponding announcement sound. it can.

The synchronous detection of the start point / end point time code added to the display unit subtitle sentence in step S7 is performed by announcement sound and subtitle sentence text including a speech recognition process for an announcement sound developed and researched by the present inventors. It can be realized with high accuracy by applying the synchronization detection technology between the two. The outline will be described below with reference to FIGS.

That is, the flow of subtitle transmission timing detection is as follows.
As shown in FIG. 4, first, a caption sentence text described in a kana-kanji mixed sentence is converted into a phonetic symbol string using a reading technique used in speech synthesis or the like. The "Japanese reading system" is used for this conversion.

Next, the acoustic model (H
MM: Hidden Markov Model), and converts these phonetic symbol strings into a speech model (HMM) called a word string pair model by a “speech model synthesis system”. Then, by performing comparison and matching through the announcement sound to the word string pair model using the "maximum likelihood matching system",
The subtitle transmission timing is synchronously detected. The algorithm (word string pair model) used for the purpose of detecting caption transmission timing employs a keyword spotting technique. As a keyword spotting technique, a method has been proposed in which a posterior probability of a word is obtained by a forward / backward algorithm, and a local peak of the word likelihood is detected.

As shown in FIG. 5, the word string pair model is applied to a point where it is desired to synchronize subtitles and audio, that is, a word string 1 (Keywords1) and a word string 2 before and after the synchronization point.
(Keywords2). The purpose is to observe the likelihood at the middle point (B) of the word sequence, detect its local peak, and obtain the utterance start time of word sequence 2 with high accuracy. And The word string is formed by concatenation of phoneme HMMs, and the garbage (Garbage) part is composed of all phonemes H.
It is configured as a parallel branch of the MM.

When the announcer reads Hara Hanawa, a pause (Pause) is inserted between word strings 1 and 2 because the position of breathing is arbitrarily determined so that the contents can be easily understood. In addition, regarding the detection of the pause time, the D-VTR
A sound and its time code are supplied from the control unit 23, and the start and end time codes whose sound level is continuous below a designated level can be easily achieved by a known technique. Then, when the creation of the subtitles for the first page is completed, the subtitle sentences from the next page on the first page are extracted, and the process proceeds to the subtitle conversion for the second page. Do.

In the subtitle display control method for open captions according to the present embodiment, the following configuration is used to detect the display position and timing using the characteristics of open captions and display the subtitles so as not to overlap with them. Automatically controlling the position and timing (Embodiment 1), further detecting the open caption as text, and comparing it with the subtitle to be displayed, displaying / non-displaying the subtitle, and correcting / uncorrecting the subtitle content Can be automatically performed (Embodiment 2).

In FIG. 1, an open caption judging section 39 detects an open caption included in a video signal input from the D-VTR 23 by utilizing a characteristic of an open caption superimposed on a video. That is, the open caption usually has the following differences as compared with the background video. (1) The position of the screen to be superimposed is limited. (2) The amplitude level of RGB is large (for example, 85% or more). (3) It is usually still for several seconds or more. (4) The color is usually bright white. (This is an element that determines the occupancy in the display area), (5) the size of the super-displayed character is almost constant (one character is approximately square with a specified size), (6) Usually, a plurality of super-displayed characters are continuous in a horizontal or vertical direction (there may be a plurality of lines). (7) For a specific super-character, a pattern matching method can be applied.

Therefore, by making good use of this difference, it is possible to automatically detect the display timing of the open caption and its contents under certain conditions. Of these differences, by properly combining (1) to (4), the timing can be detected in a considerable number of cases, and in some cases, the accuracy can be further improved by applying (5) to (7).

The character recognition unit 41 converts the open caption detected by the open caption determination unit 39 into text. The screen change determination unit 43 detects a screen switching point in a video signal input from the D-VTR 23. Specifically, the screen change determination unit 43 performs a pan operation,
The start and end of a screen designation operation on which a tilt operation, a zoom-in operation, a zoom-out operation, a wipe operation, a dissolve operation, or the like has been performed are detected.

In the subtitle display position control unit 45, the subtitle (scheduled display subtitle) from the integrating device 17 is displayed by the display position determination unit 4.
7 and display timing determination unit 49 and display / non-display / correction /
It is input to the non-correction determining unit 51. The display position determination unit 47, the display timing determination unit 49, and the display / non-display /
The determination result of the open caption determination unit 39 is input to the correction / non-correction determination unit 51. On the other hand, the text of the open caption recognized by the character recognition unit 41 is input to the display / non-display / correction / non-correction determination unit 51.

The display position determining unit 47, the display timing determining unit 49, and the display / non-display / correction / non-correction determining unit 5
1 is such that a predetermined operation is performed within the screen period specified by the screen change determination unit 43, respectively. The display subtitle data output unit 53 includes a display position determination unit 47.
The display subtitle data determined by the display timing determination unit 49 and the display / non-display / correction / non-correction determination unit 51 are stored in a storage device such as a hard disk or a floppy disk.

When the display position of the subtitle to be displayed overlaps a predetermined area of the screen where the open caption is detected, the display position of the subtitle to be displayed is moved to the same position by moving the display position of the subtitle to be displayed. A position that does not overlap in the frame, and in some cases, a position in another frame where no open caption exists, is determined, and a display subtitle to which the determined position data is attached is passed to the display subtitle data output unit 53.

In the display timing determining section 49, when the display position of the subtitle to be displayed overlaps a predetermined area of the screen where the open caption is detected, the display start / end timing of the subtitle to be displayed does not overlap in the same frame due to a change in the display timing. The position is determined, and the display position of the display scheduled subtitle is provided to the display subtitle data output unit 53 with the display planned subtitle attached with the determined position data.

In the display / non-display / correction / non-correction determination section 51, when the display position of the subtitle to be displayed overlaps a predetermined area of the screen where the open caption is detected, the contents of the open caption and the subtitle to be displayed are displayed. Are compared, it is determined whether or not to display the subtitle, and whether or not to correct the subtitle, the determined content is executed, and the subtitle data based on the execution result is passed to the display subtitle data output unit 53.

Hereinafter, a case where the display position and timing are controlled without using the character recognition unit 41 (Embodiment 1), and a case where display control and the like are performed using the character recognition unit 41 (Embodiment 2). I will explain separately.

(Embodiment 1) When the display position and the timing are controlled without using the character recognizing unit 41, the caption signal of the display unit to be transmitted from now on is determined by using the detection result of the open caption. It controls at what position and at what start / end timing. There are three control methods: (1) only the subtitle position, (2) only the start / end timing, and (3) both the position and the start / end timing, depending on the relationship between the open caption and the subtitle to be displayed. Apply.

(1) In the control of only the subtitle position, the subtitle is moved to a position different from the open caption. This is performed when it is not desired to change the start / end timing of the subtitle to be displayed. The moving position is determined from the position and size of the already detected open caption. Normally, the position of the open caption is fixed, and in that case, it can be a fixed position that does not overlap.

(2) In the control of only the start / end timing, the caption is displayed at the start / end timing different from the open caption. This is performed when it is not desired to change the position of the subtitle to be displayed. The subtitle display start / end timing is determined to be different from the start / end timing of the already detected open caption.

(3) In controlling both the subtitle display position and the start / end timing, the subtitle is displayed at a position and start / end timing different from those of the open caption. This is performed when it is desired to display subtitles more appropriately. The display position and start / end timing of the caption are determined so as to be different from the display position and start / end timing of the already detected open caption.

Generally, subtitle display position control can be performed according to the procedure shown in FIG. 6, in step S61, the open caption determination unit 39 determines whether an open caption exists in a predetermined area on the screen. As described above, the requirement for this determination is that at least the amplitude level and the stationary time have a predetermined value or more.

As a result, when it is determined that there is an open caption (step S62), the display position determining unit 47 and the display timing determining unit 49 determine the position, size, start / start of the detected open caption. By examining the end timing, it is determined whether or not the existing position overlaps the display position of the subtitle to be displayed (step S6).
3).

Next, when the position where the open caption exists and the display position of the subtitle to be displayed overlap, it is determined whether or not the overlapping time is equal to or less than a set value T (second) (step S64). The set value T is, for example, 2 seconds.

If the overlapping time is equal to or longer than the set value T (seconds), the display position determining section 47 searches for another non-overlapping position in the same frame, and if found, determines that position as the display position. (Step S65). Thereby, the display position of the subtitle is determined at a position different from the open caption. The decision result is the display subtitle data output unit 5
Passed to 3.

On the other hand, if the overlapping time is equal to or less than the set value T (seconds), the display timing determining section 49 searches for another nearest frame without an open caption, for example, and determines the display timing (step S66). . Thereby, the display start / end timing of the caption is determined to be different from the start / end timing of the already detected open caption. The determination result is passed to the display subtitle data output unit 53.

The display-captioned subtitles whose display position and timing have been determined in this way are stored in the storage device by the display subtitle data output unit 53 (step S67). If the open caption does not exist at the display position of the caption (step S62) or does not overlap even if it exists (step S63), the display caption to that effect is added to the display caption data output unit 53. Is stored in the storage device (step S67).

Next, with reference to FIG. 7, a description will be given of an example of processing for changing the subtitle display position and the start / end timing performed in the procedure shown in FIG.

In FIG. 7, open caption OP
1, OP2 and subtitles J1, J2, J3 are displayed at the timing shown in the figure, and the display positions of subtitles J2, J3 overlap. If the overlap of the subtitles J2 is, for example, 2 seconds or less, the start / end timing is delayed as in the case of the changed subtitles J2. If the overlap of the subtitles J3 is, for example, 2 seconds or more, the display position is changed like the changed subtitles J3. In this way, overlap with open captions is avoided.

The start / end timing after the change is
Video and audio may become unnatural due to cut changes and scene changes. In addition, an image may become unnatural due to overlap with an important image portion. The processing for them will be described in a second embodiment.

(Embodiment 2) When display control is performed using the character recognition unit 41, in addition to detection of open caption and display control of subtitles, (1) text conversion of open caption, (2) video cut By using the result of the detection, etc., the subtitle display can be controlled accurately. Although slightly overlapping with the above, it will be described again.

(1) Placing Open Caption in Text: When an open caption is detected, it is possible to convert the content of the open caption into text. The detected open caption is a kind of graphic data, but high-performance character / graphic recognition software for converting a character / graphic into text is commercially available, and this technology is applicable.

When the contents of the open caption can be grasped by converting the text, the necessity of displaying the neighboring caption close to the timing of the open caption, deletion of some of the neighboring captions, correction of the timing accompanying the reduction, and the like can be obtained. Enhance subtitle aptitude becomes possible. For example, subtitles whose contents match the open caption need not be displayed and are therefore reduced. In addition, it is possible to reduce the content displayed in the open caption from the corresponding subtitle content.

(2) Control of Subtitle Display Timing by Detecting Video Information: With respect to the timing of subtitle display, information on video to be considered includes cut, wipe, dissolve,
There are pan, tilt, and zoom. In particular, cuts, wipes, and dissolves are frequently used as methods for expressing temporal and spatial separation of video contents. Subtitle display that ignores this gap is often inappropriate. Therefore,
It is necessary to control to avoid such a situation. For example, there is control of start / end timing of subtitles to avoid cut timing.

Next, with reference to FIG. 8, a description will be given of a display control operation when the character recognizing unit 41 is used (an example of appropriate display control according to the contents of the open caption).

First, the processing related to “(1) Converting open caption to text” will be described. Steps that perform the same processing as in FIG. 6 are denoted by the same reference numerals. Here, different parts will be described.

In FIG. 8, when the position where the open caption exists and the display position of the subtitle to be displayed overlap (step S63), the character recognition unit 41 is activated, and the contents of the open caption are identified (step S8).
4). Next, a display / non-display / correction / non-correction determination unit 51
, The content of the open caption is compared with the content of the subtitle to be displayed, and the degree of similarity is investigated. The degree of approximation is determined in two stages (steps S85 and S86).

In step S85, it is determined whether or not the degree of approximation is greater than threshold value 1. The threshold value 1 is, for example, an approximation degree 0.9. If the degree of approximation is high, it is determined that the caption is not displayed (step S87). In step S86, it is determined whether or not the degree of approximation is greater than threshold value 2. The threshold value 2 is, for example, an approximation degree 0.5. If there is a similar part, a decision to delete or correct the similar part is made (step S88).

If the degree of approximation is low, as described with reference to FIG. 6, it is determined whether or not the overlap time is equal to or less than the set value T (step S64). Is controlled (steps S65 and S6).
6). The subtitle to be displayed, for which display / non-display / correction / non-correction is determined in this way, is output to the display subtitle data output unit 5.
3 and stored in the storage device (step S89).

Next, steps S90-S92
The processing related to “(2) Control of subtitle display timing by detecting video information” is shown. Embodiment 1
In FIG. 6 as well, it goes without saying that this “control of subtitle display timing by detecting video information” can be performed.

When the start / end timing after the change becomes unnatural for video / audio due to cut change, scene change, etc., or becomes unnatural for video due to overlap with important video parts, etc. In some cases, when a cut point of the video is detected (step S90), the cutoff timing is compared with the caption end timing (step S91), and the start / end timing of the caption is changed so as not to be unnatural (step S92). The display-captioned subtitles for which the start and end timings are determined to be changed are stored in the storage device by the display subtitle data output unit 53 (step S89).

Next, with reference to FIG. 9, a description will be given of an example of a subtitle display suitability processing performed by the procedure shown in FIG. In FIG. 9, open captions OP1, OP2 and subtitles J1,
When J2, J3, and J4 are displayed at the timing shown in the figure, overlapping occurs at the display positions of captions J2 and J3. The caption J4 straddles the cut point of the video (the point indicated by the black triangle in the figure).

Therefore, the start / end timing of the caption J4 is compared with the cut point of the video, and the start / end timing is set to be before the cut point.
Change the end timing. In this way, unnatural display in relation to video / audio is avoided.

The overlap between the captions J2 and J3 is processed by comparing the captions with the contents of the open captions OP1 and OP2. That is, when the contents of the subtitle J3 and the open caption OP2 are very similar, the display of the subtitle J3 is stopped as shown in the figure. On the other hand, if not approximate, the display position of the caption J3 is changed as described above.

In the example of subtitle J2 and open caption OP1, the contents of subtitle J2 are as follows: "Mori Mori visited the United States yesterday and met with President Bush."
Compared to the contents of No. 1, "Talks with President Bush," the new contents, "Mori visited the United States yesterday." And the modified subtitle J2 'with the start / end timing changed. In this way, overlap with the open caption is properly avoided.

[0097] If an open caption can be detected on the receiving side, further various avoidance measures can be taken.
If the subtitle receiver attaches importance to the subtitles and watches the television while paying particular attention to the subtitles, it is desirable that the subtitles are always displayed at the same position. This is because it is difficult to see if the subtitle display position changes each time. Also, when the corresponding content is displayed as subtitles, the open caption may not be visible.

For this reason, the subtitles are always displayed in a fixed position. If the contents of the subtitles to be displayed include all or most of the contents of the open captions, the open captions are deleted or the level is sufficiently reduced. In the case of, a measure is taken to move and display the open caption to a predetermined position that does not affect the caption display.

For example, in these methods, for erasure, the detected open caption signal is reproduced and the same level is subtracted from the video signal. The level reduction can be realized by subtracting at a predetermined ratio. In addition, the movement of the open caption is based on the method of erasing with the above-described method and synthesizing the image at another position. These techniques are within the technical scope that can be sufficiently realized.

[0100]

As described in detail above, according to the present invention, the display position and timing are automatically detected using the characteristics of open captions, and the subtitles to be displayed are displayed so as not to be duplicated therewith. Since the position and the timing are automatically controlled, the work manually performed can be automated. Also, not only the display position and timing of the open caption, but also the caption is automatically detected as text, and compared with the subtitle content to be displayed, subtitle display / non-display, subtitle content correction / non-correction, etc. Can be automatically determined and executed, so that the efficiency of subtitle production can be improved. Therefore, in closed caption broadcasting in which application fields and the number of programs are expected to expand in the future, such automation is expected to have a great effect on closed caption program production.

[Brief description of the drawings]

FIG. 1 is a functional block configuration diagram of an automatic subtitle program production system that implements a subtitle display control method for open captions according to the present invention.

FIG. 2 is an explanatory diagram showing a subtitle production flow in the automatic subtitle program production system in comparison with an improved current subtitle production flow.

FIG. 3 is a diagram provided for describing a division rule applied when dividing a unit subtitle sentence for each display unit subtitle sentence.

FIG. 4 is a diagram provided for describing a technique for detecting a synchronization of subtitle transmission timing with respect to an announcement sound.

FIG. 5 is a diagram provided for describing a technique for detecting the synchronization of caption transmission timing with respect to an announcement sound.

FIG. 6 is a flowchart illustrating a subtitle display control method for open captions according to Embodiment 1 of the present invention.

FIG. 7 shows a display position and start of a subtitle according to the first embodiment.
It is a figure showing the example of change of the end timing.

FIG. 8 is a flowchart illustrating a subtitle display control method for open captions according to Embodiment 2 of the present invention.

FIG. 9 is a diagram showing an example of a subtitle display suitability processing according to the second embodiment.

FIG. 10 is an explanatory diagram showing a comparison between a current subtitle production flow and an improved current subtitle production flow.

[Explanation of symbols]

Reference Signs List 11 automatic caption program production system 13 digitized original recording medium 15 synchronization detection device 17 integration device 19 morphological analysis unit 21 division rule storage unit 23 digital video tape recorder (D-V
TR) 33 unit subtitle sentence extraction unit 35 display unit subtitle conversion unit 37 timing information addition unit 39 open caption determination unit 41 character recognition unit 43 screen change determination unit 45 subtitle display position control unit 47 display position determination unit 49 display timing determination unit 51 Display / non-display / correction / non-correction determination unit 53 Display subtitle data output unit

 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Eiji Sawamura, Inventor 2-31-19 Shiba, Minato-ku, Tokyo Inside (72) Inventor Takao Kadoma 2-31-19 Shiba, Minato-ku, Tokyo Communication and Broadcasting Organization (72) Inventor Terumasa Ehara 2-31-19 Shiba, Minato-ku, Tokyo Communication and Broadcasting Corporation (72) Inventor Katsuhiko Shirai 2-31-19 Shiba, Minato-ku, Tokyo Communication and Broadcasting Corporation F-term (reference 5C023 AA18 BA01 BA11 BA16 CA01 CA04 CA05 EA03 5C063 AA01 AC01 AC05 CA40 DA03 DB09 EA03 EB01

Claims (3)

[Claims] The present invention detects presence / absence of an open caption super-displayed on a television image, and when the open caption is present at a display position of a display-scheduled subtitle produced by an automatic subtitle program production system, the display-scheduled subtitle is displayed. Determining a display position at a predetermined position that does not overlap with the open caption in the same frame, or determining a display position of the subtitle to be displayed at a predetermined position in a frame where the open caption does not exist; Display control method. 2. The system detects the presence or absence of an open caption super-displayed on a television image and the content of the open caption, and detects the presence of the open caption at a display position of a display subtitle produced by an automatic subtitle program production system. Examining the degree of similarity between the contents of the open caption and the contents of the subtitles to be displayed, and determining display / non-display or modification / non-correction of the subtitles to be displayed according to the degree of approximation examined. A subtitle display control method for open captions. 3. The subtitle display control method for an open caption according to claim 1, wherein a scene change point of the television image is detected, and the detected scene change point is compared with a subtitle end timing. A subtitle display control method for an open caption, wherein a subtitle end timing is changed based on a result.