JP3979566B2 - Time-varying text information segmentation device with moving images - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a time-varying text information dividing apparatus with moving images, and particularly, when dividing content composed of moving images / sounds and character information (time-varying text) that changes in time in synchronization therewith. The present invention relates to a time-varying text information dividing apparatus with moving images that can divide a strange text into a viewer without feeling uncomfortable.
[0002]
[Prior art]
The SMIL language exists as a format for expressing moving images, sound, and time-varying text (that is, telop (product name)). Here, the time-varying text means character information that changes with time. For example, character information that flows from the left to the right of the screen can be assumed.
[0003]
These expression formats are premised on being distributed as they are once created, and a content division technique that divides time-varying text in consideration of the relationship between moving images and audio has been proposed. Absent.
[0004]
Now, in the case of live broadcast, etc., the user who has started viewing from the middle will start receiving from the beginning of the divided data, so that the beginning data must be in a data format that can be decoded alone. There is. For example, in the case of moving image data, the head data must be the head of a key frame (an intra-frame encoded image frame). In the case of audio data, it is necessary to divide at the breaks of audio frames that can be decoded alone. In the case of time-varying text, it is represented by a chunk of data that can be decoded by itself, and the chunk has a temporal spread. For example, when expressing the line of an actor in a drama with time-varying text, the line that the actor speaks with a breath is grouped and the line is displayed only for the time the actor is speaking.
[0005]
[Problems to be solved by the invention]
However, in a receiving apparatus having a limited communication path capacity and storage capacity, such as a mobile phone, it is necessary to download and reproduce the content by subdividing it instead of downloading the entire content. In addition, as described above, in the case of live relay or the like, the user who started viewing from the middle starts reception from the top of the divided data.
[0006]
Therefore, in the above case, if content division is attempted, the division is simply performed by focusing on the key frame of the moving image (the image frame encoded in the frame), that is, dividing at the head of the key frame. Accordingly, there is a problem that the relationship between the moving image and the time-varying text at the time of division becomes indefinite and the viewer feels uncomfortable.
[0007]
More specifically, in the case of dividing by a moving image key frame, since the audio data can be divided at a time interval shorter than one cycle (for example, 1 GOP period) of the moving image key frame, the leading data The difference between the time when the image should be output and the display time of the moving image is sufficiently small. However, in the case of time-varying text, since it can be divided only at a considerably longer time interval than the moving image frame, the time when the time-varying text should be displayed and the display time of the first frame of the moving image are greatly shifted. . For example, even though a moving image of an actor talking is displayed, the dialogue data is attached to the front of the division, the dialogue data behind the division is lost, and the dialogue is displayed It can happen that it is not done.
[0008]
The present invention has been made in view of the above-described prior art, and an object of the present invention is to provide a time-varying text information dividing device with moving images that can divide time-varying text into a viewer without a sense of incongruity.
[0009]
[Means for Solving the Problems]
In order to achieve the above-described object, the present invention divides the time-varying text of content composed of moving images, sounds, and character information that changes in time synchronously (hereinafter referred to as time-varying text). in a variation text information division unit when a moving image bearing, the time-varying text, comprising a dividing means for dividing in synchronism with the head of the key frame of the moving image is divided points, the dividing means, the time of the division point The time-varying text displayed on the screen is extracted, and the time-varying text is expanded into a static time-varying text without a temporal change in a certain time resolution unit, and the start time and end time of the static time-varying text are set. The time-varying text is divided by dividing by the time of the division point .
[0011]
According to the above feature, the time-varying text can be divided in synchronization with the dividing point of the moving image, and the time-varying text can be divided without a sense of discomfort to the viewer.
[0012]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, the present invention will be described in detail with reference to the drawings. FIG. 1 is a block diagram showing a schematic configuration of a moving image reproducing apparatus including the present invention.
[0013]
As shown in the figure, when a content file with time-varying text is input to the audio / video / time-varying text separating unit 1, the separating unit 1 converts the content file with time-varying text into audio data, moving image data, and time. Separate into variable text data. The separated audio data 2a, moving image data 3a, and time-varying text data 4a are input to the division processing unit 5, and when there is a division processing instruction 7, the division processing according to the present invention is performed. If there is no such instruction, it passes as it is and is sent to the voice / video / time-varying text multiplexing unit 6. The multiplexing unit 6 multiplexes audio / video / time-varying text and outputs it in the same format as the original content file with time-varying text.
[0014]
FIG. 2 is a conceptual diagram of the audio data 2a, the moving image data 3a, and the time-varying text data 4a.
[0015]
As shown in the figure, the audio data 2a is composed of encoded frames A1, A2, A3,.
[0016]
The moving image data 3a is composed of an image sequence, and key frames (image frames encoded within the frame) are periodically inserted into the image sequence. 11 to 14 in the figure indicate the heads of the key frames. A period K from one key frame to the next key frame indicates a key frame period (1 GOP period). Note that the key frame is inserted in a moving picture coding apparatus (not shown).
[0017]
Further, the time-varying text data 4a is composed of a group of data (for example, a group of lines of data) T1, T2, T3,. As is apparent from the figure, it is clear that the time-varying text data chunks T1, T2,... Have a larger temporal spread than moving image data and audio data. In addition, since the time-varying text is inserted according to the meaning content of the moving image (for example, a drama dialogue, etc.), the break of the time-varying text data is not necessarily the key frame position 11, 12, 13,. Does not match.
[0018]
In the time-varying text, various modifications are applied to the character information. The modifications include (1) those with temporal behavior and (2) those without. Things with temporal behavior include scrolling, blinking, and wiping. Scrolling is a behavior that causes a character string to flow, blinking is a behavior that the character string blinks, and wipe is a behavior that the character color in the character string changes (for example, lyrics of karaoke). On the other hand, there are a character color, a background color, an underline, etc. that do not involve temporal behavior.
[0019]
Next, the configuration and operation of the division processing unit 5 that is the main part of the present invention will be described. FIG. 3 is a block diagram showing a configuration of the time-varying text information dividing device with moving images in the division processing unit 5. The moving image-attached time-varying text information dividing device 20 includes a divided time-varying text searcher 21 and a character modification divider 22.
[0020]
When the division instruction signal 7 is input at an arbitrary time T ′, the division processing unit 5 converts the audio data 2a, the moving image data 3a, and the time-varying text data 4a at the division point T shown in FIG. To divide. That is, since the data at the dividing point of the moving image data 3a needs to be in a data format that can be decoded by itself, it is divided so that the head data becomes a key frame (an intra-frame encoded image frame). There must be. For this reason, the dividing point T is aligned with the head of the key frame.
[0021]
The audio data 2a is divided at the boundary between the encoded frames A1 and A2 because the division point T comes in the middle of the encoded frame A2, and A1 is stored in the file before division, and A2 is stored in the file after division. .
[0022]
Next, with respect to the time-varying text data 4a, for example, as shown in FIG. 2, when the time-varying text data 4a is divided into two by the data chunk T2, consistency is maintained before and after the division. It will be necessary to supplement so that it may lean. In addition, it is necessary to modify the character information so that the character information is not lost even when reception is started for the first time from the division point while maintaining continuity before and after the division. Therefore, in this embodiment, the division process is performed as follows.
[0023]
The time-varying text information dividing device 20 with a moving image includes a divided time-varying text searcher 21 for searching for a time-varying text to be divided, and a character modification divider 22 for dividing the time-varying text and its modification. .
[0024]
First, the operation of the divided time-varying text searcher 21 will be described with reference to the flowchart of FIG. As a premise, the division time is T, the start times of the chunks T1, T2,... Of the time-varying text data 4a are ti, and the end times are ti ′. In step S1, the number i representing the block of time-varying text data is set to 0, and in step S2, the block Ti of the time-varying text data 4a is read. In step S3, it is determined whether or not the division time T satisfies ti <T <ti '. When this determination is negative, the process proceeds to step S4, where it is determined whether all time-varying text data has been determined. When this determination is negative, the process proceeds to step S5, and 1 is added to i. And it returns to step S2 and the above-mentioned process is repeated. When the determination in step S3 is affirmative, the data chunk Ti is a division target, so that the process proceeds to step S6 and is held as a time-varying text to be processed.
[0025]
Next, the operation of the character modification divider 22 will be described. The operation procedure of the character modification divider 22 is as follows.
(1) Expand time-varying text with time behavior into time-varying text without time-varying behavior (hereinafter referred to as static time-varying text).
(2) The static time-varying text at the division time is divided into two at the division time T.
[0026]
The procedures (1) and (2) will be specifically described. In (1), the state of the time-varying text is seen at a certain time resolution, and the static time-varying text in that state is sequentially output.
[0027]
Now, as time-varying text data T2, as shown in FIG. 5, for example, it is assumed that “a major incident has occurred in New York”. Assuming that the time-varying text data T2 is displayed in five characters and moves by one character per second, as shown in the figure, “scroll from 0 to 12 seconds, 1 character / second, the window has 5 characters, A major incident occurred in New York. ” Phenomenologically, as shown in the figure, it can be considered that the window W through which five characters pass moves in the direction of the arrow a every second. Note that the time resolution is determined by, for example, ½ the blink cycle in the case of blink, and one character movement speed of wipe in the case of wipe.
[0028]
When this is expanded into the static time-varying text of (1), as shown in FIG. 5, the second second “New York”, the first second “New York”, the second second “Yoke”,・ It becomes the 12th "I was born". In this case, the time resolution is 1 second.
[0029]
Next, in (2), a process of dividing the static time-varying text into two at the division time T is performed.
[0030]
If the static time-varying text of the data chunk T2 is represented as shown in FIG. 6, the division point T corresponds to somewhere in the static time-varying text. Assuming that the division point T corresponds to the 4th second “Cake important”, in the process (2), first, an operation for searching for the static time-varying text to which the division point T belongs is performed. As a result of the search, if, for example, the 4th second “Cheke is important” is detected as the static time-varying text, the fourth second “Cheke is important” is then divided at the dividing point T. . The process for searching for the static time-varying text to which the division point T belongs can employ the same process as in FIG.
[0031]
This process is a process in which the division point T is the end time of the time-varying text before division, and the division point T is the start time of the time-varying text after division. In the above example, the time 4 to T of the time-varying text “4 to 5 seconds” of 4 to 5 seconds is the end time-varying text of the time-varying text before the division, and the time T to 5 is the time-varying text of the divided time-varying text. Process to change the start time text.
[0032]
Through the above division processing, the time-varying text data 4a is complemented so that consistency is maintained before and after the division, and the modification of the character information is received from the division point while maintaining the continuity before and after the division. Even if it is started for the first time, character information can be prevented from being lost. In addition, it is possible to match with the key frame of the moving image at the division point T.
[0033]
【The invention's effect】
As is apparent from the above description, according to the present invention, time-varying data can be divided at an arbitrary time in synchronization with moving image data and audio data. For this reason, the time-varying text can be divided without a sense of incongruity to the viewer.
[0034]
Even in a receiving device that has limited channel capacity and storage capacity, such as a mobile phone, even when content is segmented and downloaded and played back, the time-varying text at the dividing point can be viewed by the viewer. It will be possible to provide without discomfort.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a schematic configuration of a playback apparatus to which the present invention is applied.
FIG. 2 is a conceptual diagram of audio data, moving image data, and time-varying text data.
FIG. 3 is a block diagram illustrating a configuration of a time-varying text information dividing apparatus with moving images according to an embodiment of the present invention.
4 is a flowchart showing the operation of the divided time-varying text searcher of FIG.
FIG. 5 is an explanatory diagram of time-varying text and static time-varying text.
6 is an explanatory diagram of an operation of the character modification divider shown in FIG. 3. FIG.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 ... File classification part, 2 ... Voice decoding part, 3 ... Moving image decoding part, 4 ... Time-variant text decoding part, 5 ... Division processing part, 6 ... Image display part 20 ... time-varying text information dividing device with moving image, 21 ... divided time-varying text searcher, 22 ... character modification divider.

Claims (2)

In a time-varying text information dividing device with moving images, which divides the time-varying text of content composed of moving images, audio, and character information that changes in time synchronously (hereinafter referred to as time-varying text).
Dividing means for dividing the time-varying text in synchronization with the beginning of a key frame of a moving image that is a dividing point ;
The dividing means extracts time-varying text displayed at the time of the dividing point, expands the time-varying text into static time-varying text that does not involve temporal change in a certain time resolution unit, and A time-varying text information dividing apparatus with moving images , wherein the time-varying text is divided by dividing a start time and an end time of the time-varying text by the time of the division point .   The time-varying text information dividing device according to claim 1, wherein the time-varying text is subjected to at least one of a modification with temporal behavior and a modification without temporal behavior.

Images