KR20110110539A - Caption data structure and caption player for synchronizing syllables between a sound source and caption data - Google Patents

Abstract

The present invention relates to a subtitle file structure for synchronizing sound information and subtitles of a sound source in syllable units, and a sound source reproducing apparatus using the same, wherein the subtitle file structure includes subtitles in which subtitles corresponding to the voice information of the sound source are recorded in line units. A data structure, an event for displaying a caption recorded in the caption data structure on a screen, and an event structure for sequentially recording a time point of occurrence of the event. The event recorded in the event structure includes a line display event for displaying a caption line recorded in the caption data structure on a screen, and a syllable inversion event for inverting and displaying the caption displayed on the screen for each syllable according to the line start event. It includes. According to the present invention, by separately storing the caption data and the event data and sequentially reading each event assigned an occurrence time, the audio information of the sound source and the caption can be synchronized in syllable units, and the inversion of the caption through various event designs is possible. Various effects such as color change can be given.

Description

CAPTION DATA STRUCTURE AND CAPTION PLAYER FOR SYNCHRONIZING SYLLABLES BETWEEN A SOUND SOURCE AND CAPTION DATA}

The present invention relates to a caption display method of a sound source playback device such as a karaoke device, an MP3 device, and a media playback device of a personal computer. More specifically, the caption file structure for synchronizing sound information and subtitles of a sound source in syllable units and using the same A subtitle reproducing apparatus.

Efforts for synchronizing the display of voice or music with subtitles have been made in various ways in order to facilitate the user's convenience or to utilize a more efficient playback device. For example, in the conventional caption tape system using a cassette tape, the caption information and the audio information are respectively stored in two tracks of the audio-only tape, and the caption cassette player outputs the audio information to the speaker and the caption information to the LCD screen. The method is being used.

However, in the caption tape method, text information, which is a digital signal, is converted into an analog signal and stored on a tape, whereby a text signal interferes with a voice signal during playback and noise is generated, or a text signal interferes with the text signal, resulting in a text error. There is a problem that voice information is output in mono instead of stereo by generating or storing character information in one track of the tape.

In order to solve the mono output problem of the subtitle cassette, a tape is divided into four tracks or stereo is synthesized by synthesizing a signal. However, when four tracks are used, four tracks for processing each track information in the player are used. In the case of a stereo implementation method using a head or signal synthesis, since the separation of the synthesized signal is incomplete, a loss of the signal may occur when analyzing the signal. There is a problem that noise is generated during playback.

Meanwhile, standard MP3 data (hereinafter referred to as "MP3") data, which has become commonplace today, refers to a moving picture expert group (MPEG) layer 3, which is one of the audio parts of the MPEG technology. Standard MP3 is an audio file format created by compressing existing data using audio data coding technology without compromising sound quality. Such standard MP3 files generally have a recording format of AAU (hereinafter referred to as 'audio decoding unit'). That is, it consists of a header, an optional error check code (CRC), audio data, and auxiliary data.

Therefore, an MP3 player that plays a standard MP3 file is generally used as an audio-only device that receives a compressed audio file and plays it as audio information.

By the way, a liquid crystal display is mounted on a standard MP3 player to display simple character information (single reference content such as a song title) separately from voice information. There is a problem that can not be displayed. In order to solve this problem, some methods of storing subtitle data in a separate memory space and synchronizing the lyrics with the lyrics have been proposed.

For example, FIG. 1 illustrates a subtitle MP3 data structure disclosed in Korean Patent Application No. 10-1999-0000235 (published August 5, 2000, registered May 18, 2001).

As shown in Fig. 1, the conventional subtitle MP3 data structure includes voice information 20 and subtitle information 22, and each subtitle information 22 includes a start part 1, a reproduction number part 2, and an information part. It includes (8). The start part 1 indicates the start of the caption information. The reproduction number section 2 is provided at the next stage of the start section 1, and stores a number indicating how many frames of the voice information 20 are interlocked among the plurality of voice information, thereby storing the voice information at the time of reproduction. Reference numeral 20 and subtitle information 22 are linked to each other.

The information section 8 is provided at the next stage of the reproduction number section 2 and stores information related to the address, type, etc. of the data to be stored, and includes the reproduction address section 3, the information identification section 4, and the selection section. (5) and the caption section 6. The reproduction address section 3 is provided at the next stage of the reproduction number section 2, and indicates a reproduction address where each subtitle information is combined with each other when a plurality of subtitle information is a word or an image. The information identification section 4 is provided at the next stage of the reproduction address section 3 to indicate the file type of the stored information, and the selection section 5 is provided at the next stage of the information identification section 4, so that the language of the stored information is provided. At least any one of a kind, an operation time, and a display system in the liquid crystal output unit 15 is shown. That is, the language type indicates, for example, a language such as Korean (KOR), Japanese (JP), English (USA), and the operation time indicates the time when the subtitle information is output in synchronization with the voice information. The display system displays sequential output or batch output when the subtitles are output in words or sentences on the liquid crystal output unit 15. Finally, the caption section 6 is provided at the next stage of the selection section 5 to store caption characters to be output from the MP3 storage medium.

The caption information group 22a including the plurality of caption information 22 configured in this way is added to the standard MP3 information structure 20a in which a plurality of pieces of audio information 20 which are audio decoding units are added. When the voice information 20 is output from the stored storage medium, subtitle characters are synchronized with each voice signal extracted from the plurality of voice information 20 to be simultaneously outputted.

That is, in the above-described prior art, the caption information includes caption data based on the number data, including number data indicating which frame of each of the plurality of standard MP3 data units the playback number included in the caption data is linked to. The information part displayed or included in the subtitle data includes a time point at which a corresponding subtitle is to be displayed, and a subtitle is displayed at each time point.

However, in the prior art as described above, since the subtitles are displayed only in sentences or measures of lyrics or lines, whether applied to a tape player or an MP3 player, the currently played lyrics or lines are expressed. It cannot indicate which syllable position is in the subtitle.

Listening to music such as songs, learning language, such as English conversation, or in all other cases, if you can provide guided subtitles that can be synchronized by syllable units to the currently playing lyrics or dialogue, the effectiveness of synchronizing subtitles This can be maximized, but there is a problem that can not be implemented by the conventional technology.

SUMMARY OF THE INVENTION In order to solve the above problems of the prior art, an object of the present invention is to provide a subtitle file structure for synchronizing sound information and subtitles of a sound source in syllable units, and a subtitle reproducing apparatus using the same.

It is still another object of the present invention to give various effects such as inversion of subtitles and change of color for synchronizing syllable units of a sound source and subtitles, and to give a display effect to subtitles by reflecting the playing time of each syllable. A subtitle file structure and a subtitle reproducing apparatus using the same are provided.

In order to achieve the above technical problem, according to an aspect of the present invention, a computer-readable recording medium storing a subtitle file structure for reading the event stored separately from the subtitle and synchronizing the voice information of the sound source and the subtitle by syllable units do. The caption file structure includes a caption data structure in which captions corresponding to audio information of the sound source are recorded line by line, an event of displaying captions recorded in the caption data structure on a screen, and a time point of occurrence of the event. Contains the event structure being populated. The event recorded in the event structure includes a line display event for displaying a caption line recorded in the caption data structure on a screen, and a syllable inversion event for inverting and displaying the caption displayed on the screen for each syllable according to the line start event. It includes.

At this time, the cumulative number of syllable inversion events recorded in the event structure preferably matches the number of syllables of the subtitles recorded in the subtitle data structure. In addition, the caption data structure is preferably recorded so that each syllable of the caption can be identified.

In addition, the caption data structure may further include information about the sound source divided by line, and the event recorded in the event structure may further include an information display event for displaying information about the sound source on a screen. . In the event structure, an end time of the event may be additionally recorded for each event of displaying the caption recorded in the caption data structure on the screen, and a countdown event of displaying the remaining time until the occurrence of the line display event. Can be recorded further.

According to a second aspect of the present invention, there is provided a subtitle reproducing apparatus for displaying on the screen by synchronizing the voice information of the sound source and the subtitle by syllable units by reading the event stored separately from the subtitle, and the subtitle file storage unit for storing the subtitle file A memory for storing the subtitles to be displayed on the screen, an event extracting unit for extracting an event list from the subtitle file stored in the subtitle file storage unit, and periodically searching the event list and generating a syllable-inverted event based on a time point of occurrence of each syllable inversion event; And a synchronous timer for inverting a syllable corresponding to a cumulative occurrence number of the syllable reversal event and storing the syllable in the memory, and a screen timer for periodically searching for and displaying a subtitle stored in the memory. The caption file may include a data structure in which a caption corresponding to the audio information of the sound source is recorded, a syllable inversion event in which the caption recorded in the data structure is inverted for each syllable, and an event in which the occurrence time of the event is sequentially recorded. Include structure.

As described above, according to the present invention, by separately storing the caption data and the event data and sequentially reading each event to which the occurrence time is assigned, the audio information of the sound source and the caption can be synchronized in syllable units. In addition, various event designs allow various effects such as subtitle inversion, color change, and countdown, and the inversion rate of syllables is changed to the song speed based on the occurrence time and end time of the event assigned to each syllable. There is an advantage to keep it naturally.

1 is a data structure diagram of a subtitle MP3 according to the prior art.
2 is a block diagram of a caption reproducing apparatus according to a preferred embodiment of the present invention.
3 is a diagram illustrating an event sequentially recorded in an XTF subtitle file and a corresponding screen according to a preferred embodiment of the present invention.
4 is an exemplary view of a caption displayed on a screen by a caption reproducing apparatus according to a preferred embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings and tables.

First, Table 1 defines a data structure of a subtitle file for synchronizing sound information of a sound source and a subtitle in syllable units according to a preferred embodiment of the present invention. Hereinafter, the subtitle file structure according to the preferred embodiment of the present invention is XTF. It will be called. Referring to Table 1, the XTF file is composed of a header section (XTF_HEADER), a data section (XTF_DATA), and an event section (XTF_EVENT).

designation  Explanation XTF_HEADER  Information about the entire file XTF_DATA  Line-breaked string XTF_EVENT  Time (sequence) values and attributes of events (lyric representation)

Table 2 defines the header portion (XTF_HEADER) of the XTF file, "Id" indicating the version information of the subtitle file structure, "Data_size" indicating the size of the above-described data portion (XTF_DATA), and the event portion (XTF_EVENT). It consists of "Event_size", indicating the size of.

designation Size (Byte)  Explanation Id 8  Version information (eg "XTF 1.0") Data_size 2  XTF_DATA size Event_size 2  XTF_EVENT size

Next, Table 3 defines the data portion (XTF_DATA) of the XTF file, where the song number, title, song information, and caption data are sequentially recorded for each line, and each line is identified by a return key (\ r \ n). . In addition, blanks are inserted to clearly distinguish each type of information (eg, song number, title, song information, and the like).

line Separator  Explanation Song_number \ r \ n  Song number is displayed as, for example, # 8084 \ r \ n  blank Subject line 1 \ r \ n  Heading 1 (string) Subject line 2 \ r \ n  Title 2 (String): Used when title 1 exceeds \ r \ n  blank Information line 1 \ r \ n  Song Information 1: Lyrics Information line 2 \ r \ n  Song Information 2: Composition Information line 3 \ r \ n  Song Information 3: Singer \ r \ n  blank Data line [0] \ r \ n  Start caption (lyrics) Data line [1] \ r \ n … \ r \ n Data line [n] \ r \ n  Close caption (lyrics)

Table 4 exemplifies the format of the data line of Table 3, in which case the lyrics (subtitles) to be displayed by inverting them are indicated in parentheses, and at the end of section 1 or 2, the '@' is displayed so that the deemed part proceeds. Make sure you know. In the case of Korean, each syllable displays one syllable, so no separate syllable separator is needed.However, in the case of English using alphabet, syllables can be distinguished by inserting '_' between each syllable in words of two or more syllables. have. If a symbol for distinguishing syllables is not inserted into an English word, one word is recognized as one syllable. Such syllable separators are not displayed on the screen, and are used for identifying syllables at the time of event execution for each syllable as described below.

format symbol  Explanation reversal '(' ')'  Lyrics shown in parentheses are inverted End '@'  '@' At end of line means end of clause Separator '_' English words are divided by syllables
(Eg, "Lo_ve") Lyrics '@'  Last line of lyrics may or may not be appended with '@'

For example, in the case where " (call once) " is recorded in the data portion XTF_DATA of the XTF file, the screen displays " call once. &Quot; Also, in the English case, the lyrics recorded as "(Eve_ry_thing's gon_na)" will be displayed as "Everything's gonna". On the other hand, in the present invention, "inverted" refers to a change in the display state (attribute) of the subtitles collectively, which is not limited to the color inversion of the letters, and various inversion effects (such as font, size, etc.) Note that you can implement effects.

Next, Table 5 defines the event unit (XTF_EVENT) of the XTF file, ID (Id) is assigned for each event type, and the occurrence time of the event identified by the ID is recorded together.

designation Size in bytes  Explanation Id One  Event type time 4  Event occurrence time (in ms)

The event occurrence time is allocated to 4 bytes, the event start time is input in 2 bytes, for example, 1/1000 second, and the event end time is input in the same 2 bytes, so that the duration of the event is maintained. Can be configured for immediate identification. For example, in the event of inverting each syllable of the subtitle, inversion is performed from the start time specified in the event occurrence time to the end time, and the speed at which the inversion of each syllable proceeds from the duration can be calculated in advance.

Meanwhile, the occurrence time of each event may be a time based on a playback time of a sound source, that is, an absolute time, or a time elapsed from a generation time of a previous event, that is, a relative time. When the relative time is input in this way, the first occurrence of the event can be implemented simply by considering the playback time and position synchronization of the sound source.

Next, Table 6 illustrates the Id of the event recorded in the event unit (XTF_EVENT) of the XTF file.

Id  Explanation 0x01  Event occurrence for each syllable of subtitle 0x02  End event for each syllable in a subtitle 0x03  Song information display event occurs 0x04  Ending song information display event 0x05  Start caption sequence 0x06  End caption sequence 0x14  Countdown "4" 0x13  Countdown "3" 0x12  Countdown "2" 0x11  Countdown "1"

As shown in Table 6, Id (0x01) indicates the occurrence of an event for inverting each syllable of the subtitle, and Id (0x02) indicates the time point when the event generated by Id (0x01) described above ends. Accordingly, since the start time and the end time of the event occurrence time of each event are designated as described above, the start time of the event 0x02 can be configured to match the end time of the preceding syllable inversion event 0x01. .

On the other hand, since Id (0x01) represents an inversion event for each syllable, the cumulative number of Id (0x01) included in the event section (XTF_EVENT) of the XTF file is the syllable of the subtitle recorded in the data section (XTF_DATA) of the XTF file. It will match the number, even for Id (0x02). Accordingly, the integrity of the XTF file can be verified by comparing the number of syllables of the subtitle and the number of syllable inversion events.

In addition, Id (0x03) and Id (0x04) are assigned to the occurrence and end of the event for displaying the song information, and the song information of the data portion (XTF_DATA) of the XTF file is displayed on the screen in response to the occurrence of the start event. In response to the occurrence of the end event, song information may be erased on the screen.

Regarding the line-by-line display of the subtitles, an event indicating the start and end of the display of the lyrics sequence is defined as Id (0x05) and Id (0x06), respectively, so that two lines of lyrics are generated at the time (0x05). It can be displayed on the screen, and the lyrics can be deleted from the screen at the time of occurrence of the event (0x06). On the other hand, the screen display of the new lyrics may be performed by a separate event (0x05) or automatically in response to the occurrence of the event (0x06). As an alternative, it may be configured to display a new subtitle line when the inversion of the subtitle line indicated by, for example, two lines is completed by the event (0x05).

On the other hand, when the subtitle file structure according to the present invention is used in karaoke or the like, it is preferable to display countdown information for guiding the start time or the remaining time of the lyrics on the screen. For this purpose, for example, an event (0x11 intrinsic 0x14) for displaying the countdown information from 1 to 4 may be defined, and the occurrence time may be set to be displayed one beat to four beats before the start of the lyrics.

As described above, the caption file structure (XTF) according to the preferred embodiment of the present invention displays a caption data structure in which captions corresponding to sound information of a sound source are recorded line by line, and a caption recorded in the caption data structure is displayed on the screen. An event structure in which an event and a time of occurrence of the event are sequentially recorded is separately created. Accordingly, the caption data structure for recording the caption can be easily created by adding symbols (identifiers) for the end of each section, the inverted portion, the syllable display, and the like to the text file including the lyrics information. In addition, the event structure may be separately created irrespective of the contents of the subtitles. For example, when the sound source is a song, an event occurrence time may be set according to the note length of the song. Also, in the case of countdown information, it can be automatically calculated from the starting point of the subtitle.

FIG. 2 illustrates a subtitle reproducing apparatus 100 for synchronizing subtitles by syllable units using the above-described XTF file, and includes a subtitle file storage unit 110, an event extracting unit 120, and a synchronization unit that store an XTF file. The timer 130, the memory 140 for storing the subtitle to be displayed on the screen, the screen timer 140, and the screen unit 150 is composed of.

First, the subtitle file storage unit 110 stores the previously described XTF subtitle file. That is, as described above, the XTF file stored in the subtitle file storage unit 110 records a subtitle file structure in which subtitles (lyrics) corresponding to voice information of a sound source are recorded, and each event and a time of occurrence of the event are sequentially recorded. Event structure.

The event extractor 120 extracts an event list from the caption file stored in the caption file storage 110. This event list is a double linked-list, which stores pointers to subtitles and events in the XTF file. Accordingly, when the event pointer of the event list is sequentially searched, the events of the XTF file can be searched according to the order of occurrence of the events, and the subtitles of the event list can be accessed to read the subtitles in syllable units or lines. have.

The synchronization timer 130 sequentially searches the event list generated by the event extractor 120, and preferably searches at intervals shorter than the minimum interval between events. From the search result, subtitle processing corresponding to the event is performed in the order of occurrence of each event and stored in the memory 140.

For example, when an event (0x01) that inverts each syllable of the caption is found, the caption inverting the syllable corresponding to the number of occurrences of the event is stored in the memory 140 based on the occurrence time of each syllable inversion event. . In addition, when an event (0x03) for displaying the song information is found, the song information is retrieved from the XTF file stored in the subtitle file storage unit 110 and stored in the memory 140 together with the occurrence time of the corresponding event.

Even when an event (0x05) indicating the start and end of the lyrics sequence is found, as described above, for example, two lines of lyrics are searched and stored in the memory 140 together with the time point of occurrence of the corresponding event. In the case of the countdown event (0x11 to 0x14), the number of the countdown event is stored in the memory 140 without separately searching for the subtitle file structure of the XTF file.

Meanwhile, the display position of the caption may be changed according to the type of the above-described event. For example, the song information such as the title of the song is preferably displayed at the center of the screen, the lyrics information is preferably displayed at the bottom of the screen. To this end, the synchronization timer 130 may be stored in a specific memory address allocated in association with a position where a corresponding subtitle (song title, lyrics, etc.) is displayed according to the type of event.

The memory 140 stores the caption event processed by the synchronization timer 130 until it is displayed on the screen. In this case, it is preferable that the event-processing subtitles are input together with time information indicating the occurrence time of the event, so that the screen timer 140 displays the subtitles on the screen as described below based on the time information.

The screen timer 140 periodically searches for subtitles stored in the memory 140, and reads the subtitles processed by the event at the time of occurrence of the event and displays them on the screen unit 150.

3 illustrates an event sequentially recorded in the above-described XTF file and a screen corresponding thereto.

First, when an event (0x03) occurs, song information such as song name, lyrics, composition, and singer is displayed on the screen during the generation time (0ms to 100ms). Thus, the song information displayed on the screen is read out from the data portion of the XTF file as the event (0x03) is executed. Subsequently, as the event 0x04 is executed, the display of the song information ends at the time 100ms, and the state is maintained until the time 120ms.

Then, an event (0x05) occurs at time (120 ms) to display the lyrics line of two lines on the screen. At this time, the lyrics displayed on the screen is displayed continuously until the time (400ms), it can be configured to change to the next lyrics after the end time.

As described above, while the lyrics are displayed on the screen, the countdown events (0x14 to 0x11) are sequentially executed with a duration of 10 ms, so that the screen additionally displays the countdown information along with the lyrics. At this time, it is preferable to set the lyrics inversion event 0x01 to be performed one beat after the time when the last countdown event 0x11 occurs.

Event 0x01 is then executed at time 170ms, thus inverting the first syllable for a duration of 10ms. Then, the event (0x02) is executed to maintain the complete inversion for the first syllable. The pair of inversion events (0x01 / 0x02) occur sequentially, and the syllable corresponding to the cumulative number of these events is inverted.

4 actually illustrates subtitles displayed on a screen by a subtitle reproducing apparatus according to a preferred embodiment of the present invention.

As shown, the syllables are sequentially inverted and stored in the memory of FIG. 2 according to the duration of the above-described event (0x01), and the screen timer displays them on the screen, from left to right in subtitles shown on the actual screen. This may cause the effect of reversal.

Although the preferred embodiment according to the present invention has been described above, this is merely exemplary and those skilled in the art will understand that various modifications and equivalent other embodiments are possible therefrom. Therefore, the protection scope of the present invention should be defined by the following claims.

100: subtitle playback device 110: subtitle file storage unit
120: event extraction unit 130: synchronous timer
140: memory 150: screen timer
160: the screen

Claims (14)

A computer-readable recording medium having a subtitle file structure for reading an event stored separately from a subtitle and synchronizing sound information of a sound source with a subtitle in syllable units.
A caption data structure in which captions corresponding to sound information of the sound source are recorded line by line;
An event structure in which an event displaying a caption recorded in the caption data structure on a screen and a time point of occurrence of the event are sequentially recorded.
Including;
The event recorded in the event structure,
A line display event for displaying a caption line recorded in the caption data structure on a screen;
And a syllable inversion event for inverting and displaying the subtitles displayed on the screen for each syllable according to the line start event.
Recording medium in which the subtitle file structure is stored. The method of claim 1,
And a cumulative number of syllable inversion events recorded in the event structure matches a syllable number of subtitles recorded in the subtitle data structure. 3. The method of claim 1 or 2, wherein the subtitle data structure is
A recording medium storing a subtitle file structure in which each syllable of a subtitle is recorded to be identified. The method of claim 3,
In the case of an English subtitle, the recording medium storing the subtitle file structure in which an identifier for distinguishing each syllable is additionally recorded. The method according to claim 1 or 2,
The caption data structure is further recorded by dividing the information on the sound source by line unit,
And the event recorded in the event structure further comprises an information display event for displaying information about the sound source on the screen. The method of claim 1 or 2, wherein the event structure
And a caption file structure in which an end time of the event is additionally recorded for each event displaying a caption recorded in the caption data structure on the screen. The method of claim 1 or 2, wherein the event structure
And a countdown event structure for displaying a remaining time until the occurrence of the line display event. 3. The subtitle file structure according to claim 1 or 2, wherein the subtitle file structure is
And a header section for indicating the size of the caption data structure and the event structure. A caption playback device that reads an event stored separately from a caption and synchronizes the audio information of the sound source with the caption in syllable units and displays it on the screen.
A caption file storage for storing a caption file, the caption file comprising: a caption data structure in which captions corresponding to audio information of the sound source are recorded; a syllable inversion event for inverting and displaying the captions recorded in the caption data structure for each syllable; A caption file storage unit including an event structure in which a time point of occurrence of the event is sequentially recorded;
Memory for storing subtitles to be displayed on the screen,
An event extraction unit for extracting an event list from the subtitle file stored in the subtitle file storage unit;
A first timer for periodically retrieving the event list, inverting syllables corresponding to the number of occurrences of the syllable reversal event based on the occurrence time of each syllable reversal event, and storing the syllables in the memory;
A second timer periodically searching for and displaying a subtitle stored in the memory
Subtitle playback device comprising a. 10. The method of claim 9,
Subtitles are recorded on a subtitle data structure of the subtitle file in line units, and a line display event for displaying a subtitle line recorded in the data structure on the screen and an occurrence time of the event are additionally recorded in an event structure of the subtitle file. ,
The first timer is configured to store a caption line recorded in the caption data structure of the caption file in the memory based on a time point of occurrence of the line display event.
Subtitle player. The method of claim 9 or 10,
And a caption data structure of the caption file so that each syllable of the caption can be identified. The method of claim 11, wherein in the case of English subtitles,
And an identifier for distinguishing each syllable is additionally recorded in the caption data structure of the caption file. The method of claim 9 or 10,
In the caption data structure of the caption file, information about the sound source is additionally recorded in line units, and in the event structure of the caption file, an information display event for displaying information about the sound source on a screen and a time point of occurrence of the event The subtitle reproducing apparatus which is additionally recorded. The method of claim 10,
In the event structure of the caption file, a countdown event for displaying the remaining time until the occurrence of the line display event and the occurrence time of the event are additionally recorded.
And the first timer stores a caption line recorded in a data structure of the caption file in the memory based on a time point of occurrence of the line display event.

Images