JP2022098735A - Subtitle generation device and subtitle generation program - Google Patents

Abstract

To provide a subtitle generation device capable of generating a subtitle file capable of appropriately displaying subtitle text as a subtitle file for distributing subtitle data in real time via a telecommunication line.SOLUTION: A subtitle generation device includes a subtitle extraction unit that extracts a subtitle text from subtitle data acquired from the outside and extracts a first display start time of the subtitle text, an end time setting unit that sets a first display end time after the first display start time, a split subtitle generation unit that generates split subtitle data in which the first display start time and the first display end time are associated with the subtitle text, a data output unit that outputs the generated split subtitle data, and a start time setting unit that sets the first display end time to the second display start time of the subtitle text. The end time setting unit sets the second display end time after the second display start time. The split subtitle generation unit generates split subtitle data in which the second display start time and the second display end time are associated with the subtitle text.SELECTED DRAWING: Figure 1

Description

The present disclosure relates to a subtitle generator and a subtitle generator.

Video data for displaying video and subtitle data for displaying subtitles are obtained from the broadcast wave of a program including video and subtitles, and the video related to the obtained video data and the subtitles related to the obtained subtitle data are obtained. There is known a video processing device that outputs a video signal for displaying together (see, for example, Patent Document 1). In particular, in Patent Document 1, if the addition result of the number of characters of the subtitle displayed in the predetermined display area and the number of characters of the subtitle of the subtitle data obtained next is the predetermined number of characters or less, the next subtitles are continuously arranged. It describes that the subtitles should be displayed continuously, and that the display time for which the subtitles should be displayed is extracted from the broadcast wave and the display time is extended according to the number of characters in the subtitles.

Japanese Unexamined Patent Publication No. 2010-268076

As described above, the technique as shown in Patent Document 1 extracts video data, subtitle data, and display time from the broadcast wave of a program including video and subtitles. Here, consider a case where the subtitle data receives the data superimposed on the video data and outputs the subtitle data as a WebVTT format subtitle file in real time. In the WebVTT format subtitle file, the display start time and the display end time are specified for the subtitle text. If the subtitle text in which only the display start time is specified and the display end time is not specified is included in the subtitle file in WebVTT format, the subtitle text is ignored by a general player, and the subtitle text is displayed. Do not show. Therefore, for example, when broadcasting data is encoded and converted in real time and distributed over a telecommunication line such as the Internet, the subtitle text is displayed because the next subtitle data is not received when a certain subtitle data is received. The end time cannot be determined, the subtitle text cannot be displayed properly in the subtitle file generated in real time at the same time as the subtitle data is received, and the video data and the subtitle data cannot be synchronized.

This disclosure is made to solve such a problem. The purpose is to synchronize the video data and the subtitle data when the subtitle data receives the data superimposed on the video data and outputs the subtitle data as a subtitle file for distribution on the telecommunications line in real time. It is an object of the present invention to provide a subtitle generator and a subtitle generator capable of generating a subtitle file capable of appropriately displaying text.

The subtitle generator according to the present disclosure has a subtitle extraction unit that extracts the subtitle text and the first display start time of the subtitle text from the subtitle data acquired from the outside, and a time after the first display start time. Generates divided subtitle data in which the first display start time and the first display end time are associated with the subtitle text extracted by the end time setting unit for setting a certain first display end time and the subtitle extraction unit. A divided subtitle generation unit, a data output unit that outputs the divided subtitle data generated by the divided subtitle generation unit, a start time setting unit that sets the first display end time to the second display start time of the subtitle text, and a start time setting unit. The end time setting unit sets the second display end time, which is a time after the second display start time, and the divided subtitle generation unit sets the subtitle text extracted by the subtitle extraction unit. The divided subtitle data in which the second display start time and the second display end time are associated with each other is generated.

In the subtitle generation program according to the present disclosure, the computer of the subtitle generator has a subtitle extraction unit that extracts the subtitle text and the first display start time of the subtitle text from the subtitle data acquired from the outside, and the first display start. The end time setting unit that sets the first display end time, which is a time after the time, and the subtitle text extracted by the subtitle extraction unit are associated with the first display start time and the first display end time. The divided subtitle generation unit that generates the divided subtitle data, the data output unit that outputs the divided subtitle data generated by the divided subtitle generation unit, and the first display end time are set as the second display start time of the subtitle text. In addition to functioning as a start time setting unit, the end time setting unit is made to set a second display end time which is a time after the second display start time, and the divided subtitle generation unit is made to set the subtitle. The subtitle text extracted by the extraction unit is made to generate divided subtitle data in which the second display start time and the second display end time are associated with each other.

According to the subtitle generation device and the subtitle generation program according to the present disclosure, when the subtitle data receives the data superimposed with the video data and outputs the subtitle data as a subtitle file for distribution on a telecommunications line in real time. It has the effect that it is possible to synchronize the video data and the subtitle data to generate a subtitle file that can appropriately display the subtitle text.

It is a block diagram which shows the functional structure of the subtitle generation apparatus which concerns on Embodiment 1. FIG. It is a figure explaining an example of the subtitle file of the WebVTT format. It is a figure explaining an example of the subtitle file of the WebVTT format generated by the subtitle generation apparatus which concerns on Embodiment 1. FIG. It is a figure explaining an example of the subtitle file of the WebVTT format generated by the subtitle generation apparatus which concerns on Embodiment 1. FIG. It is a flow diagram which shows the processing example of the subtitle generation apparatus which concerns on Embodiment 1. FIG. It is a flow diagram which shows the processing example of the subtitle generation apparatus which concerns on Embodiment 1. FIG. It is a flow diagram which shows the processing example of the subtitle generation apparatus which concerns on Embodiment 1. FIG.

A mode for implementing the subtitle generation device and the subtitle generation program according to the present disclosure will be described with reference to the attached drawings. In each figure, the same or corresponding parts are designated by the same reference numerals, and duplicate description will be appropriately simplified or omitted. In the following description, for convenience, the positional relationship of each structure may be expressed with reference to the illustrated state. It should be noted that the present disclosure is not limited to the following embodiments, and is free combination of each embodiment, modification of any component of each embodiment, or each of them, as long as the purpose of the present disclosure is not deviated. It is possible to omit any component of the embodiment.

Embodiment 1.
The first embodiment of the present disclosure will be described with reference to FIGS. 1 to 7. FIG. 1 is a block diagram showing a functional configuration of a subtitle generator. FIG. 2 is a diagram illustrating an example of a subtitle file in WebVTT format. 3 and 4 are diagrams illustrating an example of a WebVTT format subtitle file generated by the subtitle generator. 5 to 7 are flow charts showing processing examples of the subtitle generator.

As shown in FIG. 1, the subtitle generation device 10 according to this embodiment includes a subtitle extraction unit 11, an end time setting unit 12, a start time setting unit 13, a divided subtitle generation unit 14, and a data output unit 15. .. Each of these parts is realized using electronic circuits and processes electrical signals that represent information.

The subtitle generator 10 may be composed of one or more computers including a processor and a memory as hardware. The processor is also referred to as a CPU (Central Processing Unit), a central processing unit, a processing unit, an arithmetic unit, a microprocessor, a microcomputer, or a DSP. Examples of the memory include non-volatile or volatile semiconductor memories such as RAM, ROM, flash memory, EPROM and EEPROM, or magnetic disks, flexible disks, optical disks, compact disks, mini disks and DVDs.

A program as software is stored in the memory of the subtitle generation device 10. Then, the subtitle generation device 10 executes preset processing by executing a program stored in the memory by the processor, and as a result of the cooperation between the hardware and the software, the functions of the respective parts described below are performed. Realize. That is, the program stored in the memory of the subtitle generation device 10 is a subtitle generation program that causes the computer of the subtitle generation device 10 to function as each part described below.

The subtitle extraction unit 11 takes in a broadcast signal input from the outside and extracts subtitle data from the acquired broadcast signal. The broadcast signal is transmitted to the subtitle generation device 10 by, for example, SDI (Serial Digital Interface). SDI is a standard interface used in broadcasting equipment. The format of the broadcast signal is based on the standard established by ARIB (Association of Radio Industries and Businesses: Association of Radio Industries and Businesses). The subtitle data is also superimposed on the input broadcast signal in accordance with the ARIB rules. The subtitle data is stored in the vertical blanking area of HD-SDI or SD-SDI, and the subtitle extraction unit 11 extracts the subtitle data. The subtitle data may be stored in another area of the broadcast signal.

The interface of the broadcast signal input to the subtitle generation device 10 is not limited to SDI. In addition, for example, the broadcast signal input to the subtitle generator 10 may be transmitted on an IP (Internet Protocol) network using RTP (Real-time Transport Protocol) or the like. When RTP is used, the video and audio data included in the broadcast signal are encoded in the MPEG2-TS (Moving Picture Experts Group2-Transport Stream) format using, for example, a real-time encoder or the like. Further, even when RTP is used, subtitle data specified by, for example, ARIB is superimposed on the broadcast signal. In this case, the subtitle extraction unit 11 extracts the subtitle data from the MPEG2-TS elementary stream.

In this way, the subtitle extraction unit 11 acquires the subtitle data from the outside. Then, the subtitle extraction unit 11 extracts the subtitle text from the acquired subtitle data. The subtitle data also includes information that specifies when to start displaying the subtitle text. The subtitle extraction unit 11 extracts from the acquired subtitle data information that specifies the timing for starting the display of the subtitle text as the first display start time of the subtitle text. In other words, the subtitle extraction unit 11 extracts the first display start time of the subtitle text from the acquired subtitle data.

The end time setting unit 12 sets the first display end time for the subtitle text extracted by the subtitle extraction unit 11. The first display end time is a time after the first display start time extracted by the subtitle extraction unit 11. The divided subtitle generation unit 14 corresponds to the subtitle text extracted by the subtitle extraction unit 11 with the first display start time extracted by the subtitle extraction unit 11 and the first display end time set by the end time setting unit 12. Generate the attached split subtitle data.

The start time setting unit 13 sets the second display end time for the subtitle text extracted by the subtitle extraction unit 11. The second display end time is the same time as the first display end time set by the end time setting unit 12. The end time setting unit 12 sets the second display end time for the subtitle text extracted by the subtitle extraction unit 11. The second display end time is a time after the second display start time set by the start time setting unit 13. Then, the divided subtitle generation unit 14 has a second display start time set by the start time setting unit 13 and a second display end time set by the end time setting unit 12 in the subtitle text extracted by the subtitle extraction unit 11. Generates divided subtitle data associated with.

In this way, the divided subtitle generation unit 14 displays the display time of the subtitle text extracted by the subtitle extraction unit 11 from the first display start time to the first display end time and from the second display start time to the second display. Generates divided subtitle data divided up to the end time. The first display end time and the second display start time are the same time. Therefore, the subtitle data is divided so that the display time of the subtitle text is not interrupted. After the second display end time, the division is continued in the same manner so that the display time of the subtitle text is not interrupted. That is, the display end time of the previous divided subtitle data and the display start time of the divided subtitle data immediately after that are the same time.

The end time setting unit 12 sets the time interval between the first display start time and the first display end time, for example, based on the time measurement result of the timer unit 16 included in the subtitle generation device 10. Similarly, the end time setting unit 12 sets the time interval between the second display start time and the second display end time based on, for example, the timing result of the timer unit 16. The end time setting unit 12 sets the first display end time and the interval from the second display start time to the second display end time so that the interval from the first display start time to the first display end time becomes equal. Set the second display end time.

In this case, for example, the subtitle data is superimposed on each display time, that is, the interval from the first display start time to the first display end time and the interval from the second display start time to the second display end time. It may be determined in consideration of the video data encoding delay, the video data decoding delay by the player, the drawing delay of the video data by the player, and the like. By doing so, it is possible to easily match the display timings of the video data and the subtitle data.

The data output unit 15 outputs the divided subtitle data generated by the divided subtitle generation unit 14. That is, the data output unit 15 outputs the divided subtitle data in which the first display start time and the first display end time are associated with the subtitle text. Further, the data output unit 15 outputs the divided subtitle data in which the second display start time and the second display end time are associated with the subtitle text. Similarly, after the second display end time, if there is the divided subtitle data generated by the divided subtitle generation unit 14, the data output unit 15 outputs the divided subtitle data.

The file format of the output data at this time is, for example, a WebVTT (Web Video Text Track) format. Next, the structure of the WebVTT format subtitle file will be described with reference to FIG. The figure shows an example of a subtitle file in WebVTT format. "WEBVTT" in the first line is header information, and indicates that this file is a WEBVTT format file. The second line is a blank line.

The data in the third line and the data in the fourth line are a set. The third line is the display start time and display end time of the subtitle text. The fourth line is the content of the subtitle text displayed between the display start time and the display end time of the third line. Specifically, "00:00:05.000" on the line head side of "->" on the third line indicates that the display start time is 0:00:05.000. Further, "00: 00: 10.000" on the end of the line from "->" on the third line indicates that the display end time is 0:00:10.000. It should be noted that these times are relative, and are based on, for example, the reproduction time of the video in which the subtitle data is displayed. The fourth line, "Today is sunny." Is a subtitle text to be displayed between 0:00:05 and 0:00 to 0:00:10.000.

Similarly, the data is a set of the sixth line and the seventh line with the blank line of the fifth line in between. On the 6th and 7th lines, the subtitle text "Tomorrow's weather will be cloudy." Is displayed between the display start time of 0:00:11: 000 and the display end time of 0:00:16: 000. It is shown that. Further, the data is a set of the 9th line and the 10th line with the blank line of the 8th line sandwiched between them. The 9th and 10th lines indicate that the subtitle text "♪ (theme song)" is displayed between the display start time of 0:00:20: 000 and the display end time of 0:01:20: 000. ing. Then, the data is a set of the twelfth line and the thirteenth line with the blank line of the eleventh line sandwiched between them. On the 12th and 13th lines, the subtitle text "Now, the next news." Is displayed between the display start time of 0:01:22: 000 and the display end time of 0:01:25:000. Is shown.

The data output unit 15 may output two or more divided subtitle data as one subtitle file, or may output each of the two or more divided subtitle data as a separate subtitle file. When outputting two or more divided subtitle data as one subtitle file, the data output unit 15 converts the subtitle text into the divided subtitle data in which the first display start time and the first display end time are associated with each other, and the subtitle text. Outputs one subtitle file including at least the divided subtitle data in which the second display start time and the second display end time are associated with each other.

FIG. 3 shows an example in which the data output unit 15 outputs two or more divided subtitle data as one subtitle file. The example in the figure is a part of a subtitle file in WebVTT format. In this example, first, the subtitle text "Tomorrow's weather will be cloudy." Is displayed between the display start time of 0:00:11: 000 and the display end time of 0:00:16: 000. Then, in the range shown in the figure, the subtitle text "♪ (theme song)" is divided into five divided subtitle data every 5 seconds.

That is, first, in the first divided subtitle data, the subtitle text "♪ (theme song)" is displayed between the display start time of 0:00:20: 000 and the display end time of 0:00:25: 000. Next, in the second divided subtitle data, the same subtitle text "♪ (theme song)" is displayed between the display start time of 0:00:25: 000 and the display end time of 0:00:30: 000. do. Further, in the third divided subtitle data, the same subtitle text "♪ (theme song)" is displayed between the display start time of 0:00:30: 000 and the display end time of 0:00:35:000. .. Further, in the fourth divided subtitle data, the same subtitle text "♪ (theme song)" is displayed between the display start time of 0:00:35:000 and the display end time of 0:00:40.000. .. Then, in the fifth divided subtitle data, the same subtitle text "♪ (theme song)" is displayed between the display start time of 0:00:40:000 and the display end time of 0:00:45:000. .. Similarly, after 0:00:45, 000, the divided subtitle data is generated every 5 seconds of the display time.

FIG. 4 shows an example in which one WebVTT format subtitle file shown in FIG. 3 is divided into a plurality of WebVTT format subtitle files. In this example, the subtitle file is divided according to the moving image distributed by the HLS (HTTP Live Streaming) standard. EXT-INF is a tag that specifies the length of the HLS segment. In the illustrated example, EXT-INF is 10 seconds. One WebVTT format subtitle file shown in FIG. 3 is divided into a plurality of subtitle files according to the length of the HLS segment. In the example of FIG. 3, the display time of one divided subtitle data is 5 seconds. Therefore, the file is divided into one WebVTT format subtitle file for each of the two divided subtitle data according to the length of the HLS segment of 10 seconds.

Specifically, the first subtitle file displays the subtitle text "♪ (theme song)" between the display start time of 0:00:20: 000 and the display end time of 0:00:25: 000. From the divided subtitle data of, and the second divided subtitle data that displays the subtitle text "♪ (theme song)" between the display start time of 0:00:25:000 and the display end time of 0:00:30:000. Become. In addition, the second subtitle file is the third split subtitle that displays the subtitle text "♪ (theme song)" between the display start time of 0:00:30: 000 and the display end time of 0:00:35:000. It consists of data and a fourth divided subtitle data for displaying the subtitle text "♪ (theme song)" between the display start time of 0:00:35 and 000 and the display end time of 0:00:40.000. The third subtitle file is the fifth split subtitle that displays the subtitle text "♪ (theme song)" between the display start time of 0:00:40 and 000 and the display end time of 0:00:45.000. It consists of data and the sixth divided subtitle data for displaying the subtitle text "♪ (theme song)" between the display start time of 0:00:45:000 and the display end time of 0:00:50.000.

In the illustrated example, each subtitle file is given a file name including a serial number. Specifically, each is webvtt_1. vtt, webvtt_3. vtt and webvtt_4. It is vtt.

As illustrated in FIG. 2, in the WebVTT format subtitle file, the display start time and the display end time are specified for the subtitle text. Here, if the subtitle text in which only the display start time is specified and the display end time is not specified is included in the subtitle file in the WebVTT format, the general player ignores the subtitle text and the subject is concerned. Subtitle text is not displayed.

Here, in the case where the subtitle data is superimposed on the video data and transmitted, consider a case where these data are received and the subtitle data is output as a WebVTT format subtitle file in real time. In such a case, in the prior art, when a certain subtitle data is received, the display end time of the subtitle text may not be determined because the next subtitle data is not received. Therefore, in the WebVTT format subtitle file generated in real time at the same time as receiving the subtitle data, the subtitle text whose display end time is not specified is included, and the subtitle text cannot be displayed properly, and the video data and the subtitle Data may not be synchronized.

On the other hand, according to the subtitle generation device 10 according to the present disclosure configured as described above, the display end time is set for the subtitle text whose display end time cannot be determined when the subtitle data is received, and further, the subtitle text is set. For, the divided subtitle data is generated so that the display is restarted and the display is continued even after the set display end time. Therefore, even a subtitle text whose display end time cannot be determined when the subtitle data is received can be output as a WebVTT format subtitle file in which the display end time is specified. Therefore, in the case where the subtitle data is superimposed on the video data and transmitted, when receiving these data and outputting the subtitle data as a WebVTT format subtitle file for distribution on a telecommunications line such as the Internet in real time, It is possible to synchronize the video data and the subtitle data so that the subtitle text can be displayed appropriately.

Further, by outputting two or more divided subtitle data as one subtitle file, it can be easily divided into a plurality of subtitle files according to the subsequent needs and the like. Therefore, for example, in the case of live distribution in the HLS format, a segment file of subtitle data can be easily generated according to the HLS segment length of the video data.

Next, an example of the processing flow of the subtitle generation device 10 configured as described above will be described with reference to the flow chart of FIG. First, in step S11, the subtitle extraction unit 11 acquires the subtitle data from the outside. Then, the subtitle extraction unit 11 extracts the subtitle text from the acquired subtitle data. Further, the subtitle extraction unit 11 extracts the time when the subtitle text should be displayed from the acquired subtitle data, and sets this as the display start time. The time when the subtitle text should be displayed is, for example, a PTS (Presentation Time Stamp) included in the subtitle data. Then, when the subtitle extraction unit 11 acquires the next subtitle data, the display start time of the next subtitle data is set as the display end time of the current subtitle data. After step S11, the subtitle generation device 10 performs the process of step S12.

In step S12, the subtitle extraction unit 11 confirms whether or not the acquired subtitle data has the control code “TIME” or “CS” (screen erase). Then, if neither the control code "TIME" nor "CS" is present in the subtitle data, the subtitle generation device 10 then performs the process of step S13.

In step S13, the timer unit 16 starts timing. If the timer unit 16 has already measured the time when the process of step S13 is executed, the timer unit 16 temporarily stops the time measurement, resets the timer unit 16, and then performs the time measurement by the timer unit 16. Start.

After step S13, the subtitle generation device 10 returns to step S11 and continues processing for the next subtitle data. Further, in parallel with this process, the end time setting unit 12 monitors the elapsed time measured by the timer unit 16 (step S14). Then, the end time setting unit 12 determines whether or not the elapsed time measured by the timer unit 16 has reached the set time (Expired).

When the elapsed time measured by the timer unit 16 reaches the set time, the end time setting unit 12 sets the display end time. At this time, if the next subtitle data does not arrive before the elapsed time measured by the timer unit 16 reaches the set time, the subtitle text is duplicated, and the start time setting unit 13 displays the duplicated subtitle text. Reset the start time. Then, the subtitle generation device 10 returns to step S13 to continue the process, and starts the time counting by the timer unit 16.

On the other hand, if the next subtitle data arrives before the elapsed time measured by the timer unit 16 reaches the set time in step S14, the end time setting unit 12 sets the display end time at that time. do. Then, the subtitle generation device 10 performs the process of step S15.

In step S15, the divided subtitle generation unit 14 divides based on the subtitle text acquired in step S11, the display start time, and the subtitle text, the display start time, and the display end time set in steps S13 and S14. Generate subtitle data. Then, the data output unit 15 outputs the divided subtitle data generated by the divided subtitle generation unit 14 as a subtitle file.

On the other hand, in step S12, when the subtitle data has the control code "TIME", the end time setting unit 12 sets the display end time of the subtitle text according to the control code "TIME". Further, in step S12, when the subtitle data has the control code "CS", the end time setting unit 12 sets the time when the screen is erased by the control code "CS" as the display end time of the subtitle text. Then, the subtitle generation device 10 performs the process of step S15. When the process of step S15 is completed, the series of processes is completed.

In the processing example shown in FIG. 5, depending on the reception processing timing of the subtitle data, a plurality of subtitle data whose output is suspended may exist at the same time. In this case, the order can be maintained by processing the subtitle data whose output is suspended by FIFO (first in, first out).

In the configuration example described above, the display end time of the subtitle text is set at regular time intervals based on the time counting result by the timer unit 16, and the divided subtitle data is generated. That is, when the subtitle text is acquired, if the display end time of the subtitle text cannot be determined, the timer unit 16 starts timing, and the timer unit 16 clocks the fixed time to end the display of the subtitle text. The time is set. However, the method of setting the display end time of the divided subtitle data is not limited to this.

The end time setting unit 12 may set the display end time of the divided subtitle data based on the queue signal input from the outside. That is, the end time setting unit 12 sets at least one or both of the first display end time and the second display end time based on the queue signal input from the outside. Specifically, for example, the SCTE-35 signal is used as a cue signal. The SCTE-35 signal is a signal that specifies the start and end of the program and the start and end of insertion of the advertisement. SCTE is an abbreviation for Society of Cable and Telecommunications Engineers.

At this time, for example, the value of the unitue_program_id field included in the splice_insert () message of the SCTE-35 signal can be used as the program ID. In this case, for example, if the value of the unique_program_id field, that is, the program ID is not changed, it is determined that the advertisement is inserted in the same program, and the end time setting unit 12 displays the subtitle text at the timing of the SCTE-35 signal. The end time is set, and the divided subtitle generation unit 14 generates the divided subtitle data. On the other hand, if the value of the unique_program_id field, that is, the program ID is changed, it is determined that one program has ended and another program has started. When the program ID is changed, the output destination (for example, the directory and the subtitle file name) of the subtitle file by the data output unit 15 is changed according to the program. Then, the counter of the divided subtitle file output by the data output unit 15 is reset. By such processing, the subtitle file can be output according to the program unit.

A processing example of the subtitle generation device 10 in this case will be described with reference to the flow chart of FIG. When the SCTE-35 signal is input to the subtitle generation device 10 as a queue signal from the outside, first, in step S21, the subtitle generation device 10 acquires the value of the unitue_program_id field of the SCTE-35 signal, that is, the program ID.

In the following step S22, the program ID acquired in step S21 is compared with the program ID acquired when the previous SCTE-35 signal was received, and it is determined whether or not the program ID has been changed. For example, the memory of the subtitle generation device 10 holds the value of the program ID acquired when the previous SCTE-35 signal was received. Then, if there is no change in the program ID, the subtitle generation device 10 then performs the process of step S23.

In step S23, for example, by the process shown in the flow chart of FIG. 5, the subtitle data is acquired from the outside, and the subtitle file in the WebVTT format is output. After step S23, the subtitle generator 10 returns to step S21 to continue processing.

On the other hand, if the program ID is changed in step S22, the subtitle generation device 10 then performs the process of step S24. In step S24, the output destination directory and the subtitle file name of the subtitle file by the data output unit 15 are changed according to the program. Then, the counter of the divided subtitle file output by the data output unit 15 is reset. After step S24, the subtitle generator 10 returns to step S21 to continue processing.

In the above, the case where the value of the unite_program_id field of the SCTE-35 signal is used as the program ID has been described, but the method of specifying the program ID is not limited to this. Alternatively, for example, another identifier such as the splice_event_id field included in the splice_insert () message of the SCTE-35 signal may be used as the program ID. Further, the program may be uniquely specified by a combination of a plurality of identifiers as well as one identifier, and may be used as a program ID.

The subtitle generation device 10 may be capable of outputting a divided subtitle file according to the HLS segment. That is, when the acquired subtitle data contains subtitle text that is displayed in excess of the length of the HLS segment, the subtitle data in which the display time of the subtitle text is divided is generated, and the subtitle file in WebVTT format is generated. Output.

A processing example of the subtitle generation device 10 in this case will be described with reference to the flow chart of FIG. First, in step S31, the subtitle extraction unit 11 acquires the subtitle data from the outside. Then, the subtitle extraction unit 11 extracts the subtitle text and the display time of the subtitle text from the acquired subtitle data.

In the following step S32, the subtitle generation device 10 determines whether or not the subtitle text acquired in step S31 has subtitle text that starts or ends to be displayed within the HLS segment time. There are three possible cases for this determination result. The first case is a case where the subtitle text to be displayed within the HLS segment time does not exist. The second case is a case where there is a subtitle text that starts to be displayed and ends to be displayed within the HLS segment time.

The third case is a case where there is a subtitle text that starts to be displayed within the HLS segment time and does not end the display, that is, continues to be displayed beyond the HLS segment time. In this third case, even if the HLS segment time has elapsed, the next subtitle data does not arrive at the subtitle generation device 10, and the display end time of the subtitle text cannot be determined.

In the first case, that is, when the subtitle text to be displayed does not exist within the HLS segment time, the subtitle generation device 10 then performs the process of step S33. In step S33, the data output unit 15 outputs a WebVTT format subtitle file with empty contents. After step S33, the subtitle generator 10 returns to step S31 to continue processing.

In the second case of step S32, that is, when there is a subtitle text that is started to be displayed and is finished to be displayed within the HLS segment time, the subtitle generation device 10 then performs the process of step S34. In step S34, the divided subtitle generation unit 14 generates subtitle data based on the subtitle text and the display time acquired in step S31. Then, the data output unit 15 outputs the generated subtitle data as a WebVTT format subtitle file. If the subtitle file waiting for output already exists, the data output unit 15 outputs the subtitle data in the form of adding the subtitle data to the subtitle file waiting for output. After step S34, the subtitle generation device 10 performs the process of step S35.

In step S35, the data output unit 15 generates a subtitle m3u8 file. The subtitle m3u8 file is a file that defines a playlist of subtitle files. If the subtitle m3u8 file already exists, the data output unit 15 adds the subtitle file output this time to the subtitle m3u8 file. After step S35, the subtitle generator 10 returns to step S31 to continue processing.

On the other hand, in the third case of step S32, that is, when there is a subtitle text that continues to be displayed beyond the HLS segment time, the subtitle generation device 10 next performs the process of step S36. In step S36, first, the end time setting unit 12 sets the end time of the HLS segment to the display end time of the subtitle text. Then, the divided subtitle generation unit 14 generates the divided subtitle data according to the set display end time. Further, the start time setting unit 13 sets the end time of the HLS segment as the display start time of the subtitle text. Then, the end time setting unit 12 sets, for example, the end time of the next HLS segment as the display end time of the subtitle text. The divided subtitle generation unit 14 generates the divided subtitle data according to the set display start time and display end time. In this way, the subtitle text is duplicated, and the divided subtitle data divided according to the HLS segment time is generated. Then, after step S36, the subtitle generation device 10 performs the process of step S34.

By doing so, the display time of the divided WebVTT format subtitle file is divided according to the HLS segment. Therefore, even when the reproduction is started from the HLS segment in the middle, it is possible to appropriately generate a subtitle file in which the subtitle can be displayed.

It should be noted that the data output unit 15 in step S33 does not necessarily have to output an empty WebVTT format subtitle file. More specifically, in the first case described above, that is, whether or not an empty WebVTT format subtitle file should be output when the subtitle text displayed within the HLS segment time does not exist is for the subtitle generated in step S35. It depends on the specifications of the m3u8 file. That is, in the first case, when an empty WebVTT format subtitle file is described in the subtitle m3u8 file in step S35, the empty WebVTT format subtitle file must be output in step S33. On the other hand, in the first case, if the empty WebVTT format subtitle file is not described in the subtitle m3u8 file in step S35, the empty WebVTT format subtitle file may or may not be output in step S33.

10 Subtitle generator 11 Subtitle extraction unit 12 End time setting unit 13 Start time setting unit 14 Divided subtitle generation unit 15 Data output unit 16 Timer unit

Claims (6)

A subtitle extraction unit that extracts the subtitle text and the first display start time of the subtitle text from the subtitle data acquired from the outside, and
An end time setting unit that sets the first display end time, which is a time after the first display start time, and
A divided subtitle generation unit that generates divided subtitle data in which the first display start time and the first display end time are associated with the subtitle text extracted by the subtitle extraction unit.
A data output unit that outputs the divided subtitle data generated by the divided subtitle generation unit, and
A start time setting unit for setting the first display end time to the second display start time of the subtitle text is provided.
The end time setting unit sets a second display end time, which is a time after the second display start time.
The divided subtitle generation unit is a subtitle generation device that generates divided subtitle data in which the second display start time and the second display end time are associated with the subtitle text extracted by the subtitle extraction unit. The end time setting unit is the first so that the interval from the first display start time to the first display end time is equal to the interval from the second display start time to the second display end time. The subtitle generation device according to claim 1, which sets the display end time and the second display end time. The subtitle generation device according to claim 1 or 2, wherein the end time setting unit sets one or both of the first display end time and the second display end time by time counting by a timer. The subtitle generation device according to claim 1, wherein the end time setting unit sets one or both of the first display end time and the second display end time based on a cue signal input from the outside. The data output unit includes divided subtitle data in which the first display start time and the first display end time are associated with the subtitle text, and the second display start time and the second display end time of the subtitle text. The subtitle generation device according to any one of claims 1 to 4, which outputs one subtitle file including the divided subtitle data associated with the above. The computer of the subtitle generator,
A subtitle extraction unit that extracts the subtitle text and the first display start time of the subtitle text from the subtitle data acquired from the outside, and
An end time setting unit that sets the first display end time, which is a time after the first display start time, and
A divided subtitle generation unit that generates divided subtitle data in which the first display start time and the first display end time are associated with the subtitle text extracted by the subtitle extraction unit.
A data output unit that outputs the divided subtitle data generated by the divided subtitle generation unit, and
In addition to functioning as a start time setting unit that sets the first display end time to the second display start time of the subtitle text,
The end time setting unit is made to set the second display end time, which is a time after the second display start time.
A subtitle generation program that causes the divided subtitle generation unit to generate divided subtitle data in which the second display start time and the second display end time are associated with the subtitle text extracted by the subtitle extraction unit.

Images