JP7096732B2 - Content distribution equipment and programs - Google Patents

Description

The present invention relates to a content distribution device and a program for live streaming video including subtitle data via the Internet.

Conventionally, in television broadcasting, as a broadcasting service for the hearing impaired, subtitled broadcasting is provided in which the sound of a broadcast program is displayed in characters on the screen. Subtitles transmitted in a live broadcast program (hereinafter referred to as "live subtitles") are produced by hand transcription from the audio of the live broadcast program. Therefore, the live subtitles are delayed by the transcription time, and are displayed on the screen with a delay with respect to the audio of the live broadcast program.

In order to suppress this delay in live subtitles, efforts are being made to utilize voice recognition technology or a high-speed input keyboard when producing live subtitles by manual transcription. In general, the method of producing subtitles includes a method of producing directly from the sound of a broadcast program, a method of re-speaking the sound of a broadcast program in a room with less noise in order to improve the accuracy of voice recognition, and the like. At present, the delay in subtitle production, the reproducibility of subtitles with respect to the audio of broadcast programs, etc. differ depending on the difference between these methods.

On the other hand, with the spread of smartphones and video distribution technologies in recent years, there is an increasing demand for simultaneously providing broadcast programs not only on the air but also on the Internet.

Some overseas broadcasting stations have already broadcast the same program on the Internet at the same time as broadcasting the program, and it is expected that such a service will be expanded in Japan in the future. In order to provide the same service in Japan, it is necessary to realize the same service level as broadcasting on the Internet, and it is also necessary to realize the same service level as broadcasting for the subtitle service.

In addition, adaptive streaming is a technique widely used in video distribution in recent years. Adaptive streaming is a technology that realizes uninterrupted video distribution by changing the video quality of multi-bit rate content according to the communication speed of the terminal.

Specifically, the distribution side encodes the content at a plurality of bit rates and generates a file divided into several seconds. The terminal side that receives the streaming sequentially acquires files with a bit rate that matches the communication speed of the terminal itself from the distribution side, and connects the files for playback. As a result, even in a terminal whose communication speed fluctuates, it is possible to continue playing the content and realize video distribution that is not interrupted (see, for example, Non-Patent Document 1).

However, in adaptive streaming, since the distribution side creates a file every few seconds, the content of the input video / audio data is temporarily stored in the buffer and the file is generated, so that a delay of at least several seconds occurs.

When the video content of a live broadcast program is live-streamed via the Internet, the distribution side uses the same signal as the live broadcast program to perform encoding, which is a file generation process for adaptive streaming.

In this case as well, since the live subtitles are produced by manually transcribing the audio of the live broadcast program, the live subtitles are displayed on the screen later than the audio, as in the case of broadcasting.

In this way, the live subtitles corresponding to the video and audio are displayed on the screen with a delay, but the smaller the display delay of the live subtitles, the easier it is to understand the program content. Especially for hearing-impaired people, live subtitles play a major role as a material for understanding the contents of the program, so the effect is great.

Here, a technique for synchronizing program contents and live subtitles has been proposed (see, for example, Patent Document 1). This content distribution device generates text data for the input video / audio by voice recognition processing, and inputs text data of subtitle data corresponding to the video / audio. Then, the content distribution device compares these text data, and when it is determined that the two are the same, the content distribution device corrects the time of the live subtitle of the portion to the time obtained by the voice recognition process.

As a result, the time of the live subtitles can be adjusted to the video and audio program contents, and the program contents and the live subtitles can be synchronized.

Japanese Unexamined Patent Publication No. 2017-5442

A.Zambelli, “IIS Smooth Streaming Technical Overview”, Mar.2009

The technique of Patent Document 1 described above synchronizes the program content with the live subtitles, but it is mainly intended for offline processing and is not always applicable to online live streaming processing. ..

This is because, in live streaming, it is necessary to perform streaming while encoding, but the above-mentioned Patent Document 1 does not describe the processing delay, which is an important factor in live streaming.

Further, in the above-mentioned technique of Patent Document 1, the raw subtitle correction processing time from the time of the video / audio corresponding to the raw subtitle to the time of the completion of the correction of the delay of the raw subtitle becomes long, and the real-time property is deteriorated.

As described above, the technique of Patent Document 1 described above is not intended for real-time processing, and therefore cannot be applied to live streaming that requires real-time performance.

Therefore, the present invention has been made to solve the above problems, and an object thereof is real-time without causing a delay exceeding the time when the encoding process is completed (encoding process completion time) in live streaming by Internet distribution. It is an object of the present invention to provide a content distribution device and a program that suppresses delays in live subtitles with respect to program contents while ensuring the characteristics.

In order to solve the above problem, the content distribution device according to claim 1 inputs a transmission signal for broadcasting, generates distribution data for distributing the content of the broadcast program to the Internet by live streaming, and transmits the broadcast program. In a content distribution device that corrects raw subtitle data included in a signal, an encoder that encodes the broadcast transmission signal and generates the distribution data in predetermined time units, and audio for audio included in the broadcast transmission signal. Recognition processing is performed to generate voice recognition data including voice time information regarding the time when the voice is output, and the delay time of the raw subtitle data corresponding to the voice recognition data is counted to obtain the subtitle delay elapsed time. The raw subtitle data included in the broadcast transmission signal is extracted, matching is performed between the voice recognition data and the live subtitle data, and the subtitle delay elapsed time and the voice of the voice recognition data by the encoder are used. Compared with the encoding processing completion time at which the encoding of the broadcast transmission signal including the video is completed, the subtitle delay elapsed time is before the encoding processing completion time, and the live subtitle corresponding to the voice recognition data. When the data matching is completed, at the timing when the matching is completed, the subtitle time information regarding the time when the raw subtitle data is displayed on the screen, which is included in the raw subtitle data, is the audio time included in the voice recognition data. Corrected based on information, a preset fixed value, or a statistical value of the delay time of the raw subtitle data corresponding to the voice recognition data, and the corrected raw subtitle data is output as new raw subtitle data. When the subtitle delay elapsed time is not before the encoding processing completion time, new raw subtitle data is generated based on the voice recognition data at the timing of the encoding processing completion time, and the new raw subtitle data is output. It is characterized by having a processing unit.

Further, in the content distribution device according to claim 2, in the content distribution device according to claim 1, the subtitle processing unit has a subtitle extraction unit that extracts the raw subtitle data from the broadcast transmission signal, and the broadcast transmission. The voice recognition process is performed on the voice included in the signal, and the voice recognition unit generates the voice recognition data, and the subtitle extraction unit corresponding to the voice recognition data generated by the voice recognition unit extracts the voice. The delay time of the raw subtitle data is counted, the subtitle delay elapsed time is obtained, the voice recognition data and the raw subtitle data are matched, and when the matching is completed, it is included in the voice recognition data. The matching unit that calculates the difference between the audio time information and the subtitle time information included in the raw subtitle data and obtains the subtitle delay confirmation time based on the difference, and the raw subtitle data are extracted. The time when the predetermined matching correction processing time until the new raw subtitle data is output is added to the subtitle delay confirmation time is set as the matching correction completion scheduled time, and the subtitle delay elapsed time is before the encoding processing completion time. If the scheduled matching correction completion time is before the encoding processing completion time, the matching unit completes the matching, and the matching correction completion time is based on the voice time information included in the voice recognition data. The subtitle time information included in the raw subtitle data is corrected, the corrected raw subtitle data is output as the new raw subtitle data, the subtitle delay elapsed time is before the encoding processing completion time, and the said. If the scheduled matching correction completion time is not before the encoding processing completion time, the subtitle time information included in the raw subtitle data is input based on the fixed value or the statistical value at the timing when the matching is completed by the matching unit. The corrected raw subtitle data is output as the new raw subtitle data, and if the subtitle delay elapsed time is not before the encoding processing completion time, the voice recognition data is displayed at the timing of the encoding processing completion time. Based on the above, the new raw subtitle data is generated, and the caption correction unit for outputting the new raw subtitle data is provided.

Further, in the content distribution device according to claim 3, in the content distribution device according to claim 2, the subtitle delay elapsed time obtained by the subtitle correction unit by the matching unit is before the encoding processing completion time, and If the scheduled matching correction completion time is before the encoding processing completion time, the first correction type is determined, the subtitle delay elapsed time is before the encoding processing completion time, and the matching correction completion scheduled time is A correction type determination unit that determines the second correction type if it is not before the encoding processing completion time, and determines a third correction type if the subtitle delay elapsed time is not before the encoding processing completion time, and the correction type determination unit. When the first correction type is determined, the subtitle time information included in the raw subtitle data is corrected to the voice time information included in the voice recognition data, and the corrected raw subtitle data is used as the new raw caption data. When the second correction type is determined by outputting as raw subtitle data, the fixed value or the statistical value is subtracted from the time of the subtitle time information included in the raw subtitle data to obtain the subtraction result, and the raw subtitle data is obtained. When the subtitle time information included in the subtitle data is corrected to the subtraction result, the corrected raw subtitle data is output as the new live subtitle data, and the third correction type is determined, the voice recognition data. It is characterized in that it is provided with a subtitle time correction unit that generates new raw subtitle data based on the above and outputs the new raw subtitle data.

Further, in the content distribution device according to claim 4, in the content distribution device according to claim 3, when the subtitle time correction unit determines the second correction type, each of the content distribution devices corresponds to the type of the broadcast program. Using the table in which the fixed value or the statistical value is stored, the fixed value or the statistical value corresponding to the broadcast program of the broadcast transmission signal is read from the table, and the subtitle included in the live subtitle data. The fixed value or the statistical value is subtracted from the time of the time information to obtain the subtraction result, the subtitle time information included in the raw subtitle data is corrected to the subtraction result, and the corrected raw subtitle data is newly used. The feature is that it is output as raw subtitle data.

Further, the program of claim 5 is characterized in that the computer functions as the content distribution device according to any one of claims 1 to 4.

As described above, according to the present invention, in live streaming by Internet distribution, it is possible to suppress the delay of live subtitles with respect to the program content while ensuring real-time performance without causing a delay exceeding the encoding processing completion time. ..

It is a schematic diagram which shows the whole configuration example of the content distribution system including the content distribution apparatus by embodiment of this invention, and is the block diagram which shows the configuration example of the content distribution apparatus. It is a block diagram which shows the structural example of the subtitle processing part. It is a block diagram which shows the structural example of the subtitle correction part. It is a flowchart which shows the processing example of the correction type determination part. (1) It is a figure explaining the case of subtitle delay elapsed time t ≦ encoding processing completion time Et, and subtitle delay confirmation time Jt + matching correction processing time R ≦ encoding processing completion time Et (when complete synchronization is possible). (2) It is a figure explaining the case of subtitle delay elapsed time t ≦ encoding processing completion time Et, and subtitle delay confirmation time Jt + matching correction processing time R> encoding processing completion time Et (when complete synchronization is difficult). (3) It is a figure explaining the case of subtitle delay elapsed time t> encoding process completion time Et (when perfect synchronization is difficult). It is a flowchart which shows the processing example of the subtitle time correction part.

Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the drawings. The present invention is characterized in that the time correction process of live subtitles is changed according to the degree of delay of live subtitles with respect to the program content in live streaming by Internet distribution, and the time correction process of live subtitles is performed until the encoding process completion time. To do it. As a result, it is possible to suppress the delay of live subtitles with respect to the program content while ensuring real-time performance.

[Content distribution device]
First, the content distribution device according to the embodiment of the present invention will be described. FIG. 1 is a schematic diagram showing an overall configuration example of a content distribution system including a content distribution device according to an embodiment of the present invention, and a block diagram showing a configuration example of the content distribution device. This content distribution system is a system for live streaming content via the Internet, and includes a content distribution device 1, a distribution server 2, and a terminal device 3.

The content distribution device 1 receives the broadcast transmission signal of the content, encodes the broadcast transmission signal and divides it into a plurality of files, generates distribution data D of the plurality of files, and generates a playlist. As the broadcast signal, for example, an SDI (Serial Digital Interface) signal is used.

The content distribution device 1 corrects the time of the live subtitle data included in the broadcast transmission signal, synchronizes the live subtitle data with the program content of the content which is video and audio, and edits the playlist. Then, the content distribution device 1 transmits the distribution data D, the synchronized (corrected) raw subtitle data a', and the playlist to the distribution server 2.

The broadcast transmission signal input by the content distribution device 1 is composed of video, audio, live subtitle data, and the like. Each of the video, audio, and live subtitle data includes time information based on a common time. As described above, since the live subtitle data is data produced by manually transcribing the audio of the live broadcast program, it is delayed from the video and audio program contents. The delay time of the raw subtitle data with respect to the program content varies depending on the operator to be produced and the raw subtitle data to be produced. Further, the playlist is metadata in which the acquisition destination, configuration content, time information, etc. of the distribution data D (video, audio, and live subtitle data) generated by the encoding process are described.

The distribution server 2 receives the content distribution data D, the synchronized raw subtitle data a', and the playlist from the content distribution device 1 and stores them in the memory.

The terminal device 3 is, for example, a video viewing player such as a smartphone, acquires a playlist from the content distribution device 1 via the Internet, and grasps the file structure based on the playlist. Then, the terminal device 3 acquires the distribution data D and the synchronized raw subtitle data a'based on the playlist by HTTP (Hypertext Transfer Protocol) via the Internet.

The terminal device 3 connects the distribution data D and the synchronized raw subtitle data a'according to the time of the playlist, displays the video and the subtitle on the screen, and outputs the audio to reproduce the content.

As a result, the terminal device 3 can reproduce the content having a small delay in displaying the subtitles for the video and audio, and the smaller the display delay of the subtitles, the easier it is for the user to understand the program contents. Especially for hearing-impaired people, live subtitles play a major role as a material for understanding the contents of the program, so the effect is great. Since the terminal device 3 can freely set the number of display lines and the number of display columns when displaying the subtitles on the screen, the subtitles can be displayed in synchronization with the program content.

In FIG. 1, the content distribution device 1 includes a distribution unit 10, an encoder 11, and a subtitle processing unit 12. The distribution unit 10 receives the broadcast transmission signal of the content, distributes the broadcast transmission signal, and outputs the broadcast transmission signal to the encoder 11 and the subtitle processing unit 12.

The encoder 11 inputs a broadcast transmission signal from the distribution unit 10, encodes the broadcast transmission signal, divides it into a file in units of several seconds, generates distribution data D, and generates a playlist. Then, the encoder 11 obtains the encoding processing time E along with the encoding, and extracts the program information from the playlist. The encoding processing time E is the processing time from the input of the broadcast transmission signal by the encoder 11 to the output of the distribution data D corresponding to the broadcast transmission signal.

The encoder 11 transmits the distribution data D to the distribution server 2, and outputs the encoding status including the encoding processing time E, the program information, and the playlist to the subtitle processing unit 12. The encoding status, program information, and playlist output frequency are arbitrary and may be in seconds, programs, or days.

The encoder 11 outputs the encoding status to the subtitle processing unit 12, and the subtitle processing unit 12 performs synchronization processing between the distribution data D and the raw subtitle data a. On the other hand, the encoder 11 may transmit the encoding status to the distribution server 2, and the distribution server 2 may perform such synchronization processing.

The subtitle processing unit 12 inputs a broadcast transmission signal from the distribution unit 10, and inputs an encoding status, program information, and a playlist from the encoder 11.

The subtitle processing unit 12 extracts the raw subtitle data a from the broadcast transmission signal, performs voice recognition processing on the broadcast transmission signal, and generates voice recognition data b. The subtitle processing unit 12 counts the time based on the time at which the broadcast transmission signal including the voice of the voice recognition data b is input, and obtains the subtitle delay elapsed time t. Here, the time means the time point of the moment in the passage of time, and the time means the length of the passage of time.

The subtitle delay elapsed time t indicates the time elapsed with respect to the time of the audio included in the broadcast transmission signal. The time period from the reference time of the timing at which the broadcast transmission signal including the voice of the voice recognition data b is input to the subtitle delay elapsed time t is defined as the subtitle delay time J. The subtitle delay elapsed time t is counted until the raw subtitle data a corresponding to the voice recognition data b (raw subtitle data a having the same content as the voice recognition data b) is extracted.

The subtitle processing unit 12 integrates the encoding processing time E included in the encoding status each time the encoding status is input to obtain an average, and at the time when the broadcast transmission signal including the voice of the voice recognition data b is input. Add the average of the encoding processing time E. Then, the subtitle processing unit 12 sets the time after addition to the encoding processing completion time Et.

The subtitle processing unit 12 performs matching between the raw subtitle data a and the voice recognition data b. Then, when the subtitle processing unit 12 extracts the raw subtitle data a corresponding to the voice recognition data b (when matching is completed), the subtitle processing unit 12 includes the time information (subtitle time information) included in the raw subtitle data a and the voice recognition data b. Calculate the difference from the time information (voice time information). The subtitle processing unit 12 adds the difference to the time at which the broadcast transmission signal including the voice of the voice recognition data b is input, and sets the time after the addition to the subtitle delay confirmation time Jt. The time corresponding to the difference in these time information corresponds to the delay time when the live subtitle data a is produced by manually transcribing from the audio of the live broadcast program.

Here, the subtitle processing unit 12 integrates the processing time from the input of the broadcast transmission signal including the raw subtitle data a to the completion of the processing of the raw subtitle data a for each raw subtitle data a. The average is obtained, and it is assumed that this is set and held in the matching correction processing time R. In the matching correction processing time R, after the subtitle processing unit 12 inputs the broadcast transmission signal including the raw subtitle data a, the raw subtitle data a is extracted, matched, and corrected, and the raw subtitle data a'. Is the processing time until the output of.

The subtitle processing unit 12 determines the correction type of the raw subtitle data a and performs correction processing based on the subtitle delay elapsed time t, the subtitle delay confirmation time Jt, the encoding processing completion time Et, the matching correction processing time R, and the like. Generate raw subtitle data a'and edit the playlist. The subtitle processing unit 12 transmits the raw subtitle data a'and the playlist to the distribution server 2.

FIG. 2 is a block diagram showing a configuration example of the subtitle processing unit 12 shown in FIG. The subtitle processing unit 12 includes a subtitle extraction unit 20, a voice recognition unit 21, a matching unit 22, and a subtitle correction unit 23.

The subtitle extraction unit 20 inputs a broadcast transmission signal from the distribution unit 10, extracts raw subtitle data a from the broadcast transmission signal, and outputs the raw subtitle data a to the matching unit 22. The raw subtitle data a includes time information regarding the time t _a when the raw subtitle is displayed on the screen.

The voice recognition unit 21 inputs a broadcast transmission signal from the distribution unit 10, performs known voice recognition processing on the voice included in the broadcast transmission signal, generates voice recognition data b, and generates voice recognition data b. Output to the matching unit 22. The voice recognition data b includes time information regarding the time t _b at which the voice is output.

The matching unit 22 inputs the raw subtitle data a from the subtitle extraction unit 20, and also inputs the voice recognition data b from the voice recognition unit 21. Then, the matching unit 22 matches the raw subtitle data a and the voice recognition data b, and identifies the raw subtitle data a and the voice recognition data b that are determined to be the same by the matching.

In this case, the matching unit 22 first inputs the voice recognition data b, and then inputs the raw subtitle data a corresponding to the voice recognition data b. The matching unit 22 determines that the voice recognition data b and the corresponding raw subtitle data a are the same (corresponding to each other) by matching the voice recognition data b and the corresponding raw subtitle data a.

The matching unit 22 counts the time based on the time when the broadcast transmission signal including the voice of the voice recognition data b is input by the voice recognition unit 21, adds the count value to the reference time, and the subtitle delay elapsed time t. Ask for.

When the matching unit 22 determines the raw subtitle data a corresponding to the voice recognition data b by matching, the matching unit 22 stops counting the subtitle delay elapsed time t. Then, the matching unit 22 calculates the difference between the time t a included in the raw subtitle data a and the time t _b included in the voice recognition data _b , and adds the difference to the reference time to set the subtitle delay. Set to the fixed time Jt.

The matching unit 22 acquires the processing time of the subtitle extraction unit 20, the processing time of the matching unit 22, and the processing time of the subtitle correction unit 23, and obtains an addition time obtained by adding these processing times. Then, each time the processing by the subtitle correction unit 23 is completed, the matching unit 22 integrates the addition time to obtain an average, and sets this as the matching correction processing time R.

The matching unit 22 completes matching of the raw subtitle data a corresponding to the voice recognition data b after inputting the voice recognition data b, and until immediately before setting the subtitle delay confirmation time Jt, the voice recognition data b. , The matching data including the subtitle delay elapsed time t and the matching correction processing time R is output to the subtitle correction unit 23.

Further, when the matching is completed and the subtitle delay confirmation time Jt is set, the matching unit 22 sets the raw subtitle data a, the voice recognition data b, the subtitle delay confirmation time Jt, and the matching correction processing time corresponding to the voice recognition data b. The matching data including R is output to the subtitle correction unit 23.

The matching data may be in sentence units, character units, word units, or a plurality of sentence units.

The subtitle correction unit 23 inputs matching data from the matching unit 22, and also inputs an encoding status, program information, and a playlist from the encoder 11. Then, the subtitle correction unit 23 integrates the encoding processing time E included in the encoding status to obtain an average, and adds the average to the time at which the broadcast transmission signal including the voice of the voice recognition data b is input. The time of the addition result is set to the encoding process completion time Et.

Here, the encoder 11 outputs the encoding status including the encoding processing time E to the subtitle correction unit 23. On the other hand, as will be described later, when a fixed value encoding processing time E is used instead of the actually calculated encoding processing time E, the encoder 11 does not have to obtain the encoding processing time E, and the encoder 11 does not have to obtain the encoding processing time E. , It is not necessary to output the encoder status including the encoder processing time E. In this case, the distribution data D and the raw subtitle data a can be synchronized with the distribution server 2 in the subsequent stage.

The subtitle correction unit 23 determines the correction type based on the subtitle delay elapsed time t, the subtitle delay confirmation time Jt, the matching correction processing time R, the encoding processing completion time Et, and the like included in the matching data.

The subtitle correction unit 23 corrects the raw subtitle data a according to the correction type, generates the raw subtitle data a', and edits the playlist. The subtitle correction unit 23 transmits the corrected raw subtitle data a'and the playlist to the distribution server 2.

FIG. 3 is a block diagram showing a configuration example of the subtitle correction unit 23 shown in FIG. The subtitle correction unit 23 includes an input unit 30, a communication unit 31, a correction type determination unit 32, and a subtitle time correction unit 33.

The input unit 30 inputs matching data from the matching unit 22, and outputs the matching data to the correction type determination unit 32.

The communication unit 31 inputs (receives) the encoding status, program information, and playlist from the encoder 11, outputs the encoding status to the correction type determination unit 32, and outputs the program information and playlist to the subtitle time correction unit 33. ..

The communication unit 31 receives the encode status, the program information, the playlist, and the like from an external system (a system that manages the encode status, the program information, the playlist, and the like) instead of receiving the encode status, the program information, and the playlist from the encoder 11. May be good. Further, the encoding status, the program information, and the reception frequency of the playlist are arbitrary, and may be in units of several seconds, units of programs, or units of one day.

[Correction type determination unit 32]
The correction type determination unit 32 inputs matching data from the input unit 30, and also inputs the encoding status from the communication unit 31. Then, the correction type determination unit 32 obtains the degree of delay of the raw subtitle data a based on the matching data and the encoding status. The correction type determination unit 32 determines the correction type according to the degree of delay of the raw subtitle data a, and outputs the correction type and matching data to the subtitle time correction unit 33.

FIG. 4 is a flowchart showing a processing example of the correction type determination unit 32. The correction type determination unit 32 inputs matching data from the input unit 30 (step S401) and inputs the encoding status from the communication unit 31 (step S402).

As described above, the correction type determination unit 32 obtains the encoding processing completion time Et using the encoding processing time E included in the encoding status (step S403). The processing of steps S401 to S403 shall be constantly updated in the processing example of FIG. 4, as in step S405 described later.

Here, when the matching of the raw subtitle data a corresponding to the voice recognition data b is not completed in the matching unit 22, the matching data includes the voice recognition data b, the subtitle delay elapsed time t, and the matching correction processing time R. Is done. When the matching is completed in the matching unit 22, the matching data includes the raw subtitle data a, the voice recognition data b, the subtitle delay confirmation time Jt, and the matching correction processing time R.

The correction type determination unit 32 initializes the subtitle sentence number i with the subtitle sentence number as i (i = 0, step S404). The subtitle delay elapsed time t, the subtitle delay confirmation time Jt, the matching correction processing time R, and the encoding processing completion time Et of the subtitle sentence number _i are set to ti, Jt _i , _{Ri, and Et i ,} respectively. By the following processing, the correction type is determined for the subtitle of the subtitle sentence number i.

The correction type determination unit 32 updates the input processing of steps S401 and S402, and also updates the processing of step S403 (step S405).

The correction type determination unit 32 determines whether or not the subtitle delay elapsed time ti is before (earlier or the same as _{) the encoding processing completion time Et i (ti ≤ Et i ) ( step} S406). When the correction type determination unit 32 determines in step S406 that the subtitle delay elapsed time ti is before the encoding processing completion time Et _i (ti ≤ Et _{i )} (step S406: Y ₎ , the subtitle delay time is set. It is determined whether or not the matching is completed, that is, whether or not the subtitle delay confirmation time Jt _i has been input after the matching is completed (step S407).

When the correction type determination unit 32 determines in step S407 that the subtitle delay time has not been determined (step S407: N), the process proceeds to step S405. On the other hand, when the correction type determination unit 32 determines in step S407 that the time of the subtitle delay is fixed (step S407: Y), the process proceeds to step S408.

The correction type determination unit 32 shifts from step S407 (Y) and adds the matching correction processing time R _i to the subtitle delay confirmation time Jt _i . The correction type determination unit 32 sets the time of the addition result as the matching correction completion scheduled time (Jt _i + R _i ), and the matching correction completion scheduled time (Jt _i + R _i ) is before (earlier or the same as) the encoding processing completion time Et _i . ) (Jt _i + R _i ≤ Et _i ) (step S408).

The scheduled matching completion time (Jt _i + _Ri ) is the raw subtitle data a corresponding to the voice recognition data b, based on the time when the subtitle processing unit 12 inputs the broadcast transmission signal including the voice of the voice recognition data b. It corresponds to the time when the extraction is completed, the matching is completed, and the correction of the raw subtitle data a is scheduled to be completed.

It should be noted that the comparison processing using the subtitle delay elapsed time t _i and the encoding processing completion time Et _i in step S406, and the subtitle delay confirmation time Jt _i , the matching correction processing time R _i , and the encoding processing completion time Et _i in step S408 are set. The technical significance of the comparative process used will be described with reference to FIGS. 5 to 7 described later.

In step S408, the correction type determination unit 32 determines that the scheduled matching correction completion time (Jt _i + Ri) is before the encoding processing completion time Et _i (Jt _i + _Ri ≤ Et _{i )} (step S408: Y). ), It is determined that the delay time of the raw subtitle data a is short and the correction process of the raw subtitle data a is completed before the encoding processing completion time Et _i elapses, and the correction type is determined to be A (step S409).

The correction type A is a type capable of completely matching the time of the live subtitles (complete synchronization) with respect to the program content, and the time t of the raw subtitle data a is the time t of the voice recognition data _b by the subtitle time correction unit 33 in the subsequent stage. Corrected to _b .

When the correction type determination unit 32 determines in step S408 that the scheduled matching correction completion time (Jt _i + Ri) is not before the encoding processing completion time Et _i (Jt _i + _Ri > Et _{i )} (step S408: N). , It is determined that the delay time of the raw subtitle data a is not so short, and the correction type is determined to be B (step S410).

The correction type B is _a type in which it is difficult to completely match (completely synchronize) the time of the live subtitles with respect to the program content, and the subtitle time correction unit 33 determines the time ta of the live subtitle data _a with reference to the time ta. It is corrected based on the fixed value P of.

When the correction type determination unit 32 determines in step S406 that the subtitle delay elapsed time t _i is not before the encoding processing completion time Et _i (ti> Et _{i )} (step S406: N), the subtitle delay elapsed time t _i Is determined to exceed the encoding processing completion time Et _i , it is determined that the delay time of the raw subtitle data a is long, and the correction type is determined to be C (step S411).

The correction type C is a type in which it is difficult to completely match the time of the live subtitles (complete synchronization) with respect to the program content, and the subtitle time correction unit 33 applies a part or all of the voice recognition data b to the live subtitle data a'. Or do not perform the process of generating the raw subtitle data a'.

In this way, by the processing of steps S404 to S411, the time correction processing of the raw subtitle is _performed based on the subtitle delay elapsed time ti, the subtitle delay confirmation time Jt _i , the matching correction processing time R _i , and the encoding processing completion time Et _i . The correction types A, B, and C indicating the type are determined.

The correction type determination unit 32 shifts from steps S409 to S411 and outputs the correction type and matching data to the subtitle time correction unit 33 (step S412).

The correction type determination unit 32 determines whether or not the processing is completed for all the subtitles (step S413), and if it is determined that the processing is not completed (step S413: N), the subtitle sentence number i is incremented. (I = i + 1, step S414), the process proceeds to step S405. As a result, the processing of steps S405 to S412 is performed for the subtitle of the next subtitle sentence number i.

When the correction type determination unit 32 determines in step S413 that the processing has been completed for all the subtitles (step S413: Y), the correction type determination unit 32 ends the processing.

The correction type determination unit 32 inputs the encoding status including the actually calculated encoding processing time E from the communication unit 31, but uses a preset fixed value encoding processing time E. You may. Further, the correction type determination unit 32 inputs the matching data including the actually calculated matching correction processing time R from the input unit 30, but uses a preset matching correction processing time R. You may do so.

Further, the correction type determination unit 32 may use a fixed value according to the time or a fixed value for each program as the preset encoding processing time E, and may use the preset matching correction processing time R as the matching correction processing time R. A fixed value according to the time or a fixed value for each program may be used.

(Comparison processing of steps S406 and S408)
Next, the comparison process of steps S406 and S408 shown in FIG. 4 will be described in detail. Hereinafter, the notation of the subtitle sentence number i will be omitted.

FIG. 5 describes the case of (1) subtitle delay elapsed time t ≤ encoding processing completion time Et, and subtitle delay confirmation time Jt + matching correction processing time R ≤ encoding processing completion time Et (when complete synchronization is possible). It is a figure. More specifically, it is a figure explaining the case where "Y" is determined in step S406 of FIG. 4, "Y" is determined in step S408, and the correction type A is determined. The horizontal axis is the time axis.

The distribution data D is set to the encoding processing completion time Et after the encoding processing time E has elapsed, based on the timing of the video and audio broadcasting transmission signal (the timing at which the broadcasting transmission signal including the audio of the audio recognition data b is input). Generated.

Further, it is assumed that the raw subtitle data a corresponding to the voice recognition data b is delayed by the subtitle delay time J with reference to the timing of the video and audio broadcast transmission signals, and the matching is completed at this timing. The time at this timing is the subtitle delay confirmation time Jt. It is assumed that t ≦ Et and the condition of Jt + R ≦ Et is satisfied.

The encoding processing time E is a substantially fixed value, the subtitle delay time J is a variable value, and the matching correction processing time R is a substantially fixed value. The same applies to FIGS. 6 and 7 described later.

Then, the timing of the raw subtitle data a (a') after the matching correction becomes the timing of Jt + R before the encoding processing completion time Et. The timing range (range of the subtitle delay confirmation time Jt) of the raw subtitle data a satisfying this condition (t ≦ Et, Jt + R ≦ Et) is as shown by the broken line of the arrow in FIG.

In this case, it is determined that the delay time of the raw subtitle data a is short and the correction process of the raw subtitle data a is completed before the encoding process completion time Et elapses. Therefore, the subtitles can be completely synchronized with the program content, and the correction type A is determined. Then, when the matching of the raw subtitle data a is completed, the time ta of the raw subtitle data a is corrected to the time t _b of the voice recognition data _b (t _a ← t _b ). As a result, the distribution data D and the corrected raw subtitle data a'corresponding to the distribution data D are aligned at the encoding processing completion time Et.

FIG. 6 illustrates the case of (2) subtitle delay elapsed time t ≤ encoding processing completion time Et, and subtitle delay confirmation time Jt + matching correction processing time R> encoding processing completion time Et (when complete synchronization is difficult). It is a figure. More specifically, it is a figure explaining the case where "Y" is determined in step S406 of FIG. 4, "N" is determined in step S408, and the correction type B is determined. The horizontal axis is the time axis.

The distribution data D is set to the encoding processing completion time Et after the encoding processing time E has elapsed, based on the timing of the video and audio broadcasting transmission signal (the timing at which the broadcasting transmission signal including the audio of the audio recognition data b is input). Generated.

Further, it is assumed that the raw subtitle data a corresponding to the voice recognition data b is delayed by the subtitle delay time J with reference to the timing of the video and audio broadcast transmission signals, and the matching is completed at this timing. The time at this timing is the subtitle delay confirmation time Jt. It is assumed that t ≦ Et and the condition of Jt + R> Et is satisfied.

Then, assuming that the raw subtitle data a is corrected by the subtitle processing unit 12, the timing of the raw subtitle data a (a') after the matching correction becomes the timing of Jt + R that exceeds the encoding processing completion time Et. The timing range (range of the subtitle delay confirmation time Jt) of the raw subtitle data a satisfying this condition (t ≦ Et, Jt + R> Et) is as shown by the broken line of the arrow in FIG.

In this case, although the delay time of the raw subtitle data a is not so short, it is determined that the correction process of the raw subtitle data a is completed at a predetermined time exceeding the encoding process completion time Et. Therefore, it is difficult to completely synchronize the subtitles with respect to the program content, and the correction type B is determined. Then, when the matching of the raw subtitle data a is completed, the correction is made based on _a predetermined fixed value P with the time ta of the raw subtitle data _a as _a reference (ta ← ta −P). As a result, the distribution data D and the corrected raw subtitle data a'corresponding to the distribution data D are aligned at the encoding processing completion time Et.

FIG. 7 is a diagram for explaining the case of (3) subtitle delay elapsed time t> encoding processing completion time Et (when perfect synchronization is difficult), and “N” is determined in step S406 of FIG. It is a figure in the case of determining C. The horizontal axis is the time axis.

The distribution data D is set to the encoding processing completion time Et after the encoding processing time E has elapsed, based on the timing of the video and audio broadcasting transmission signal (the timing at which the broadcasting transmission signal including the audio of the audio recognition data b is input). Generated.

Further, the raw subtitle data a corresponding to the voice recognition data b is delayed by the subtitle delay time J with reference to the timing of the video and audio broadcast transmission signal, and the subtitle delay elapsed time t exceeds the encoding processing completion time Et. It shall be.

Then, the timing range (range of the subtitle delay confirmation time Jt) of the raw subtitle data a satisfying this condition (t> Et) is shown by the broken line of the arrow in FIG.

In this case, it is determined that the delay time of the raw subtitle data a is long, it is difficult to completely synchronize the subtitles with respect to the program content, and the correction type C is determined. Then, at the timing of the encoding processing completion time Et, a part or all of the voice recognition data b is applied to generate the raw subtitle data a', or the raw subtitle data a'is not generated. As a result, the distribution data D and the raw subtitle data a'corresponding to the distribution data D are aligned at the encoding processing completion time Et.

[Subtitle time correction unit 33]
Returning to FIG. 3, the subtitle time correction unit 33 inputs the correction type and matching data from the correction type determination unit 32, and inputs the program information and the playlist from the communication unit 31.

The subtitle time correction unit 33 performs processing for correcting the time information of the raw subtitle data a included in the matching data according to the correction type, and generates the raw subtitle data a'. Further, the subtitle time correction unit 33 corrects the time information and the like of the raw subtitle data a included in the playlist, and sets the raw subtitle data a as the raw subtitle data a'to edit the playlist.

The subtitle time correction unit 33 transmits the corrected raw subtitle data a'and the playlist to the distribution server 2. Here, when correcting the time information of the raw subtitle data a, a fixed value P preset according to the program information is used depending on the correction type.

FIG. 8 is a flowchart showing a processing example of the subtitle time correction unit 33. The subtitle time correction unit 33 inputs the correction type and matching data from the correction type determination unit 32 (step S801). Further, the subtitle time correction unit 33 inputs program information and a playlist from the communication unit 31 (step S802).

The correction type is A, B or C. The matching data for the correction types A and B are raw subtitle data a, voice recognition data b, subtitle delay confirmation time Jt, and matching correction processing time R. The matching data for the correction type C are voice recognition data b, subtitle delay elapsed time t, and matching correction processing time R.

The subtitle time correction unit 33 selects a fixed value P according to the program information from a plurality of preset fixed values P1, P2, ... (Step S803).

For example, a table in which the types of programs included in the program information and the fixed values P1, P2, ... Are associated with each other is stored in the memory in advance. The subtitle time correction unit 33 reads a fixed value P corresponding to the type of the program included in the program information input from the communication unit 31 from the table, and selects the read fixed value P.

The fixed values P1, P2, ... Are delay times preset by the operator according to the type of the program, and are stored in the table. In general, news programs require less time to transcribe subtitles than information programs. Therefore, the fixed value P1 of the news program stores a value smaller than the fixed value P2 of the information program.

As a result, in step S806 described later, the time information (time ta) of the raw subtitle data _a is appropriately corrected according to the type of the program based on the fixed value P.

Further, the subtitle time correction unit 33 measures the fixed subtitle delay time of the raw subtitle data a corresponding to the voice recognition data b for each type of program and stores it in the memory, and the accumulated statistical value of the addition result (for example). The average value) may be obtained, and the statistical value P'may be used instead of the fixed value P. In this case, the statistical value P'is used instead of the fixed value P in step S806 described later.

For example, a table in which the types of programs included in the program information and the above-mentioned statistical values P1', P2', ... Are associated with each other is stored in the memory in advance. The subtitle time correction unit 33 may read the statistical value P'corresponding to the type of the program included in the program information from the table and select the read statistical value P'.

Although the subtitle time correction unit 33 selects the fixed value P according to the program information, the fixed value P set in advance may be used without depending on the program information.

The subtitle time correction unit 33 determines the correction type (step S804). When the subtitle time correction unit 33 determines the correction type A in step S804 (step S804: A), the subtitle time correction unit 33 determines that the live subtitles can be completely synchronized with the program content, and sets the time ta of the live subtitle data _a as audio. The recognition data b is corrected to the time t _b (step S805). As a result, the time t a of the raw subtitle data a is corrected to the time _{t b .} This subtitle time correction process is performed at the timing when the matching is completed. That is, the correction processing of the subtitle time is surely performed at the timing before the encoding processing completion time Et, and is not performed at the timing exceeding the encoding processing completion time Et.

On the other hand, when the subtitle time correction unit 33 determines the correction type B in step S804 (step S804: B), the subtitle time correction unit 33 determines that complete synchronization of the live subtitles with respect to the program content is difficult, and the time ta of the live subtitle data _a . Is corrected based on the fixed value P selected in step S803 (step S806).

Specifically, the subtitle time correction unit 33 subtracts _a fixed value P from the time ta of the raw subtitle data a, and sets the subtraction result as a new time (t _a −P) of the raw subtitle data a. As _{a result, the time ta of the raw subtitle data a is corrected to the time (t a −P )} obtained by subtracting the fixed value P from the time ta. This subtitle time correction process is performed at the timing when the matching is completed. That is, the correction processing of the subtitle time is performed at a timing almost before the encoding processing completion time Et.

When the correction type B is determined in step S804, the subtitle time correction unit 33 may correct the time t of the raw subtitle data a to the time t _b of the voice recognition data _b , as in step S805. good.

When the subtitle time correction unit 33 determines the correction type C in step S804 (step S804: C), the subtitle time correction unit 33 determines that it is difficult to completely synchronize the live subtitles with the program content, and performs a part or all of the voice recognition data b. The raw subtitle data generation process for applying and generating new raw subtitle data is performed, or no processing is performed (the raw subtitle data generation process is not performed) (step S807).

Specifically, when the subtitle time correction unit 33 applies a part or all of the voice recognition data b to generate new raw subtitle data, the subtitle time correction unit 33 extracts text data from the voice recognition data b and is one of the text data. Part or all of the data is the subtitle of the new subtitle data, and the time t _b of the voice recognition data b is the time of the new subtitle data. As a result, the new subtitle data becomes the data to which the voice recognition data b is applied. This subtitle time correction process is performed at the timing of the encoding process completion time Et.

The subtitle time correction unit 33 shifts from step S805, step S806 or step S807 to generate the corrected raw subtitle data a as raw subtitle data a', or uses new subtitle data as raw subtitle data a'. , The time ta of the raw subtitle data a included in the playlist is edited to the time t _b , ta −P of the raw subtitle data _a ′ ₍ step S808).

As a result, in the case of the correction type A, instead of the raw subtitle data a including the time t _a , the raw subtitle data a'that includes the time t _b and has the same subtitle as the raw subtitle data a is generated and is displayed in the playlist. The time t a of the included raw subtitle data a is edited to the time _{t b of} the raw subtitle data a'.

Further, in the case of the correction type B, instead of the raw subtitle data _a including the time ta, the raw subtitle data _a'that includes the time ta −P and has the same subtitle as the raw subtitle data a is generated, and the playlist is generated. The time ta of the raw subtitle data _a included in is edited to the time ta −P of the raw subtitle data _a ′.

Further, in the case of the correction type C, the raw subtitle data a'is generated in which the time is set to time t _b and part or all of the text data of the voice recognition data b is subtitled. Further, the time t a of the raw subtitle data a included in the playlist is edited to the time _{t b of} the raw subtitle data a'. Or, in the case of the correction type C, the correction processing is not performed.

The playlist is generated as metadata in which the acquisition destination and the configuration contents of the distribution data are described in the encoder 11, and then the raw subtitle data a'is normally displayed on the terminal device 3 in the subtitle time correction unit 33. Edited to be displayed.

The subtitle time correction unit 33 transmits the raw subtitle data a'and the playlist to the distribution server 2 (step S809).

The raw subtitle data a'transmitted from the subtitle time correction unit 33 to the distribution server 2 is in a format (WebVTT, TTML, ARIB-TTML, etc.) that matches the distribution format (HLS, MPEG-DASH, etc.).

As described above, according to the content distribution device 1 of the embodiment of the present invention, the matching unit 22 has a time based on the time when the broadcast transmission signal including the voice of the voice recognition data b is input by the voice recognition unit 21. Is counted, and the subtitle delay elapsed time t is obtained.

The matching unit 22 is from the timing at which the broadcast transmission signal including the voice of the voice recognition data b is input by the voice recognition unit 21 until the matching of the raw subtitle data a corresponding to the voice recognition data b is completed. Matching data including voice recognition data b, subtitle delay elapsed time t, and matching correction processing time R is output to the subtitle correction unit 23.

Further, when the matching of the raw subtitle data a corresponding to the voice recognition data b is completed, the matching unit 22 between the time t a included in the raw subtitle data a and the time t _b included in the voice recognition data _b . The difference is calculated, the subtitle delay confirmation time Jt reflecting the difference is obtained, and the matching data including the raw subtitle data a, the voice recognition data b, the subtitle delay confirmation time Jt and the matching correction processing time R is output to the subtitle correction unit 23. do.

The subtitle correction unit 23 determines the correction type A when t ≦ Et and Jt + R ≦ Et. Then, the subtitle correction unit 23 corrects the time t of the raw subtitle data a to the time t _b of the voice recognition data _b at the timing before the encoding processing completion time Et, and generates the raw subtitle data a'.

Further, the subtitle correction unit 23 determines the correction type B when t ≦ Et and Jt + R> Et. Then, the subtitle correction unit 23 subtracts _a predetermined fixed value P from the time ta of the raw subtitle data _a at the timing before the encoding processing completion time Et, thereby setting the time ta of the raw subtitle data _a (ta −). Correct to P) and generate raw subtitle data a'.

Further, when t> Et, the subtitle correction unit 23 determines the correction type C, and applies a part or all of the voice recognition data b to the raw subtitle data a'at the timing of the encoding processing completion time Et. Generate or do not perform correction processing.

As a result, the correction of the live subtitle data a is performed at the timing until the encoding process completion time Et, when the encoding of the broadcast transmission signal including the video and audio corresponding to the live subtitle data a is completed. Further, the time t a of the raw subtitle data a for which matching is completed is corrected to the time t _b of the voice recognition data _{b, or is corrected to a time close to the time t b .}

Therefore, in live streaming by Internet distribution, it is possible to suppress the delay of live subtitles with respect to the program content. In particular, even when the live subtitles are significantly delayed with respect to the program content, the raw subtitle data a can be corrected at the timing of the encoding processing completion time Et without waiting for the arrival of the live subtitles.

That is, in live streaming by Internet distribution, it is possible to suppress the delay of live subtitles for the program content while ensuring real-time performance without causing a delay exceeding the encoding processing time, and it is possible to provide a more understandable program. ..

Although the present invention has been described above with reference to embodiments, the present invention is not limited to the above-described embodiment and can be variously modified without departing from the technical idea. For example, in the content distribution device 1 of the above embodiment, the subtitle correction unit 23 of the subtitle processing unit 12 has a subtitle delay elapsed time t in a state where the raw subtitle data a is not input to the matching unit 22 in step S406 of FIG. If it is determined that the encoding process is not completed at Et or less, the correction type C is determined, and a part or all of the voice recognition data b is applied to generate the raw subtitle data a', or the correction process is not performed. did.

On the other hand, the subtitle correction unit 23 may extract the time t _b from the voice recognition data b and transmit the information about the time t _b to the distribution server 2 as the time of the raw subtitle data a.

Further, in the content distribution device 1 of the embodiment, the subtitle processing unit 12 generates raw subtitle data a'for Internet distribution and transmits it to the distribution server 2.

On the other hand, the subtitle processing unit 12 may output the generated raw subtitle data a'to a device (not shown in FIG. 1) provided in front of the content distribution device 1. In this case, the device multiplexes the raw subtitle data a'with the broadcast transmission signal, and outputs the broadcast transmission signal after multiplexing to the content distribution device 1. The content distribution device 1 inputs a broadcast transmission signal after multiplexing from the device, encodes the broadcast transmission signal to generate distribution data D, generates a playlist, and raw subtitle data from the broadcast transmission signal. To extract. Then, the content distribution device 1 transmits the distribution data, the raw subtitle data, and the playlist to the distribution server 2.

Further, in the content distribution system shown in FIG. 1, the distribution server 2 and the terminal device 3 are connected via the Internet. On the other hand, the content distribution device 1 and the distribution server 2 may also be connected via the Internet.

As the hardware configuration of the content distribution device 1 according to the embodiment of the present invention, a normal computer can be used. The content distribution device 1 is composed of a computer provided with a volatile storage medium such as a CPU and RAM, a non-volatile storage medium such as a ROM, and an interface.

Each function of the distribution unit 10, the encoder 11, and the subtitle processing unit 12 provided in the content distribution device 1 is realized by causing the CPU to execute a program describing these functions. The subtitle processing unit 12 is composed of a subtitle extraction unit 20, a voice recognition unit 21, a matching unit 22, and a subtitle correction unit 23. The subtitle correction unit 23 includes an input unit 30, a communication unit 31, a correction type determination unit 32, and a subtitle time. It is composed of a correction unit 33.

These programs are stored in the storage medium, read by the CPU, and executed. In addition, these programs can be stored and distributed in storage media such as magnetic disks (floppy (registered trademark) disks, hard disks, etc.), optical disks (CD-ROM, DVD, etc.), semiconductor memories, etc., and can be distributed via a network. You can also send and receive.

1 Content distribution device 2 Distribution server 3 Terminal device 10 Distribution unit 11 Encoder 12 Subtitle processing unit 20 Subtitle extraction unit 21 Voice recognition unit 22 Matching unit 23 Subtitle correction unit 30 Input unit 31 Communication unit 32 Correction type determination unit 33 Subtitle time correction unit a, a'Live subtitle data b Voice recognition data D Distribution data t, ti Subtitle delay elapsed time Jt, Jti Subtitle delay confirmation time Et, Eti Encoding processing completion time R, Ri Matching correction processing time J Subtitle delay time E Encoding processing time i Subtitle text number

Claims (5)

In a content distribution device that inputs a broadcast transmission signal, generates distribution data for live streaming the content of a broadcast program to the Internet, and corrects raw subtitle data included in the broadcast transmission signal.
An encoder that encodes the broadcast transmission signal and generates the distribution data in predetermined time units.
Voice recognition processing is performed on the voice included in the broadcast transmission signal, voice recognition data including voice time information regarding the time when the voice is output is generated, and the raw subtitle data corresponding to the voice recognition data is generated. Count the delay time to find the elapsed time of the subtitle delay,
The raw subtitle data included in the broadcast transmission signal is extracted, matching is performed between the voice recognition data and the live subtitle data, and the data is matched.
The subtitle delay elapsed time is compared with the encoding processing completion time at which the encoding of the broadcast transmission signal including the voice and video of the voice recognition data is completed by the encoder.
When the subtitle delay elapsed time is before the encoding processing completion time and the matching of the raw subtitle data corresponding to the voice recognition data is completed, it is included in the raw subtitle data at the timing when the matching is completed. , The subtitle time information regarding the time when the live subtitle data is displayed on the screen, the voice time information included in the voice recognition data, a preset fixed value, or a delay of the live subtitle data corresponding to the voice recognition data. It is corrected based on the statistical value of time, and the corrected raw subtitle data is output as new raw subtitle data.
If the elapsed subtitle delay time is not before the encoding processing completion time, new raw subtitle data is generated based on the voice recognition data at the timing of the encoding processing completion time, and the new subtitle data is output. Processing unit and
A content distribution device characterized by being equipped with. In the content distribution device according to claim 1,
The subtitle processing unit
A subtitle extraction unit that extracts the raw subtitle data from the broadcast transmission signal,
A voice recognition unit that performs the voice recognition process on the voice included in the broadcast signal to generate the voice recognition data, and a voice recognition unit.
The delay time of the raw subtitle data extracted by the subtitle extraction unit corresponding to the voice recognition data generated by the voice recognition unit is counted, and the subtitle delay elapsed time is obtained.
When the voice recognition data and the raw subtitle data are matched and the matching is completed, the voice time information included in the voice recognition data and the subtitle time information included in the raw subtitle data are used. A matching unit that calculates the difference and obtains the subtitle delay confirmation time based on the difference,
The time when the predetermined matching correction processing time from the extraction of the raw subtitle data to the output of the new raw subtitle data is added to the subtitle delay confirmation time is set as the matching correction completion scheduled time, and the subtitle delay elapses. When the time is before the encoding processing completion time and the matching correction completion scheduled time is before the encoding processing completion time, the voice recognition data includes the matching at the timing when the matching is completed by the matching unit. Based on the audio time information, the subtitle time information included in the raw subtitle data is corrected, and the corrected raw subtitle data is output as the new live subtitle data.
When the subtitle delay elapsed time is before the encoding processing completion time and the matching correction completion scheduled time is not before the encoding processing completion time, the fixed value or the said at the timing when the matching is completed by the matching unit. Based on the statistical value, the subtitle time information included in the raw subtitle data is corrected, and the corrected raw subtitle data is output as the new raw subtitle data.
If the elapsed subtitle delay time is not before the encoding processing completion time, the new raw subtitle data is generated based on the voice recognition data at the timing of the encoding processing completion time, and the new raw subtitle data is output. Subtitle correction part and
A content distribution device characterized by being equipped with. In the content distribution device according to claim 2,
The subtitle correction unit
When the subtitle delay elapsed time obtained by the matching unit is before the encoding processing completion time and the matching correction completion scheduled time is before the encoding processing completion time, the first correction type is determined and the subtitle. If the delayed elapsed time is before the encoding processing completion time and the matching correction completion scheduled time is not before the encoding processing completion time, the second correction type is determined, and the subtitle delay elapsed time is the encoding processing completion time. If not before, the correction type determination unit that determines the third correction type,
When the first correction type is determined by the correction type determination unit, the subtitle time information included in the raw subtitle data is corrected to the voice time information included in the voice recognition data, and the corrected raw subtitle is corrected. The data is output as the new raw subtitle data,
When the second correction type is determined, the fixed value or the statistical value is subtracted from the time of the subtitle time information included in the raw subtitle data to obtain a subtraction result, and the subtitle included in the raw subtitle data is obtained. The time information is corrected to the subtraction result, and the corrected raw subtitle data is output as the new raw subtitle data.
When the third correction type is determined, a subtitle time correction unit that generates new raw subtitle data based on the voice recognition data and outputs the new raw subtitle data, and a subtitle time correction unit.
A content distribution device characterized by being equipped with. In the content distribution device according to claim 3,
The subtitle time correction unit
When the second correction type is determined, the table corresponding to the broadcast program of the broadcast transmission signal is used by using the table in which the fixed value or the statistical value corresponding to the type of the broadcast program is stored. The fixed value or the statistical value is read from the table, the fixed value or the statistical value is subtracted from the time of the subtitle time information included in the raw subtitle data to obtain a subtraction result, and the above-mentioned included in the raw subtitle data. A content distribution device characterized in that the subtitle time information is corrected to the subtraction result, and the corrected raw subtitle data is output as the new live subtitle data. A program for causing a computer to function as the content distribution device according to any one of claims 1 to 4.

Images