JP3811751B2 - Synthetic timing adjustment system - Google Patents

Description

[0001]
[Technical field to which the invention belongs]
The present invention relates to a technique for adjusting the timing of synthesizing subtitle broadcast video and subtitles on the broadcast station side, and in particular, in subtitle broadcast where subtitle editing cannot take time, video and subtitles are appropriately received by a receiver. The present invention relates to a technique for adjusting the synthesis timing of video and subtitles so that they are displayed at a proper timing.
[0002]
[Prior art]
Among television broadcasts, there is a caption broadcast, which is a broadcast that displays captions within a range of 1 to 3 lines as about 15 characters per line on the receiver screen. The subtitles displayed on the receiver screen are displayed for about 5 to 6 seconds in consideration of the time that humans can read, and then the new subtitles of the receiver are replaced by totally replacing the first three lines. Display on the screen. Thus, in order to display subtitles on the receiver screen, the substation data corresponding to the audio of the program to be broadcast is created on the broadcast station side, and the created subtitle data is transmitted to the viewer by the text broadcasting method. It is transmitted and synthesized with video data by a dedicated teletext decoder.
[0003]
By the way, in a program with subtitles, it usually takes about one week to edit subtitles. For example, in a program such as a drama program that takes time to edit captions, first, caption data is created by a word processor or the like based on recorded video and audio. Then, the television data and the caption data are displayed so that the caption displayed on the receiver screen feels natural to the viewer (the caption is displayed before the audio is output (approximately 0.5 seconds before)). And synthesize.
[0004]
However, when subtitle editing must be performed on the day of the program broadcast, it is not possible to spend time on subtitle editing. For this reason, the broadcasting station must synthesize the caption data with the television data and broadcast it while creating the caption data based on the television data. In this case, as a method of creating caption data, a method of converting audio data of television data into caption data using a speech recognition program, and a method of creating caption data using a word processor while an operator listens to sound You could think so.
[0005]
And the method of converting to subtitle data using the voice recognition program has a constant conversion time from the voice data to the subtitle data, but the current voice recognition program has a voice recognition capability of about 90%. An error may be mixed in the subtitle data.
[0006]
On the other hand, the method of creating caption data by the operator varies in the conversion time from the audio data to the caption data, but errors are rarely mixed in the caption data created from the audio data of the television data.
[0007]
As a result, caption data created by an operator is used as caption data for which accuracy is required.
[0008]
Furthermore, in the broadcast with subtitles, it is necessary to appropriately combine the television data and the subtitle data so that the subtitles displayed on the receiver match the audio output timing so as not to give the viewer a sense of incongruity. As described above, as a method of appropriately combining the television data and the caption data, a method of adjusting the output of the television data in accordance with the conversion timing to the caption data and combining the caption data and the television data, and an operator In consideration of the subtitle conversion time, television data is given to the operator before the broadcast time, and the television data is converted into subtitle data based on the television data, and the subtitle data and the television data are combined.
[0009]
[Problems to be solved by the invention]
However, in the former method, since the output of the television data fluctuates, the video broadcast by the receiver is fluctuated, giving the viewer a sense of incongruity. That is, the output of television data must be constant.
[0010]
Therefore, in the present invention, in consideration of the caption conversion time of the operator, the television data is given to the operator in advance at an appropriate timing and converted into the caption data, so that the caption data and the television data are appropriately combined. It is to provide the technology to adjust.
[0011]
[Means for Solving the Problems]
  A first invention for solving the above-mentioned problem is a system for adjusting the timing of synthesizing caption data to television data composed of audio data and video data,
  Storage means for storing television data;
  Output means for outputting broadcast television data and audio data of the television data stored in the storage means;
  Generating means for reproducing audio from the audio data output from the output means, and generating subtitle data by operator input based on the reproduced audio;
  Conversion means for converting audio data of broadcast television data output from the output means into subtitle data at a constant conversion speed by voice recognition;
  Synthesizing means for synthesizing the broadcast television data output from the output means and the caption data generated by the generating means;
  Of the subtitle data generated by the generating means, the same subtitle data as the arbitrary subtitle data is searched from the subtitle data converted by the converting means, and the time at the end of conversion of the searched subtitle data in the converting means Subtracting the conversion time calculated on the basis of the time at the end of generation of the arbitrary subtitle data in the generation unit and the conversion speed of the conversion unit to calculate the time difference of synthesis timing in the synthesis unitCalculation means;
  Control means for adjusting the speed of the sound reproduced in the generating means by controlling the sound data output from the output means based on the deviation time calculated by the calculating means.
[0012]
  According to a second invention for solving the above-mentioned problem, in the first invention,
  The control means includesBased on the deviation time calculated by the calculating means, the speed of the sound reproduced by the generating means is adjusted by increasing or decreasing the number of samples of the audio data output from the output means. Characterize.
[0013]
According to a third invention for solving the above-described problem, in the first or second invention, the control unit is configured to determine the television data and the caption data in the synthesizing unit based on the time calculated by the calculation unit. If the timing of synthesizing the subtitle data is behind the appropriate synthesis timing, the number of samples of the audio data output from the output means is reduced and the audio reproduced by the generating means When the subtitle data is ahead of the synthesis timing, the number of samples of the audio data output from the output means is increased to reduce the speed of the reproduced audio.
[0014]
According to a fourth invention for solving the above-mentioned problem, in the second invention, the control means decreases or increases the number of samples of the current audio data stepwise to approach the number of samples of the target audio data. Control is performed so that the speed of the sound reproduced by the generating means is gradually reduced or accelerated.
[0015]
According to a fifth invention for solving the above-mentioned problem, in the first or second invention, the synthesis timing adjustment system is
First accumulating means for accumulating subtitle data generated by the generating means;
Second storage means for storing subtitle data converted by the conversion means,
The calculating means includes the same continuous character as the predetermined continuous character selected from the caption data stored in the first storage means included in the caption data stored in the second storage means. It is configured to calculate whether or not the difference time is calculated when the same consecutive characters are included.
[0016]
  A sixth invention for solving the above-described problem is a system for adjusting the timing of synthesizing caption data to television data composed of audio data and video data,
  Storage means for storing television data;
  Output means for outputting broadcast television data and audio data of the television data stored in the storage means;
  Generating means for reproducing audio from the audio data output from the output means, and generating subtitle data by operator input based on the reproduced audio;
  Broadcast output from the output meansConstant conversion speed by voice recognition for audio data of television dataConversion means for converting to subtitle data with
  Synthesizing means for synthesizing the caption data generated by the generating means and the television data for broadcasting output from the output means;
  First storage means for storing subtitle data generated by the generation means;
  Second storage means for storing the caption data converted by the conversion means;
  Generated by the generating meansA first table in which the characters of the caption data and the time when the characters are accumulated in the first accumulation means are stored in association with each other;
  Converted by the converting meansA second table in which the characters of the caption data are stored in association with the time at which the characters were accumulated in the second accumulation means;
  The time when each character of the caption data generated by the generating means is stored in the first storage means, and each character of the caption data converted by the conversion means is stored in the second storage means The time at the time is monitored, and each character of the subtitle data is stored in the first table or the second table in association with the time at which each character is accumulated in the first accumulation means or the second accumulation means. Means to
  It is determined whether or not the subtitle data stored in the second storage unit includes the same continuous character as the predetermined continuous character selected from the subtitle data stored in the first storage unit. Discriminating means to perform,
  When it is determined by the determining means that the same continuous character is included, the same character as this character is searched from the first table and the second table, and the time associated with the searched character Reading means for reading from the first table and the second table;
  The time read from the first table and the conversion time calculated based on the conversion speed of the conversion means are subtracted from the time read from the second table to calculate the synthesis timing shift time in the synthesis means. Calculation means;
  Control means for adjusting the speed of the sound reproduced by the generating means by increasing or decreasing the number of samples of the sound data output from the output means based on the deviation time calculated by the calculating means. And
[0017]
According to a seventh invention for solving the above-mentioned problem, in the sixth invention, the control means is a timing for synthesizing the television data and the caption data in the synthesizing means based on the time calculated by the calculating means. If the timing of synthesizing the caption data is behind the appropriate synthesis timing, the number of samples of the audio data output from the output means is reduced and the speed of the audio reproduced by the generating means is reduced. When the subtitle data is advanced from the synthesis timing, the number of samples of the audio data output from the output means is increased to reduce the speed of the reproduced audio.
[0018]
According to an eighth invention for solving the above-mentioned problem, in the sixth invention, the control means decreases or increases the number of samples of the current audio data stepwise to approach the number of samples of the target audio data. Control is performed so that the speed of the sound reproduced by the generating means is gradually reduced or accelerated.
[0020]
First, the configuration of the present embodiment will be described.
[0021]
FIG. 1 is a block diagram of a synthesis timing adjustment system. FIG. 2 is a diagram for explaining the error time.
[0022]
The synthesis timing adjustment system according to the present embodiment includes a server 10, a speech recognition device 20, a caption creation device 30, a comparison device 40, and an inserter 50.
[0023]
The server 10 includes a storage unit 11, a timer 12, an output unit 13, and an output adjustment unit 14.
[0024]
The voice recognition device 20 includes a storage unit 21 and a control unit 22.
[0025]
Furthermore, the caption creation device 30 includes a speaker 31, an input unit 32, and a control unit 33.
[0026]
The comparison device 40 includes buffers 41 and 42, a timer 43, a storage unit 44, and a control unit 45.
[0027]
Each configuration will be described below.
[0028]
The storage unit 11 of the server 10 stores broadcast television data.
[0029]
The timer 12 of the server 10 measures the current time (time).
[0030]
The output unit 13 of the server 10 monitors the timer 12, and when the timer 12 reaches a preset broadcast time, the storage unit 11 stores the same television data as the television data output from the output adjustment unit 14. The read television data is output to the inserter 50 as television data for broadcasting.
[0031]
The output adjustment unit 14 of the server 10 is set with a broadcast time and a preceding time. The preceding time is a rough time required for creating caption data from audio data in the caption generation device 30, and is an experience value.
[0032]
Further, the output adjustment unit 14 of the server 10 monitors the timer 12, and when the time of the timer 12 reaches the time obtained by subtracting the preceding time from the broadcast time, the television 10 stores the television data stored in the storage unit 11 and reads this data. The television data is output to the caption creating device 30 as television data for creating caption data. For example, when the broadcast time is 10h00m00s, where h is hours, m is minutes, and s is seconds, and the preceding time is 5s, the output adjustment unit 14 sets the time of the timer 12 to (10h00m00s− 5s = 9h59m55s), the television data stored in the storage unit 11 is output to the subtitle creating apparatus 30.
[0033]
Further, the output adjustment unit 14 receives an error time (shift time) transmitted from the comparison device 40. The error time indicates how much time is shifted from the timing at which the caption data from the caption creation device 20 and the television data for broadcasting output from the server 10 can be appropriately combined in the inserter 50. It's time. For example, when the error time is “0 s”, it indicates that the caption data and the television data are appropriately combined.
[0034]
The output adjustment unit 14 checks the received error time. When the error time is “0 s”, the output adjustment unit 14 maintains the current state. On the other hand, when the error time is other than “0 s”, the output adjustment unit 14 increases or decreases the number of audio data samples of the television data read from the storage unit 11, and the audio data obtained by increasing or decreasing the number of samples is newly updated. Let it be audio data.
[0035]
Incidentally, the transfer rate of audio data output from the server 10 is fixed. As a result, in order to change the voice speed that can be heard by the operator (human), the total amount of voice data to be reproduced by the caption creating device 30 must be increased or decreased. That is, the audio data with the number of samples increased or decreased must be output from the server 10 at a fixed transfer rate. For example, when the number of samples per unit time of the current audio data is X sample and the audio speed heard by the operator of the caption creation device 30 is multiplied by α, the output adjustment unit 14 determines that the number of audio data samples is “X / Α "sample, increase / decrease and output at Xsample per unit time. Note that the technology for creating new audio data by increasing or decreasing the number of samples of the current audio data is an existing technology, and thus detailed description thereof is omitted.
[0036]
Further, the output adjustment unit 14 determines the duration (duration) and how long the audio data of the increased / decreased data is to be continuously transmitted to the caption creation device 30. Therefore, the output adjustment unit 14 sets the duration based on the received error time. calculate. For example, when the error time is T and the voice speed heard by the operator of the caption production device 30 is α times the normal voice speed, the output adjustment unit 14 | T / (α−1) | ( Calculate the duration by calculating s).
[0037]
Then, the output adjustment unit 14 continues to transmit the television data in which the calculated duration and the number of samples of the audio data are increased / decreased to the caption creation device 30, and then the television data based on the number of samples of the audio data is obtained. It transmits to the caption production device 30.
[0038]
In this way, by continuing to output from the output adjustment unit 14 the duration calculated based on the error time and the number of samples of audio data, the audio speed heard by the operator changes. As a result, the operator changes the speed of subtitle character input in accordance with the change in voice speed. For this reason, the subtitle data creation speed in the subtitle creation apparatus 30 changes, and the inserter 50 appropriately synthesizes the subtitle data and the television data.
[0039]
The storage unit 21 of the speech recognition apparatus 20 stores a speech recognition program that creates caption data from the audio data of the television data. Note that conversion from voice data to subtitle data by the voice recognition program is a known technique, and thus detailed description thereof is omitted.
[0040]
DETAILED DESCRIPTION OF THE INVENTION
  When receiving the television data from the server 10, the control unit 22 of the speech recognition device 20 receives the speech recognition program.MuAccordingly, the audio data of the received television data is converted into subtitle data. Note that the voice recognition device 20 converts voice data into subtitle data at a substantially constant conversion speed. For example, the voice recognition device 20 has caption data at a conversion speed of 5 (characters / s).InConvert. This conversion speed is assumed to be known in advance.
[0041]
When receiving the television data from the server 10, the control unit 33 of the caption generation device 30 causes the audio data of the received television data to be output from the speaker 31 as sound. When the operator who has heard this voice inputs a subtitle character from the input unit 32, the control unit 33 generates subtitle data based on the input character and transmits it to the comparison device 40.
[0042]
By the way, normally, there is a relatively large margin for the input operation of subtitle characters by the operator. As a result, even when the voice speed heard by the operator increases, the operator can input subtitle characters.
[0043]
The buffer 41 of the comparison device 40 is a FIFO buffer, and accumulates subtitle data from the speech recognition device 20 for the number of subtitle characters to be displayed on the screen of a receiver (not shown).
[0044]
The buffer 42 of the comparison device 40 is a FIFO buffer, and stores the subtitle data from the subtitle creation device 30 as many times as the number of subtitle characters to be displayed on the receiver screen in the same manner as the buffer 41.
[0045]
The timer 43 of the comparison device 40 measures the time after the synthesis timing adjustment system operates.
[0046]
The storage unit 44 of the comparison device 40 stores a table A and a table B.
[0047]
In the table A, each character of the caption data from the speech recognition device 20 is associated with the time at which each character is accumulated in the buffer 41.
[0048]
In the table B, each character of the caption data from the caption creating device 30 is associated with the time when each character is accumulated in the buffer 42.
[0049]
The control unit 45 of the comparison device 40 accumulates the caption data transmitted from the speech recognition device 20 in the buffer 41, acquires the time at which each character of the caption data is accumulated in the buffer 41 from the timer 43, Each character of the caption data is associated with the acquired time and stored in the table A. Similarly, the caption data transmitted from the caption creation device 30 is accumulated in the buffer 42, and the time at which each character of the caption data is accumulated in the buffer 42 is obtained from the timer 43, and each character of the caption data The acquired time is associated with and stored in the table B.
[0050]
Further, the control unit 45 calculates the error time at a constant cycle (for example, a cycle of 10 seconds). Specifically, first, two consecutive kanji characters from the beginning of the caption data stored in the buffer 42 are searched, and two kanji characters identical to the searched two-character kanji are stored in the buffer 41. Search from the beginning of the closed caption data. Then, it is determined whether or not the same two Chinese characters can be searched. As a result, if the same two-character kanji can be searched, the same kanji as the searched two-character kanji is searched from the table A and associated with the first one of the searched two-character kanji. The stored time is read from table A. Similarly, the same Kanji as the previously searched two-character kanji is searched from the table B, and the time associated with the first one of the searched two-character kanji is read from the table B.
[0051]
Further, as shown in FIG. 2, the time (H) read from the table B is subtracted from the time (K) read from the table A, and the conversion time (d) calculated from the conversion speed of the speech recognition device 20 from the subtraction result. Is subtracted to obtain an error time (T). That is, the control unit 45 calculates T = K−H−d. The time thus obtained is transmitted to the server 10 as an error time (T).
[0052]
The inserter 50 synthesizes the television data transmitted from the server 10 and the caption data transmitted from the caption generation device 30.
[0053]
Next, the operation in the above configuration will be described.
[0054]
FIG. 3 is a diagram for explaining the acquisition of time at the time when each character of the caption data is accumulated in the buffer 41. FIG. 4 is a diagram for explaining the table A. FIG. 5 is a diagram for explaining the acquisition of the time at which each character of the caption data is accumulated in the buffer 42. FIG. 6 is a diagram for explaining the table B.
[0055]
The voice speed that the operator can hear is about 0.75 to 1.5 times the normal voice speed. Therefore, in the present embodiment, when the error time value is a negative value (when the audio speed in the caption production device 30 needs to be accelerated), the television data output from the output adjustment unit 14 of the server 10 When the number of samples in the audio data is 1 / 1.5 times and the error time value is a positive value (when the audio speed in the caption production device 30 needs to be reduced), the output adjustment unit It is assumed that the number of samples in the audio data of the television data output from 14 is 1 / 0.75 times.
[0056]
When the synthesis timing adjustment system starts operating, the output adjustment unit 14 of the server 10 starts monitoring the time counted by the timer 12. Then, when the time of the timer 12 reaches the time obtained by subtracting the preceding time from the broadcast time, the output adjustment unit 14 reads the television data stored in the storage unit 11 and uses the read television data to generate caption data. For this purpose, it is transmitted to the caption production device 30 as television data.
[0057]
On the other hand, the output unit 13 of the server 10 also monitors the time of the timer 12, and when the timer 12 reaches the broadcast time, the storage unit 11 stores the same television data as the television data output from the output adjustment unit 14. The read television data is transmitted to the inserter 50 as television data for broadcasting.
[0058]
By the way, in the caption production device 30 that has received the television data from the server 10, the control unit 33 outputs the audio data of the television data as sound from the speaker 31. The operator who has heard the output voice inputs subtitle characters from the input unit 32. And the control part 33 produces | generates caption data based on the character input from the input part 32, and transmits to the inserter 50 and the comparison apparatus 40 via the communication part 31. FIG.
[0059]
When the inserter 50 receives the television data from the server 10, the inserter 50 synthesizes the received television data with the caption data from the caption creation device 30.
[0060]
On the other hand, in the voice recognition device 20 that has received the television data from the server 10, the control unit 22 executes the voice recognition program in the storage unit 21, and converts the voice data of the received television data into subtitle data according to the voice recognition program. To do. The converted caption data is transmitted to the comparison device 40.
[0061]
Then, when receiving the caption data from the speech recognition device 20, the control unit 45 of the comparison device 40 accumulates the received caption data in the buffer 41 and sets the time at which each character of the caption data is accumulated in the buffer 41. It is acquired from the timer 43, and each character of the caption data is associated with the acquired time and stored in the table A. For example, as shown in FIG. 3, the order of the subtitle data transmitted from the speech recognition device 20 is “Today it was fine, but tomorrow is the West Japan Karate luck breaks out. The time when the subtitle data “now” is accumulated in the buffer 41 is 10.51 s, and the subtitle data “day” is accumulated in the buffer 41 in the order in which they are transmitted. Is 10.56 s, and the time when the subtitle data “is” stored in the buffer 41 is 10.61 s..., The control unit 45 stores the subtitle data “now” in the buffer 41. Time (10.51 s) is acquired, and the time (10.56 s) when subtitle data “date” is accumulated in the buffer 41 is obtained, and the subtitle data “is” is accumulated in the buffer 41. Time (10.61s To get the, to get the .... Further, as shown in FIG. 4, the control unit 45 associates the caption data “now” with the acquired time “10.51 s”, associates the caption data “date” with the acquired time “10.56 s”, and displays the caption. The data “is associated with the acquired time“ 10.61 s ”and stored in the table A.
[0062]
Similarly, when receiving the caption data from the caption generation device 30, the control unit 45 of the comparison device 40 accumulates the received caption data in the buffer 42 and sets the time at which each character of the caption data is accumulated in the buffer 42. It is acquired from the timer 43, and each character of the caption data is associated with the acquired time and stored in the table B. For example, as shown in FIG. 5, the subtitle data transmitted from the subtitle creation device 30 is in the order of “It was clear, but the weather will be lost from western Japan tomorrow. If the time at which each character of the caption data is stored in the buffer 42 is 6.01s, 6.06s, 6.11s,. The unit 45 acquires the time (6.01 s, 6.06 s, 6.11 s,...) When each character of the caption data is accumulated in the buffer 42. Further, as illustrated in FIG. 6, the control unit 45 associates the caption data “clear” with the acquired time “6.01s”, associates the caption data “re” with the acquired time “6.06 s”, and displays the caption. The data “in” is associated with the acquired time “6.11 s” and stored in the table B.
[0063]
Next, the control unit 45 of the comparison device 40 searches the subtitle data stored in the buffer 42 sequentially for two consecutive kanji characters from the top of the buffer 42. It is determined whether or not the same two-character kanji as the retrieved two-character kanji can be searched from the caption data stored in the buffer 41. For example, the subtitle data “It was clear in buffer 42, but the weather will fall from west Japan tomorrow.” Is stored, and the subtitle data stored in buffer 41 is “clear today, but tomorrow. When “Nippon Karate” comes down, the award is stored, and the control unit 45 searches for two consecutive kanji characters from the top of the subtitle data stored in the buffer 42 and selects “Tomorrow”. Search for. Further, the control unit 45 searches for the same two-character kanji as the searched subtitle data “Tomorrow” from the top of the subtitle data stored in the buffer 41, and determines whether or not the same two-character kanji can be searched. Determine.
[0064]
As a result, when it is determined that the same two-character kanji cannot be searched, the control unit 45 searches the subtitle data stored in the buffer 42 for the next two-character kanji, and the searched two-character kanji and It is determined whether or not the same kanji can be searched from subtitle data stored in the buffer 41. When such operations are repeated and the same two-character kanji cannot be searched from the subtitle data stored in the buffer 42 and the subtitle data stored in the buffer 41, further subtitle data is stored in the buffer 41 and the buffer 42. Wait until it accumulates.
[0065]
On the other hand, when it is determined that the same two kanji characters can be searched, the control unit 45 searches the table A for the time associated with the first kanji character among the two kanji characters searched from the buffer 41. The retrieved time is read from the table A. Further, the control unit 45 searches the table B for the time associated with the first one of the two kanji characters searched from the buffer 42 and reads the searched time from the table B. For example, in the above example, the subtitle data “Tomorrow” is stored in the buffer 41 and the buffer 42. Therefore, the same two-character kanji as the subtitle data “Tomorrow” is searched from the table A, The time “11.001 s” associated with the first Chinese character “Ming” of the two characters searched is read from the table A. Similarly, the control unit 45 searches the table B for the same two-character kanji character “Tomorrow” as the previously searched two-character kanji character “Tomorrow”. The time “6.36 s” associated with the Chinese character “Ming” is read from the table B.
[0066]
Then, the control unit 45 subtracts the time read from the table B from the time read from the table A. That is, the control unit 45 calculates (11.01 s−6.36 s = 4.65 s). Further, the control unit 45 subtracts the conversion time (5 s) calculated based on the conversion speed of the speech recognition apparatus 20 from the calculated time (4.65 s), and thereby calculates the time (4.65 s−5 s = -0.35 s) is transmitted to the server 10 as an error time.
[0067]
The output adjustment unit 14 of the server 10 that has received the error time from the comparison device 40 checks the received error time.
[0068]
When the error time is “0 s”, the output adjustment unit 14 maintains the current state.
[0069]
On the other hand, when the error time value is a negative value, the output adjusting unit 14 decreases the number of samples in the audio data of the television data read from the storage unit 11. For example, when the error time (T) is “−1 s” and the number of samples per unit time of the current audio data is 48 k samples, the output adjustment unit 14 determines that the number of samples of the audio data is “48/1”. .5 = 32 k sample ”and output at 48 k sample per unit time. Thereby, the voice speed heard by the operator of the caption creating device 30 is increased.
[0070]
When the error time value is a positive value, the output adjustment unit 14 increases the number of audio data samples of the television data read from the storage unit 11. For example, when the error time (T) is “+ 1s”, the output adjustment unit 14 converts the number of audio data samples to be “48 / 0.75 = 64 k sample”, and 48 k sample per unit time. To output. As a result, the voice speed heard by the operator of the caption creating device 30 is reduced.
[0071]
Further, the output control unit 14 calculates the duration based on the received error time. For example, when the error time (T) is “−1 s”, the output control unit 14 calculates “| −1 / (1.5−1) | = 2.0 s” from the error time.
[0072]
Then, the output control unit 14 continues to transmit the television data in which the calculated duration and the number of samples of the audio data are increased or decreased to the caption creating device 30.
[0073]
After the lapse of time, the output adjustment unit 14 transmits the increased or decreased audio data samples to the subtitle creating apparatus 30 so that the number of samples is the original number of samples (48 k samples).
[0074]
As described above, the output adjustment unit 14 increases or decreases the number of samples of the audio data, so that the audio speed heard by the operator of the caption creation device 30 changes. As a result, the subtitle data generation speed in the subtitle generation apparatus 30 changes, and the inserter 50 can appropriately combine the subtitle data and the television data.
[0075]
Note that the number of subtitle data stored in the buffers 41 and 43 has been described as the number of subtitle characters to be displayed on the receiver screen, but the present invention is not limited to this case. For example, subtitle data for the number of characters larger or smaller than the number of subtitle characters displayed on the receiver screen may be accumulated.
[0076]
In the present embodiment, the case where two consecutive kanji characters are searched from the subtitle data stored in the buffer 41 and the buffer 42 has been described. However, the present invention is not limited to this case. For example, a continuous three-character hiragana may be used.
[0077]
Further, although the television data transmitted from the server 10 to the caption creating device 20 is television data composed of video data and audio data, the present invention is not limited to this case. For example, the data transmitted from the server 10 to the caption creating device 20 may be only audio data.
[0078]
Moreover, although this Embodiment demonstrated the case where control of the television data output from the server 10 was implement | achieved by the output part 13 and the output adjustment part 14, it does not restrict to this case. For example, the output unit 13 and the output adjustment unit 14 may be realized by a digital signal processor.
[0079]
In the present embodiment, the case where the number of samples of the audio data is increased by 1 / 1.5 or 1 / 0.75 has been described. However, the present invention is not limited to this case. This magnification may be arbitrarily variable.
[0080]
A second embodiment will be described.
[0081]
In the first embodiment, when the number of samples of audio data output from the server is (1 / 0.75) times or (1 / 1.5) times, that is, for the operator of the caption generation device. The case where the sound speed to be heard is suddenly changed to 0.75 times or 1.5 times has been described.
[0082]
However, if the voice speed is changed rapidly, the operator may not be able to cope with the change.
[0083]
Therefore, in the second embodiment, the number of samples in the audio data of the television data output from the server is decreased or increased in stages, and the audio speed heard by the operator of the caption generation device is increased or decreased in stages. It is characterized by.
[0084]
Hereinafter, the second embodiment will be described in detail.
[0085]
In the output adjustment unit 14 of the server 10, update information is set in addition to the broadcast time and the preceding time described in the first embodiment.
[0086]
Here, the update information is obtained by increasing or decreasing the number of samples per unit time in the audio data of the television data output from the output adjustment unit 14 to the target number of samples per unit time. It is information indicating whether or not to approach.
[0087]
Further, in addition to the function described in the first embodiment, the output adjustment unit 14 increases or decreases the number of audio data samples in a stepwise manner according to the set update information, and samples the target audio data. It will be closer to the number.
[0088]
Next, the operation in the above configuration will be described.
[0089]
The operation until the number of samples of the audio data of the television data output from the server 10 is increased or decreased according to the error time calculated by the comparison device 40 is the same as the operation described in the first embodiment. In this embodiment, it is omitted.
[0090]
When increasing or decreasing the number of samples of audio data, the output adjustment unit 14 increases or decreases the number of samples of the current audio data stepwise according to the set update information to obtain the number of samples of the target audio data. Move closer. For example, the number of samples per unit time of the current audio data is “48 k sample”, the number of samples of the target audio data is “48 / 0.75 = 64 k sample”, and the update information is updated every 0.5 s. In the case of information for instructing to gradually reduce or increase the number of samples per unit time by 10% so as to approach the number of samples of the target audio data, the output adjustment unit 14 The number of audio data samples “48 k sample” is updated to “52.8 k sample”, and the number of audio data samples “52.8 k sample” becomes “57.6 k sample” 0.5 seconds later. Update so that “59.2 k sample” becomes “64 k sample”.
[0091]
As described above, the output adjustment unit 14 decreases or increases the number of samples of the audio data in stages, so that the audio speed heard by the operator of the caption creation device 30 increases or decreases in stages. It can correspond to.
[0092]
In the first and second embodiments, the case where subtitles and audio are simultaneously output from the receiver has been described. However, the present invention is not limited to this case. For example, a viewer who is watching a broadcast with subtitles displays subtitles on the receiver screen just before the audio is output (for example, the subtitles are received by the receiver 0.5 seconds before the audio is output). Can be read without discomfort. In this way, miscalculation time may be taken into consideration so that subtitles can be read without a sense of incongruity. That is, when the error time is “+0.5 s”, it corresponds to the error time “0 s” described in the first embodiment.
[0107]
【The invention's effect】
According to the present invention, it is possible to adjust the synthesis timing of the television data and the caption data so that the caption is displayed on the receiver screen at an appropriate timing even when the caption editing does not take time. Has an effect.
[0108]
According to the present invention, since the number of samples of the audio data of the television data output from the server is decreased or increased to the target number of samples, the voice speed heard by the operator changes stepwise, and the change An excellent effect is that the operator can follow the above.
[Brief description of the drawings]
FIG. 1 is a block diagram of a synthesis timing adjustment system.
FIG. 2 is a diagram for explaining an error time.
FIG. 3 is a diagram for describing acquisition of time at which each character of subtitle data is accumulated in a buffer 41;
FIG. 4 is a diagram for explaining a table A;
FIG. 5 is a diagram for describing acquisition of time at which each character of subtitle data is accumulated in a buffer;
FIG. 6 is a diagram for explaining a table B;
FIG. 7 is a block diagram of a synthesis timing adjustment system.
[Explanation of symbols]
10 servers
20 Voice recognition device
30 Subtitle creation device
40 comparison device
50 Inserter

Claims (8)

A system for adjusting the timing of synthesizing caption data to television data composed of audio data and video data,
Storage means for storing television data;
Output means for outputting broadcast television data and audio data of the television data stored in the storage means;
Generating means for reproducing audio from the audio data output from the output means, and generating subtitle data by operator input based on the reproduced audio;
Conversion means for converting audio data of broadcast television data output from the output means into subtitle data at a constant conversion speed by voice recognition;
Synthesizing means for synthesizing the broadcast television data output from the output means and the caption data generated by the generating means;
Of the subtitle data generated by the generating means, the same subtitle data as the arbitrary subtitle data is searched from the subtitle data converted by the converting means, and the time at the end of conversion of the searched subtitle data in the converting means Calculating means for subtracting a conversion time calculated based on a time at the end of generation of the arbitrary subtitle data in the generation means and a conversion speed of the conversion means, and calculating a deviation time of the synthesis timing in the synthesis means; ,
Control means for adjusting the speed of the sound reproduced in the generating means by controlling the sound data output from the output means based on the deviation time calculated by the calculating means;
A synthesis timing adjustment system comprising: The control means is configured to adjust the speed of the sound reproduced by the generating means by increasing or decreasing the number of samples of the sound data output from the output means based on the deviation time calculated by the calculating means. The synthesis timing adjustment system according to claim 1, wherein:   The control means determines a deviation in the synthesis timing between the television data and the caption data in the synthesis means based on the time calculated by the calculation means, and the timing at which the caption data is synthesized is determined from an appropriate synthesis timing. If it is delayed, the number of samples of the audio data output from the output means is reduced to accelerate the speed of the audio reproduced by the generating means, and when the subtitle data is ahead of the synthesis timing, the output means 3. The synthesis timing adjustment system according to claim 2, wherein the number of samples of audio data to be output is increased to reduce the speed of the reproduced audio.   The control unit decreases or increases the current number of audio data samples stepwise to approach the target number of audio data samples and gradually decreases or accelerates the speed of the sound reproduced by the generation unit. The synthesis timing adjustment system according to claim 2, wherein the synthesis timing adjustment system is controlled as follows. Synthetic timing adjustment system
First accumulating means for accumulating subtitle data generated by the generating means;
Second storage means for storing the caption data converted by the conversion means,
In the subtitle data stored in the second storage unit, the calculation unit includes the same continuous character as the predetermined continuous character selected from the subtitle data stored in the first storage unit. 3. The composition timing according to claim 1, wherein when the same consecutive characters are included, a shift time is calculated. 3. Adjustment system. A system for adjusting the timing of synthesizing caption data to television data composed of audio data and video data,
Storage means for storing television data;
Output means for outputting broadcast television data and audio data of the television data stored in the storage means;
Generating means for reproducing audio from the audio data output from the output means, and generating subtitle data by operator input based on the reproduced audio;
Conversion means for converting audio data of broadcast television data output from the output means into subtitle data at a constant conversion speed by voice recognition ;
Synthesizing means for synthesizing the caption data generated by the generating means and the television data for broadcasting output from the output means;
First storage means for storing subtitle data generated by the generation means;
Second storage means for storing the caption data converted by the conversion means;
A first table in which the characters of the caption data generated by the generating means and the time at which the characters are accumulated in the first accumulating means are stored in association with each other;
A second table in which characters of the caption data converted by the conversion means and a time at which the characters are accumulated in the second accumulation means are stored in association with each other;
The time when each character of the caption data generated by the generating means is stored in the first storage means, and each character of the caption data converted by the conversion means is stored in the second storage means The time at the time is monitored, and each character of the subtitle data is stored in the first table or the second table in association with the time at which each character is accumulated in the first accumulation means or the second accumulation means. Means to
It is determined whether or not the subtitle data stored in the second storage unit includes the same continuous character as the predetermined continuous character selected from the subtitle data stored in the first storage unit. Discriminating means to perform,
When it is determined by the determining means that the same continuous character is included, the same character as this character is searched from the first table and the second table, and the time associated with the searched character Reading means for reading from the first table and the second table;
The time read from the first table and the conversion time calculated based on the conversion speed of the conversion means are subtracted from the time read from the second table to calculate the synthesis timing shift time in the synthesis means. A calculation means;
Control means for adjusting the speed of the sound reproduced by the generating means by increasing or decreasing the number of samples of the sound data output from the output means based on the deviation time calculated by the calculating means. Synthetic timing adjustment system.   The control means determines a deviation in the synthesis timing between the television data and the caption data in the synthesis means based on the time calculated by the calculation means, and the timing at which the caption data is synthesized is determined from an appropriate synthesis timing. If it is delayed, the number of samples of the audio data output from the output means is reduced to accelerate the speed of the audio reproduced by the generating means, and when the subtitle data is ahead of the synthesis timing, the output means 7. The synthesis timing adjustment system according to claim 6, wherein the number of samples of audio data to be output is increased to reduce the speed of the reproduced audio.   The control unit decreases or increases the current number of audio data samples stepwise to approach the target number of audio data samples and gradually decreases or accelerates the speed of the sound reproduced by the generation unit. The synthesis timing adjustment system according to claim 6, wherein the synthesis timing adjustment system is controlled as follows.

Images