JP2007259121A - Closed caption data processing method, closed caption data processing program and closed caption data processor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a closed caption data processing method, a closed caption data processing program and a closed caption data processor by which more correct sentence is acquired and more useful data is efficiently generated even when a series of character groups are displayed over a plurality of regions. <P>SOLUTION: A display selection closed caption data processing part 705 performs rearrangement processing for every segment when there are two or more lines of closed captions and there are lines including two or more segments in one screen after performing broadcast closed caption ruby processing for deleting segments of ruby matched to ruby specification conditions. In the rearrangement processing, when conditions of continuous segments are satisfied, text data of the segments is transferred so that the segments are arranged immediately after the preceding segment. Thus, text data (closed caption meta data) in which spoken words separated in closed caption data becomes a series of character groups is generated and supplied to a time code addition processing part 707. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a caption data processing method, a caption data processing program, and a caption data processing apparatus for enabling search of a broadcast by using caption information included in a broadcast signal.

  Usually, the data of a program broadcast as a television broadcast includes video data and audio data. The broadcast program data may include subtitle data that allows the user to select whether to display the program data. Such selectable caption data is generally referred to as closed caption, and is mainly developed for the hearing impaired. This subtitle includes not only the performer's conversation but also explanations such as BGM and sound effects.

  In this closed caption, a voice-coded audio code or the like is inserted into the 21st horizontal scanning line of the television signal. The closed caption data can be separated from the television signal by a dedicated decoder.

  A technique relating to a video search device that searches for video based on such closed caption data is disclosed (see, for example, Patent Document 1). In the video search device described in Patent Document 1, upon receiving an instruction for “preparation for search” from the input unit, the video playback unit starts playback of the medium, and the decoder decodes the television signal to obtain closed caption data. And store it in memory. When “search request setting” is received from the input unit, the sentence search unit searches closed caption data in the memory, and the search result is stored in the memory. Further, the search result recorded in the memory is read out and sent to the decoder, which converts it into a television signal and displays it on the video display unit. Thereby, the content of the video can be easily searched using the character information of the closed caption.

Also disclosed is a technique for generating metadata using text data that allows the user to select whether or not to display (see, for example, Patent Document 2). In the information processing apparatus described in Patent Document 2, a broadcast signal is acquired, and identification information that can uniquely distinguish a program is acquired from the broadcast signal. Then, time information and distinction information are added to the broadcast signal. Thereby, it is possible to search using the metadata having the time information corresponding to the text data and the discriminating information for discriminating the program.
Japanese Patent Laid-Open No. 7-212708 (first page) JP 2005-198206 A (first page)

  By the way, on a dialogue screen or the like, a caption of the utterance may be displayed near each speaker. At this time, when the utterances are long, the utterances may be divided and displayed in multiple columns. In this case, when subtitle data is extracted in the scanning direction of the video data to generate text data, the utterances of a plurality of speakers may not be nested without considering the columns. Thus, even if metadata is generated from the text data, accurate data cannot be generated, and it is difficult to perform an accurate search with such metadata.

The present invention has been made to solve the above-described problems, and an object of the present invention is to generate more accurate text data even when a series of character groups are displayed over a plurality of lines. A subtitle data processing method, a subtitle data processing program, and a subtitle data processing apparatus are provided.

  In order to solve the above problem, the invention described in claim 1 is a caption data processing method for processing caption data extracted from a video signal, and acquires the number of lines of caption data included in one screen, A step of performing a multi-line determination process for determining whether or not there are two or more lines, and when there are two or more lines of caption data, in each line of each segment included in each line of the caption data based on a segment determination condition And a connection determination process for determining whether or not a connection determination condition for determining that the segment is in a connection relationship with a segment in another row from the specified position is determined. A rearrangement step of rearranging the segments determined to be connected in the connection determination process so that the segments are adjacent to each other, and an order of the rearranged segments And summarized in that to perform a generating step of generating the text subtitle data in accordance with the.

The invention according to claim 2 is the subtitle data processing method according to claim 1, wherein the segment determination condition is an operation position designation included in the subtitle data.
The invention according to claim 3 is the caption data processing method according to claim 1 or 2, wherein the segment determination condition is a color arrangement designation included in the caption data.

  According to a fourth aspect of the present invention, in the subtitle data processing method according to any one of the first to third aspects, the connection determination condition is included in a line immediately before a line including the first segment. The start position of the first segment is indented or the same as the start position of the second segment, and The color arrangement is the same as the color arrangement of the second segment, and the arrangement changing step is performed when the connection determination condition is satisfied based on the positions of the first and second segments. The gist is to rearrange the second segment after the first segment.

  According to a fifth aspect of the present invention, in the caption data processing method according to any one of the first to fourth aspects, before the multi-line determination process, presence / absence of caption data having a different character size is displayed on the one screen. A character size determination step for determining, and when there is subtitle data having a different character size, a deletion step for determining a ruby condition for each segment including a character with a small character size and deleting a segment that matches the ruby condition; Is further executed.

  According to a sixth aspect of the present invention, in the caption data processing method according to the fifth aspect, as the ruby condition, a character string of a segment is composed of hiragana, katakana, or a blank, and the size of adjacent characters in the same line is The gist is that the next line is used on the condition that it is adjacent to a character of a larger size.

  The invention according to claim 7 is a caption data processing method for processing caption data extracted from a video signal, wherein the number of lines of caption data included in one screen is acquired, and a plurality of determinations are made as to whether or not there are two or more lines. A step of performing a row determination process, and a step of specifying the position of each segment included in each row of the caption data in each row based on a segment determination condition when the caption data includes two or more rows. Performing a group determination process for determining whether or not a group relationship condition is satisfied based on the identified position, and assigning an order to the segments determined to have a group relationship in the group determination process Grouping the segments, rearranging the individual segments in the group to line up in the order, and And summarized in that to perform a generating step of generating the text subtitle data in accordance with the turn.

  The invention according to claim 8 is a caption data processing program for processing caption data extracted from a video signal, wherein the number of lines of caption data included in one screen is acquired and a plurality of judgments are made to determine whether or not there are two or more lines. A means for performing line determination processing; a specification means for specifying a position in each line of each segment included in each line of the caption data based on a segment determination condition when the caption data includes two or more lines; Means for performing a connection determination process for determining whether or not a connection determination condition for determining that the segment is connected to a segment in another row from which the segment is different is determined in the connection determination process. The arrangement changing means for rearranging the segments determined to be adjacent to each other, and the subtitle text data according to the order of the rearranged segments. And summarized in that to function as a generating means for generating data.

  The invention according to claim 9 is a caption data processing program for processing caption data extracted from a video signal, wherein the number of lines of caption data included in one screen is acquired and a plurality of judgments are made to determine whether or not there are two or more lines. A means for performing line determination processing; a specification means for specifying a position in each line of each segment included in each line of the caption data based on a segment determination condition when the caption data includes two or more lines; Means for performing a group determination process for determining whether or not a group relation condition is satisfied based on the determined position, and assigning an order to the segments determined to be in a group relation in the group determination process Means for grouping, means for rearranging the individual segments in the group in order, and And summarized in that to function as a generating means for generating text data of the caption in accordance with the turn.

  The invention according to claim 10 is a caption data processing device that processes caption data extracted from a video signal, and obtains the number of lines of caption data included in one screen and determines whether or not there are two or more lines. A means for performing line determination processing; a specification means for specifying a position in each line of each segment included in each line of the caption data based on a segment determination condition when the caption data includes two or more lines; Means for performing a connection determination process for determining whether or not a connection determination condition for determining that the segment is connected to a segment in another row from which the segment is different is determined in the connection determination process. Repositioning means for rearranging the segments determined to be adjacent to each other, and subtitle text data according to the order of the rearranged segments And summarized in that with a generating means for generating.

  The invention according to claim 11 is a caption data processing device for processing caption data extracted from a video signal, and obtains the number of lines of caption data included in one screen and determines whether or not there are two or more lines. A means for performing line determination processing; a specification means for specifying a position in each line of each segment included in each line of the caption data based on a segment determination condition when the caption data includes two or more lines; Means for performing a group determination process for determining whether or not a group relation condition is satisfied based on the determined position, and assigning an order to the segments determined to be in a group relation in the group determination process Means for grouping, means for rearranging the individual segments in the group so as to line up in the order, and order of the rearranged segments Depending the gist further comprising generating means for generating text data of subtitles.

(Function)
According to the invention described in claim 1, 8 or 10, when there are two or more subtitle data included in one screen, the other lines having different segments from the position in each line of the individual segments of the subtitle data. It is determined whether or not a connection determination condition for determining that the segment has a connection relationship is satisfied. Then, the segments determined to be connected are rearranged so that they are adjacent to each other, and subtitle text data is generated according to the order of the rearranged segments. Usually, when a series of character groups are displayed over a plurality of lines due to the display relationship, they are in a predetermined connection relationship. For this reason, from the position in each row, it is determined whether or not the segments are in a predetermined connection relationship. If it is determined that the segments are in a connection relationship, the segments are adjacent to each other as a series of character groups. Sort by. Thereby, even if it is a case where a series of character groups are displayed over several lines, a series of character groups can be acquired more correctly. Therefore, more useful data can be generated efficiently.

According to the second aspect of the present invention, it is possible to determine whether or not the segment is based on the operation position designation included in the caption data.
According to the third aspect of the present invention, it is possible to determine whether or not the segment is based on the color scheme designation included in the caption data.

  According to the invention of claim 4, the connection determination condition is that the first segment is adjacent to the second segment included in the row immediately preceding the row including the first segment, The start position is indented or the same as the start position of the second segment, and the color scheme of the first segment is the same as the color scheme of the second segment. When this connection determination condition is satisfied, the rearrangement is performed so that the second segment is arranged next to the first segment. In the case of a series of characters, another segment is displayed in the same color on the following line, and the segment immediately after is often “indented” or “left-aligned” for easy viewing. By using these conditions, it is possible to distinguish between a group of characters that are close but not a series of characters and a group of characters that are not, and more reliably determine the connection relationship.

  According to the fifth aspect of the present invention, when there is subtitle data having a different character size on one screen, the ruby condition is determined for each segment including a character having a small character size, and a segment matching the ruby condition is determined. delete. Thereby, since unnecessary ruby is removed as text data, only necessary text data can be extracted and more useful data can be generated.

  According to the invention of claim 6, as the ruby condition, the character string of the segment is composed of hiragana, katakana or white space, and the size of adjacent characters on the same line is not different, and the next line has a larger size. It must be adjacent to the character. Since ruby indicates how to read kanji and the like, it is usually displayed smaller than characters other than ruby and is usually displayed in hiragana or katakana. In addition, a ruby may be configured including a blank for adjusting the position of the ruby. Therefore, it is possible to determine whether the segment is a ruby segment from the character size and position of the other segment. Therefore, unnecessary ruby can be removed more reliably.

  According to the invention described in claim 7, 9 or 11, when there are two or more subtitle data included in one screen, whether or not the group relationship is satisfied from the position in each row of the individual segments of the subtitle data. Determine whether. Then, the segments determined to have the group relationship are rearranged in order, and subtitle text data is generated according to the rearranged segment order. Usually, when a series of character groups are displayed over a plurality of lines due to the display relationship, they are in a predetermined group relationship. Therefore, from the position of each segment, it is determined whether or not it is a series of characters, that is, whether or not the segments are in a predetermined group relationship. As a group, a plurality of segments are grouped in order and rearranged in this order. Thereby, even if it is a case where a series of character groups are displayed over a some area | region, a series of character groups can be acquired more correctly. Therefore, more accurate text data can be generated efficiently.

According to the present invention, even when a series of character groups is displayed over a plurality of areas, the series of character groups can be more accurately acquired as one line, and more accurate text data can be generated. .

  Hereinafter, an embodiment embodying the present invention will be described with reference to FIGS. FIG. 1 is an explanatory diagram for explaining a configuration of a caption data processing apparatus to which the present invention is applied. In this embodiment, as shown in FIG. 1, a broadcast signal from a broadcast station is received using a television receiver 30. A display selection subtitle decoder 40 is connected to the television receiver 30, and a home server 50 is connected to the display selection subtitle decoder 40. The home server 50 is connected to a metadata creation server 70 as a caption data processing device via the Internet I as a network. The metadata creation server 70 is connected to an operator terminal (not shown) as character data input means.

  The broadcasting station 10 is a facility that broadcasts programs using terrestrial waves or satellite waves. The broadcast signal of the broadcast program includes video data and audio data. The video data is moving image data, and the audio data is data relating to audio that is reproduced in synchronization with the video data.

  Also, the broadcast signal includes subtitles that are always displayed as video and subtitles that are displayed by selection. The former includes introductions of program titles and casts, Japanese subtitles in overseas works, etc. On the other hand, subtitle data (so-called closed caption) that can be displayed and selected as in the latter case may include text data corresponding to a performer's conversation, and explanations about broadcast content such as BGM and sound effects. is there. In this way, caption data that can be selected to be displayed or not is referred to as “display selected caption data”.

  Next, the display selection subtitle data will be described. For example, in NTSC analog terrestrial broadcasting, 525 scanning lines are used for video signals. Of these 525 lines, the first 21 lines of each field (one field is composed of 2 fields) is called VBI (Vertical Blanking Interval), and is used as an interval for starting scanning. Assigned. The closed caption is configured to be transmitted by multiplexing a 7-bit character code in the 21st VBI of the VBI of each field. Each field can be used to transmit two types of character sets at about 60 characters / second. This display selection subtitle data is decoded from the video data at the time of reproduction, and can be displayed simultaneously with the video.

  On the user side, the television receiver 30 is used to receive a broadcast signal composed of video data and audio data including display selection subtitle data. The television receiver 30 includes a tuner 31, a signal processing unit 32, and an output unit 33 including a display and a speaker. When the display selection subtitle data is not displayed, the television receiver 30 demodulates the broadcast signal selected by the tuner 31 in the signal processing unit 32. Then, in the output unit 33, among the demodulated broadcast signals, the video signal is output to the display, and the audio signal is output to the speaker. Thereby, the user can view the program.

  The display selection subtitle decoder 40 acquires the broadcast signal selected by the tuner 31 of the television receiver 30 and decodes the video signal and the audio signal. Then, the display selection subtitle decoder 40 extracts display selection subtitle data based on the user's operation input, further decodes it, and outputs it to the output unit 33. Further, the display selection subtitle decoder 40 executes search and summarization processing using the metadata acquired by the home server 50 from the metadata creation server 70 via the Internet I.

FIG. 3 is a configuration block diagram of the display selection subtitle decoder 40. Video signal decoder 4
01 decodes the supplied video signal and supplies the decoded video data to the memory 403. The memory 403 is a frame memory that temporarily holds the supplied video signal. The audio signal decoder 402 decodes the supplied audio signal and outputs the decoded audio data.

  The display selection subtitle data decoder 405 acquires video data from the memory 403 when outputting the received broadcast signal in real time. When the display selection subtitle data is instructed to be displayed, the display selection subtitle data decoder 405 decodes the display selection subtitle data included in the acquired video data and converts the corresponding text data into an OSD (On Screen Display) 406. And the video data to the composition processing unit 407.

  The OSD 406 converts the supplied text data into OSD data that is image data to be displayed superimposed on the display screen, and supplies the OSD data to the composition processing unit 407. The composition processing unit 407 superimposes the supplied OSD data on the supplied video data, and outputs it from the output terminal to the display of the output unit 33 of the television receiver 30. On the other hand, the audio processing unit 408 acquires the audio data decoded by the audio signal decoder 402 and outputs it to the speaker of the output unit 33 of the television receiver 30.

  When the video signal and the audio signal are output to the home server 50 for recording, the output signal generation unit 409 causes the program ID extraction unit 410 to display the program included in the program management data from the video data held in the memory 403. ID is extracted. The output signal generation unit 409 refers to the timer 411, acquires the time (absolute time) at which recording was started, and out of the video data supplied from the memory 403 or the audio data supplied from the audio signal decoder 402 The acquired time information is added to at least one of the above. Further, the program ID supplied from the program ID extraction unit 410 is added to the video data and audio data, and an output signal for recording is generated and supplied to the home server 50. The time information added here is used as a time stamp in search and extraction processing described later.

  Through such processing, the display selection subtitle decoder 40 decodes the display selection subtitle based on the user's operation input and displays it on the video, generates recording data, and supplies it to the home server 50. It is possible to record. Further, the display selection subtitle decoder 40 can execute search and summary processing using time information and program ID added to the video data and audio data.

  On the other hand, the home server 50 receives the recording data with the time stamp generated by the display selection subtitle decoder 40 based on the user's operation input, and records it in the internal data storage means. The home server 50 receives supply of encrypted metadata including text data corresponding to display selection caption data and a time code corresponding to the text data from the metadata creation server 70 via the Internet I. Further, the home server 50 decrypts the encrypted metadata by using the key data, and executes a matching process between the text input by the user and the metadata using the metadata. If the time code corresponding to the text input by the user is detected as a result of the matching process, the recorded data is searched based on the time code and supplied to the display selection subtitle decoder 40.

  FIG. 4 is a block diagram of the home server 50. The operation input unit 501 includes, for example, input devices such as buttons, keys, a touch panel, a touch pad, and a lever, and receives user operation inputs. The recording control unit 502 instructs the display selection subtitle decoder 40 to output a video signal and an audio signal when a broadcast program recording instruction is input from the operation input unit 501. The display unit 504 is composed of, for example, an LCD (Liquid Crystal Display) or a CRT (Cathode Ray Tube), and displays various types of information.

  The recorded data storage unit 52 stores video data and audio data to which the program ID and time stamp supplied from the display selection subtitle decoder 40 are added. For example, the recorded data storage unit 52 may be configured by a large-capacity recording medium such as a hard disk or a removable recording medium such as a DVD (Digital Versatile Disk) or a magnetic tape.

  The network IF unit 503 is an interface that performs communication via the Internet I. Here, data is transmitted to and received from the metadata creation server 70. The encrypted metadata storage unit 53 stores the encrypted metadata acquired from the metadata creation server 70 via the network IF unit 503. The key data storage unit 54 stores a decryption key in advance in order to decrypt the encrypted metadata. The decryption key is provided and recorded from the metadata creation server 70 via the network IF unit 503 for each program ID. The decryption processing unit 505 decrypts the encrypted metadata recorded in the encrypted metadata storage unit 53 using the decryption key stored in the key data storage unit 54 and records it in the metadata storage unit 55. To do.

  When the search target program ID and the text as the search key are input to the operation input unit 501, the matching processing unit 506 refers to the metadata recorded in the metadata storage unit 55 and performs the matching process. Execute. Then, when the text including the search key is specified, the matching processing unit 506 supplies the time code associated with the text to the recorded data search processing unit 507.

  Here, when only the program ID is specified in the operation input unit 501, the matching processing unit 506 extracts the metadata recorded in association with the program ID from the metadata storage unit 55. The time code and program ID included in this metadata are used to search the recorded data storage unit 52. Therefore, the extracted time code is supplied to the recorded data search processing unit 507.

  The recorded data search processing unit 507 searches the recorded data storage unit 52 based on the matching result (program ID and time code) supplied from the matching processing unit 506. Then, in recording with the program ID, the image specified by the time code is supplied to the display selection subtitle decoder 40.

On the other hand, the metadata creation server 70 acquires a broadcast signal with display-selected caption data created by the broadcast station 10 by receiving it via various networks and radio waves. Then, the metadata creation server 70 creates metadata using the broadcast signal, and encrypts the created metadata. Further, the metadata creation server 70 distributes the encrypted metadata to users via the Internet I. The metadata creation server 70 includes control means including a CPU (Central Processing Unit) as control means, and data storage means such as ROM (Read Only Memory), RAM (Random Access Memory), and HDD (Hard Disk Drive). .

  Next, the configuration of the metadata creation server 70 will be described in detail. FIG. 5 shows a block configuration of the metadata creation server 70. The CPU of the metadata creation server 70 executes a subtitle data processing program to perform a multi-line determination process, a specific stage, a connection determination process, an arrangement change stage, a generation stage, a character size determination stage, and a deletion Perform stages and so on. As a result, the metadata creation server 70 implements the functions shown in the following functional block diagram.

  The broadcast signal acquisition unit 701 acquires a broadcast signal via a network or broadcast radio wave and supplies it to the decoder 702. The decoder 702 decodes the broadcast signal supplied from the broadcast signal acquisition unit 701. Here, the decoder 702 decodes only the video signal including the program management information including the program ID information and the display selection subtitle data necessary for creating the metadata among the broadcast signals.

  The program ID information extraction unit 703 extracts program ID information that can specify a broadcast program from the program management data included in the video data decoded by the decoder 702 and supplies the extracted program ID information to the metadata generation unit 708. Further, the program ID information extraction unit 703 supplies this video data to the display selection subtitle data processing unit 705. The display selection subtitle data processing unit 705 functions as a means for performing a multi-line determination process, a specifying means, a means for performing a connection determination process, an arrangement changing means, a generating means, a character size determining means, and a deleting means, and acquires acquired video data The display selection subtitle data included in is decoded and subtitle metadata is generated.

  Next, a caption processing method according to the present invention in the display selection caption data processing unit 705 will be described with reference to FIGS. Here, the display selection subtitle data processing unit 705 repeatedly performs subtitle processing for each screen.

  The display selection subtitle data processing unit 705 first decodes subtitle data including 8-unit codes included in the video data decoded by the decoder 702. The subtitle data includes information such as a control code such as a character display position and size, and text data of the character. The display selection subtitle data processing unit 705 first performs broadcast subtitle ruby processing using the decoded subtitle data (step S1-1). This broadcast subtitle ruby process is a process for deleting broadcast subtitle ruby from text data.

  This broadcast subtitle ruby process will be described in detail with reference to FIG. In this broadcast subtitle ruby process, the display selection subtitle data processing unit 705 first determines whether or not the sizes of characters in the screen are all the same (step S2-1). If all the characters in the screen have the same size (in the case of “YES” in step S2-1), the screen does not include ruby, and the broadcast subtitle ruby processing ends.

  If the sizes of the characters in the screen are not all the same (in the case of “NO” in step S2-1), the display selection subtitle data processing unit 705 determines whether the segment matches the ruby identification condition for each segment. Is determined (step S2-2). Here, the display selection subtitle data processing unit 705 specifies a segment composed of a series of character groups from control codes such as designation of an operation position included in the character string and designation of a color scheme such as a character color or background color of the character string. To do. It is possible to determine whether or not the segment is based on the operation position designation and the color scheme designation included in the caption data.

  Then, it is determined whether this segment is ruby. In this case, since ruby indicates how to read kanji and the like, it is usually displayed smaller than characters other than ruby and is displayed in hiragana or katakana. In addition, a ruby may be configured including a blank for adjusting the position of the ruby. Therefore, in this embodiment, as ruby specifying conditions for specifying ruby, (1) the character size of different characters is a small size, and (2) the segment is “Hiragana”, “Katakana”, or “blank”. (3) The standard size character is not adjacent on the same line, and (4) the next line is adjacent to the standard size character. For example, (1) is determined using a character size designation control code or the like. Further, (2) is determined using a character code or the like. Further, (3) and (4) are determined using a position designation control code, a character size designation control code, and the like.

  If the segment does not match all the above four ruby identification conditions (“NO” in step S2-2), the display selection subtitle data processing unit 705 determines the next segment. Do.

  On the other hand, if the segment satisfies all of the four ruby identification conditions described above (“YES” in step S2-2), the display selection subtitle data processing unit 705 determines that it is ruby, and this The segment is deleted (step S2-3).

  If another segment remains in the line from which the segment is deleted, the display selection subtitle data processing unit 705 performs the processing from step S2-2 onward for the other segment in the same line (step S2- “YES” in 4). Here, whether or not another segment remains in the same line is determined using a position-designated control code, a character size-designated control code, or the like. On the other hand, if no segment remains (in the case of “NO” in step S2-4), the line is deleted (step S2-5).

  The display selection subtitle data processing unit 705 executes the broadcast subtitle ruby process for performing the processes of steps S2-2 to S2-5 for each segment for each screen (step S1-1), and the data other than the data determined to be ruby. Extract subtitle data.

  Next, processing after the broadcast subtitle ruby processing will be described with reference to FIG. The display selection subtitle data processing unit 705 determines whether or not there is a subtitle of two or more lines on one screen (step S1-2). Here, when there are no two or more subtitles on the screen (in the case of “NO” in step S1-2), the subtitle processing on the screen is terminated and the subtitle processing is performed on the next screen.

  On the other hand, when there are two or more lines of subtitles on the screen (in the case of “YES” in step S1-2), it is determined whether or not there is a line including two or more segments on this screen (step S1- 3). Specifically, for each line having two or more subtitles, it is determined whether or not two or more segments are included from a position designation control code, a character size designation control code, or the like. Here, when there is no row including two or more segments (in the case of “NO” in step S1-3), the caption processing on the screen is terminated and caption processing is performed on the next screen.

  If there are two or more subtitles on the screen and there are two or more segments (“YES” in step S1-3), rearrangement processing is performed (step S1-4). In this rearrangement process, the sufficiency of the continuous segment condition shown in FIG. 8 is determined for each segment (step S3-1). In this embodiment, as the continuous segment condition, “adjacent to the segment in the previous line”, “indentation” or “left alignment”, and “displayed in the same color” are used.

  Specifically, the display selection caption data processing unit 705 first uses the position designation control code, the character size designation control code, and the like to display the display position of the segment that is the determination target of the satisfaction of the continuous segment condition, Compare the position of each segment in the previous line. As a result, it is assumed that it is determined that the segment to be determined is adjacent to one of the segments on the immediately preceding line and “indentation” or “left alignment” is performed on this segment. In this case, using the control code for the color scheme, the character color of the segment to be determined is adjacent to the segment of the immediately preceding line that has been determined to be “indented” or “left aligned”. Compare text color. If it is determined by this comparison that the character color is the same color, all of the continuous segment conditions are satisfied. If it is not adjacent to the previous line segment, is located to the left of the previous line segment, or is displayed in a different color, all elements of the continuous segment condition are not satisfied. Therefore ("NO" in step S3-1), the process for this segment is terminated, and the process for the next segment is performed.

  On the other hand, when the continuous segment condition is satisfied (in the case of “YES” in step S3-1), the text data of this segment is moved so that this segment is arranged immediately after the preceding segment (step S3- 2). As a result, the process for this segment is completed, and the process for the next segment is performed. Then, when the process is performed for all the segments included in the screen, the rearrangement process is completed (step S1-4).

  When the rearrangement process is completed, the display selection subtitle data processing unit 705 notifies the operator terminal of the rearrangement detection (step S1-5). As a result, the operator confirms the caption metadata generated by the sorting process (step S1-4) performed at the operator terminal. In the operator terminal, manual correction is performed as necessary (step S1-6). Here, when the operator satisfies the continuous segment condition although it is not a series of character groups, they are rearranged as a series of character groups. Specifically, there is a case where each speaker speaks twice or more on one screen, and they are displayed to satisfy the continuous segment condition. If text data is generated as it is, it becomes subtitle metadata in which different utterances of the same speaker are connected. In such a case, at the operator terminal, the text data is manually moved so that the text data of the segment moved as a series of character groups is returned to the original order.

  And the process of step S1-1-S1-6 mentioned above is repeated for every screen, and if the process about all the screens is completed, a caption data detection process will be completed and caption data will be produced | generated.

  Next, the caption data detection process described above will be described with a specific example. As shown in FIG. 9, the display data includes three lines of caption data. Segment A is included in the first row, and two segments (B, C) are included in the next second row. Furthermore, segment D is included in the third row. Here, it is assumed that the characters constituting each segment (A, B, C, D) have the same size. In addition, a series of characters spoken by each speaker includes control codes that are displayed in the same color, that is, segment A and segment C are displayed in the same color, and segment B and segment D are displayed in the same color. To do.

  First, in the caption data processing, broadcast caption ruby processing (step S1-1) is performed. In this case, since the characters constituting each segment (A, B, C, D) are all configured to have the same size, it is determined that the sizes of the characters in the screen are all the same (in step S2-1, “ YES ”), broadcast subtitle ruby processing does not delete data.

  In the caption data shown in FIG. 9, there are two or more lines of captions on one screen (“YES” in step S1-2). This screen includes a line including two or more segments (B, C) (“YES” in S1-3). For this reason, a rearrangement process is performed (step S1-4).

  Here, first, for the segment A included in the first line of the caption on this screen, the display selection caption data processing unit 705 determines whether the continuous segment condition is satisfactory (step S3-1). Since there is no segment in the immediately preceding line of segment A and the continuous segment condition is not satisfied, it is determined whether or not the next segment satisfies the continuous segment condition (step S3-1).

Here, the next segment is the segment B of the first segment included in the next row of the segment A. Therefore, the display selection subtitle data processing unit 705 performs the following operation on segment B.
Satisfaction of the continuous segment condition is determined (step S3-1). In this case, as shown in FIG. 9, since there is no character adjacent to the previous line for segment B, the condition “adjacent to the character on the previous line” is not met, and the continuous segment condition is not satisfied. Therefore, the display selection subtitle data processing unit 705 determines “NO” in step S 3-1, and determines whether or not the next segment C satisfies the continuous segment condition (step S 3-1).

  Here, the segment C is adjacent to the segment A in the immediately preceding row, and the segment A is indented and displayed in the same color as the segment A. Therefore, the display selection subtitle data processing unit 705 assumes that the continuous segment condition is satisfied (“YES” in step S3-1), and the text of the segment C is arranged so that the segment C is arranged immediately after the preceding segment A. Data is moved (step S3-2). As a result, segment C does not follow segment B but follows segment A.

  Then, the display selection subtitle data processing unit 705 determines whether the next segment, that is, the segment D in the line immediately after the line including the segments B and C, satisfies the continuous segment condition (step S3-1). Here, the segment D is adjacent to the segment B in the immediately preceding row, and the segment B is indented and displayed in the same color as the segment B. Therefore, the display selection subtitle data processing unit 705 determines that the text data of the segment D is arranged immediately after the preceding segment B, assuming that the continuous segment condition is satisfied (“YES” in step S3-1). Is moved (step S3-2). In this case, since the segment C has already moved to be behind the segment A, the segment D follows the segment B. Accordingly, the display selection subtitle data processing unit 705 confirms that the text data has been moved and ends the processing.

  As described above, when the process for the last segment D is finished, the rearrangement process (step S1-4) for the screen composed of the caption data included in the caption broadcast signal for screen display shown in FIG. 9 is finished. Thereby, text data (caption metadata) in which words separated in the caption data become a series of character groups can be generated. Then, the display selection subtitle data processing unit 705 notifies the operator terminal of detection of the rearrangement position (step S1-5), and manually corrects it as necessary (step S1-6). Thus, the caption data detection processing in the display selection caption data processing unit 705 is completed. Then, the display selection subtitle data processing unit 705 supplies the subtitle metadata generated by the subtitle data detection processing to the time code addition processing unit 707.

  As shown in FIG. 5, when receiving a registration instruction for subtitle metadata generated by the display selection subtitle data processing section 705, the time code addition processing section 707 uses a timer 706 to set the time when the registration instruction is received. Add as code. For example, in the case of text corresponding to a display selection subtitle, a time code corresponding to the start time of the display selection subtitle is added. When the broadcast signal acquisition unit 701 acquires a broadcast signal in real time for the broadcast, the time code addition processing unit 707 adds the time code to the text data based on the current time indicated by the timer 706. If the time code addition time is delayed with respect to the program broadcast time, the time code addition processing unit 707 corresponds to the broadcast time of the program based on the delay time and the current time indicated by the timer 706. The time code is calculated and added to the text data.

The metadata generation unit 708 generates metadata by adding the program ID information supplied from the program ID information extraction unit 703 to the text data to which the time code supplied from the time code addition processing unit 707 is added. As shown in FIG. 2, the metadata includes a time code in which the start time of the text group is written to the text data. And
The metadata to which the program ID information supplied from the program ID information extraction unit 703 is added is supplied to the encryption processing unit 709.

  The encryption processing unit 709 is connected to a key data storage unit 72 that stores an encryption key for each program ID. The encryption processing unit 709 extracts the encryption key from the key data storage unit 72 based on the program ID included in the metadata supplied from the metadata generation unit 708, and uses this encryption key to extract the metadata. Encryption is performed and recorded in the encrypted metadata storage unit 73. The encrypted metadata storage unit 73 records the video information file based on the searched characters. The encrypted metadata is provided to the home server 50 via the Internet I from the communication unit 710 as a transmission unit for each program in response to a request from the user.

According to this embodiment, the following effects can be obtained.
○ In the present embodiment, the display selection subtitle data processing unit 705 has two or more subtitles on one screen and a row including two or more segments (in the case of “YES” in step S1-3) ) Performs a rearrangement process (step S1-4). In this process, if “indentation” or “left alignment” is performed adjacent to the segment in the immediately preceding line and displayed in the same color, it is determined that the condition for the continuous segment is satisfied (step “YES” in S3-1). Then, the display selection subtitle data processing unit 705 moves the text data corresponding to the segment so as to arrange this segment immediately after the preceding segment (step S3-2). Then, by performing the rearrangement process (step S1-4) for all the segments for each screen, text data (captions) in which the character groups separated in the caption data included in the caption broadcast signal become a series of character groups. Metadata). Normally, when a series of character groups are displayed over a plurality of lines due to display relationships, their positions are adjacent to each other. For this reason, from the position in each line, it is determined whether or not they are adjacent segments. If it is determined that they are adjacent, they are automatically rearranged so that the segments are adjacent to each other as a series of character groups. be able to. Furthermore, since the caption metadata is generated from the text data generated in this way, even when a series of character groups are displayed over a plurality of lines, the series of character groups is more accurately acquired as one line, More accurate text data can be generated.

  In the present embodiment, broadcast subtitle ruby processing (step S1-1) is performed before the rearrangement processing (step S1-4). In this broadcast subtitle ruby process, the display selection subtitle data processing unit 705 determines that a segment is generated for each segment when the sizes of characters in the screen are not all the same (in the case of “NO” in step S2-1). It is determined whether or not the ruby specific conditions are met (step S2-2). If all of these ruby specific conditions are met (in the case of “YES” in step S2-2), the display selection subtitle data processing unit 705 determines that the ruby is for the next line, and this segment. Is deleted (step S2-3). If another segment remains in the line from which the segment is deleted, the display selection subtitle data processing unit 705 performs the processing from step S2-2 onward for the other segment in the same line, and the segment remains. If not ("NO" in step S2-4), the line is deleted (step S2-5). By deleting the segment that matches the ruby condition in this way, it is possible to remove unnecessary ruby as text data, so that only necessary text data can be extracted to generate text data. .

○ In this embodiment, (1) the character size of different characters is a small size, (2) the segment is composed only of “Hiragana”, “Katakana” or “blank”, and (3) If it is not adjacent in the same line and (4) is adjacent to a standard size character in the next line, it is specified as ruby. Therefore, by using these four ruby specifying conditions, it is possible to automatically determine whether or not the segment is a ruby segment from the character size and position of the other segment.

  ○ In the broadcast subtitle ruby process of the present embodiment, the display selection subtitle data processing unit 705 determines ruby identification when all the characters in the screen have the same size (in the case of “NO” in step S2-1). It is not determined whether or not the conditions are matched (step S2-2). Therefore, the broadcast subtitle ruby process (step S1-1) is performed only on the screen where ruby exists, so that more accurate text data can be generated efficiently.

  In the present embodiment, the display selection subtitle data processing unit 705 performs rearrangement processing (step S1) when there are two or more subtitles even if ruby is deleted and there are two or more segments. -4) is performed to notify the operator terminal of the rearrangement detection (step S1-5). Even if ruby is deleted by executing the rearrangement process (step S1-4), in this case, the continuous segment condition is satisfied even though it is not a series of character groups, and text data is moved as a series of character groups. (Step S3-2). Accordingly, it is possible to notify the operator terminal of the screen on which the rearrangement process (step S1-4) has been performed and make corrections, so that more accurate text data can be generated. In addition, if the ruby is deleted and the subtitles become one line, the rearrangement process is not performed, and in this case, the operator terminal is not notified. Therefore, more accurate caption metadata can be generated only by the operator confirming only a screen with a possibility of error.

Moreover, you may change the said embodiment as follows.
In the above embodiment, as the ruby determination condition, (1) the character size of different characters is a small size, (2) the segment is composed only of “Hiragana”, “Katakana”, or “blank”, and (3 Four conditions were used: a) not adjacent to a standard size character on the same line, and (4) adjacent to a standard size character on the next line. Not limited to this, other determination conditions may be used as ruby determination conditions. For example, if there is a kanji or foreign character that can be read in the same screen at a position immediately below a segment composed of only “Hiragana”, “Katakana”, or “Blank”, it may be determined as ruby.

  In the above embodiment, as the condition for the continuous segment, three conditions are “adjacent to the segment in the previous line”, “indentation” or “left alignment”, and “displayed in the same color”. Was used. However, the present invention is not limited to this, and it may be determined whether or not segments are continuous under other conditions. For example, the segment is continuous if it is adjacent to the segment in the previous line and is the same color, or if the segment in the previous line is “indented” or “left-justified” and is the same color May be determined.

  In the above embodiment, in the rearrangement process (step S1-4), whether the determination target segment for determining whether or not the continuous segment condition is satisfied satisfies the condition in relation to the segment in the immediately preceding row. Was determined (step S3-1). Instead of this, it may be determined whether or not the determination target segment satisfies the condition in relation to the segment in the immediately following row. In this case, as a condition of the continuous segment, for example, whether or not it is adjacent to the segment of the immediately following line, whether the segment of the immediately following line is “indented” or “left aligned” with respect to the determination target segment , Three conditions of the same color are used.

In the above-described embodiment, in the rearrangement process (step S1-4) performed for each row, it is determined for each segment whether or not the segment satisfies the continuous segment condition (step S3-1). Instead, rearrangement may be performed by other methods. Specifically, the display selection subtitle data processing unit 705 is caused to function as means for performing multi-line determination processing, specifying means, means for performing group determination processing, means for grouping, means for rearranging, and generation means. For example, for a segment that satisfies the continuation condition, a group number that can be recognized as a series of character groups and the order in the group (what number segment is) are temporarily recorded. Then, when group numbers and order are assigned to all segments in one screen, the segments are rearranged for each group number and in the order in the group. Furthermore, text data is generated in the order of the groups on the screen according to the display position of the first segment of each group. Even in this case, from the display position of each segment, it is determined whether or not a series of character groups are adjacent to each other over a plurality of lines, and a series of character groups can be obtained more accurately. Can do. Therefore, more accurate text data can be generated efficiently.

  In the above embodiment, when there are two or more subtitles and there is a line including two or more segments (in the case of “YES” in step S1-3), the operator terminal is notified of the detection of the rearrangement position. (Step S1-5). Instead, in the sorting process in step S1-3, the text data is moved so that the continuous segment is actually arranged immediately after the preceding segment (step S3-2). You may notify the detection of replacement. Specifically, when step S3-2 is executed, a notification flag indicating that rearrangement has been performed is recorded. When the rearrangement process (step S1-4) is completed for all segments included in the screen, the display selection subtitle data processing unit 705 determines whether or not the notification flag is recorded, and is recorded. Only in such a case, the detection of the rearrangement position is notified to the operator terminal (step S1-5). Accordingly, when the rearrangement process (step S1-4) is not performed even on a screen including subtitles of two or more lines, notification to the operator terminal can be omitted, and the load on the operator can be reduced. .

  In the above embodiment, the display selection caption data processing unit 705 of the metadata creation server 70 performs the broadcast caption ruby process and the rearrangement process. The hardware that executes these processes is not limited to this, and an analysis processing unit that performs external character processing may be provided on the broadcast receiver side, for example, the home server 50.

  In the above embodiment, the time code is used as the pointer information for specifying the image based on the metadata. However, the time code is not limited to this, and it is also possible to use the frame data allocated for each screen. In this case, the caption data processing apparatus includes a frame counter, and uses frame data as pointer information when recording metadata. Then, when creating a digest or playing back a recording, this frame data can be used to specify an image and a playback start position. In this case, the influence of the time shift in the time code can be eliminated.

  In the above embodiment, the display selection subtitle data processing unit 705 provides subtitle metadata including text data generated by the rearrangement process. However, the search-converted text file and the text-converted display file are stored in the user's A combination may be provided upon request.

The system schematic of embodiment of this invention. Explanatory drawing of the metadata of embodiment of this invention. Explanatory drawing of the display selection subtitle decoder of embodiment of this invention. Explanatory drawing of the home server of embodiment of this invention. Explanatory drawing of the block configuration of the metadata production server of embodiment of this invention. Explanatory drawing of the process sequence of the caption data detection process in embodiment of this invention. Explanatory drawing of the process sequence of the broadcast caption ruby process in embodiment of this invention. Explanatory drawing of the process sequence of the rearrangement process in embodiment of this invention. Explanatory drawing explaining the data rearrangement process in embodiment of this invention.

Explanation of symbols

  A, B, C, D ... segment, 10 ... broadcasting station, 30 ... recording device, 40 ... display subtitle decoder, 50 ... home server, 70 ... metadata creation server as subtitle data processing device, 705 ... multiple line determination processing A display selection subtitle data processing unit as a means for performing, a specifying means, a means for performing connection determination processing, an arrangement changing means, a generation means, a character size determination means and a deletion means

Claims (11)

A caption data processing method for processing caption data extracted from a video signal,
Obtaining a number of lines of subtitle data included in one screen and performing a multiple line determination process for determining whether or not there are two or more lines;
When the caption data has two or more lines, a specific stage for identifying the position of each segment included in each line of the caption data in each line based on a segment determination condition;
Performing connection determination processing for determining whether or not a connection determination condition is determined to be a connection relationship with respect to a segment in another row from which the segment is different from the identified position;
An arrangement change stage in which the segments determined to be connected in the connection determination process are rearranged to be adjacent to each other;
A subtitle data processing method, comprising: generating a subtitle text data in accordance with the rearranged segment order.   2. The caption data processing method according to claim 1, wherein the segment determination condition is an operation position designation included in the caption data.   The subtitle data processing method according to claim 1, wherein the segment determination condition is designation of a color scheme included in the subtitle data. The connection determination condition is:
The first segment is adjacent to the second segment contained in the row immediately preceding the containing row;
The start position of the first segment is indented or the same as the start position of the second segment, and the color scheme of the first segment is the same as the color scheme of the second segment There is,
In the arrangement changing step, the second segment is arranged after the first segment when the connection determination condition is satisfied based on the positions of the first and second segments. The subtitle data processing method according to any one of claims 1 to 3, wherein the subtitle data is rearranged into the subtitle data. Before the multi-line determination process,
A character size determination step for determining the presence or absence of subtitle data having a different character size on the one screen;
When there is subtitle data having a different character size, a ruby condition is determined for each segment including a character having a small character size, and a deletion step of deleting a segment that matches the ruby condition is further performed. The subtitle data processing method according to any one of claims 1 to 4. As the ruby condition,
The segment string consists of hiragana, katakana, or white space,
The size of adjacent characters on the same line is not different,
6. The caption data processing method according to claim 5, wherein the next line is used on condition that the character is adjacent to a character of a larger size. A caption data processing method for processing caption data extracted from a video signal,
Obtaining a number of lines of subtitle data included in one screen and performing a multiple line determination process for determining whether or not there are two or more lines;
When the caption data has two or more lines, a specific stage for identifying the position of each segment included in each line of the caption data in each line based on a segment determination condition;
Performing a group determination process for determining whether or not a group relation condition is satisfied based on the identified position;
A step of grouping the segments by assigning an order to the segments determined to be in a group relationship in the group determination process;
Rearranging the individual segments in this group in the order described above;
A subtitle data processing method, comprising: generating a subtitle text data in accordance with the rearranged segment order. A caption data processing program for processing caption data extracted from a video signal,
Means for obtaining a number of lines of subtitle data included in one screen and performing a multi-line determination process for determining whether or not there are two or more lines;
When the subtitle data has two or more lines, based on a segment determination condition, specifying means for specifying the position of each segment included in each line of the subtitle data in each line;
Means for performing connection determination processing for determining whether or not a connection determination condition for determining that the segment is connected to a segment in another row from which the segment is different is satisfied.
It is made to function as an arrangement changing means for rearranging the segments determined to be in a connection relationship in the connection determination processing so as to be adjacent to each other, and a generating means for generating subtitle text data according to the order of the rearranged segments. A subtitle data processing program characterized by the above. A caption data processing program for processing caption data extracted from a video signal,
Means for obtaining a number of lines of subtitle data included in one screen and performing a multi-line determination process for determining whether or not there are two or more lines;
When the subtitle data has two or more lines, based on a segment determination condition, specifying means for specifying the position of each segment included in each line of the subtitle data in each line;
Means for performing group determination processing for determining whether or not a group relation condition is satisfied based on the specified position;
Means for grouping the segments by assigning an order to the segments determined to be in a group relationship in the group determination processing;
A subtitle data processing program that functions as means for rearranging the individual segments in the group so as to be arranged in the order, and generating means for generating subtitle text data in accordance with the order of the rearranged segments. . A caption data processing device for processing caption data extracted from a video signal,
Means for obtaining a number of lines of subtitle data included in one screen and performing a multi-line determination process for determining whether or not there are two or more lines;
When the subtitle data has two or more lines, based on a segment determination condition, specifying means for specifying the position of each segment included in each line of the subtitle data in each line;
Means for performing connection determination processing for determining whether or not a connection determination condition for determining that the segment is connected to a segment in another row from which the segment is different is satisfied.
Arrangement changing means for rearranging the segments determined to be in the connection relation in the connection determination processing so as to be adjacent to each other, and generation means for generating subtitle text data according to the order of the rearranged segments A closed caption data processing apparatus. A caption data processing device for processing caption data extracted from a video signal,
Means for obtaining a number of lines of subtitle data included in one screen and performing a multi-line determination process for determining whether or not there are two or more lines;
When the subtitle data has two or more lines, based on a segment determination condition, specifying means for specifying the position of each segment included in each line of the subtitle data in each line;
Means for performing group determination processing for determining whether or not a group relation condition is satisfied based on the specified position;
Means for grouping the segments by assigning an order to the segments determined to be in a group relationship in the group determination processing;
Subtitle data processing apparatus comprising: means for rearranging individual segments in this group so as to be arranged in the order; and generation means for generating subtitle text data in accordance with the order of the rearranged segments .

Images