JP2006100880A - Method, system and program for analyzing caption in multimedia program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a method, a system and a computer program for processing video data. <P>SOLUTION: A data processing system identifies texts in captions of multimedia program data to produce a series of texts, analyzes a series of the texts to produce a result of the analysis, identifies video segments to be altered on the basis of the result of the analysis, generates the identified video segments and alters the identified segments. Further, the system may execute color correction in order to enhance the visibility of the texts in the captions. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates generally to an improved data processing system, and more particularly to a method and apparatus for processing data. More particularly, the present invention relates to a method, apparatus, and computer instructions for processing multimedia program data.

  Personal video recorders (PVRs) are becoming increasingly popular with general consumers. Using these recorders, also called digital video recorders (DVRs), the user can play back recorded programs while recording new programs. In some cases, another program can be recorded on a different channel while watching a program being broadcast on one channel. Further, the user can stop at a certain scene or play back the scene while viewing the program being broadcast. Generally, a PVR is connected to a cable or satellite system to receive digital video and audio content. PVR makes it possible to watch a program after it is broadcast, like a traditional video cassette recorder, but there are many others, such as recording all other broadcasts of a program. You can enjoy the functions. Such systems typically include a hard disk drive and are used to store programs.

  The PVR also provides other functions such as a function of sharing a recorded program with other PVRs via a network, a function of storing a digital image, and a function of storing an MP3 file. However, one of the functions that the PVR does not have is a function that eliminates unpleasant content. In some cases, a user may want to watch a program and at the same time want to eliminate unpleasant content from the program, but such functionality is not currently available.

  Accordingly, it would be advantageous to have an improved method, system, and computer program for managing programs on a PVR.

  The present invention provides a method, system, and computer program for processing video data. Identify text in subtitles in multimedia program data and generate a series of text. The series of texts is analyzed to form an analysis result. Based on the analysis result, a video segment to be modified is identified, an identification video segment is formed, and the identification segment is modified. In addition, corrections can be performed to improve the visibility of text in subtitles.

  The novel features believed characteristic of the invention are set forth in the appended claims. However, the invention itself, as well as preferred modes of use, and other objects and advantages of the invention, are best understood when the following detailed description of the illustrative embodiments is read in conjunction with the accompanying drawings. Let's go.

  With reference now to the figures and in particular with reference to FIG. 1, a block diagram of a data processing system is shown in which the present invention may be implemented. Data processing system 100 is an example of a personal video recorder (PVR), also called a digital video recorder (DVR). As shown, the components within data processing system 100 are interconnected via a bus system 102.

  The data processing system 100 includes a processing unit 104, a memory 106, an audio unit 108, a video unit 110, a communication unit 112, a storage device 114, and a subtitle / video analysis unit 116. The memory 106 stores instructions that can be executed by the processing unit 104 to provide various PVR functions. Such functions include, for example, a function for recording a program, a function for reproducing a program, a function for analyzing video for processing, and a function for managing programs that can be stored in the data processing system 100.

  The audio unit 108 includes components used to receive audio from input and output audio. Such components may include, for example, an audio analog-to-digital converter (ADC) and an audio digital-to-analog converter (DAC). The video unit 110 is used to receive video and output video within the data processing system 100. Video unit 110 may include, for example, an audio-visual (AV) encoder / decoder (codec). The video unit 110 can output a video presented on a display device connected to the data processing system 100 such as the display 118.

  Depending on the specific implementation, the components in the audio unit 108 and the video unit 110 can be implemented in the processing unit 104 as hardware components. The communication unit 112 provides a connection for receiving multimedia programs. In this example, the multimedia program includes video (moving and / or still images, and so on) and audio data. Multimedia programs can also include closed caption data such as closed captions. Such subtitles can be displayed or not displayed depending on user preference. Examples of multimedia programs include television programs, movies, and music videos. These multimedia programs can be obtained by connecting the communication unit 112 to various program sources via the Internet, cable networks, terrestrial digital broadcasting or satellite.

  Storage device 114 provides a place to store multimedia programs. The subtitle / video analysis unit 116 analyzes the text in the subtitles of the multimedia programs and identifies whether to silence, hide the video or delete the segments completely in certain segments of those programs Provide a mechanism to In this way, the user can view the multimedia program in a form that removes a part of the program that may be unpleasant for him.

  The subtitle / video analysis unit 116 decodes the video portion of the multimedia program for processing. Subtitle information is generally assigned to a different channel from the video in the video stream. The caption information is superimposed on the video in the video adapter or unit's frame buffer for presentation when the user wishes to view the caption. This subtitle is also called the closed caption portion of the video.

  The text in the subtitle is identified. The text can be identified in different ways, depending on the specific implementation. In the illustrated example, optical character recognition (OCR: Optical character reader) can be performed on a closed caption portion of a video dedicated to subtitle output. The text output from this process can be entered into a filter to identify portions of the multimedia program that may be uncomfortable.

  In these examples, the filtering is performed using a Bayesian filter, which can be implemented in the caption / video analysis unit 116. Bayesian filtering is currently used in filtering email SPAM. This type of filtering can also be applied to rank different parts of a multimedia program. The Bayesian filter contains text that often appears in a PG (parentalguidance) movie but rarely appears in a G movie in a scene or segment of a multimedia program. If displayed in subtitles, Bayesian inference can be used that the segment of the multimedia program containing the text is likely to be PG designated. When the selection criterion is set to a multimedia program designated by G, the specific scene can be corrected or deleted. For that segment, the video can be invisible, the audio can not be heard, or both mute and erase can be performed.

  In these examples, a video segment is a video portion while one caption is displayed. When a new subtitle is displayed, the segment of the multimedia program is changed to a new segment.

  Information used to filter multimedia programs can be set by a user of the data processing system 100. It is possible to create default setting files for various movie ratings such as G (general: general), PG, PG-13 (under 13 years old, parental guardian is desirable), R (restricted: for adults) it can. Such default files can be stored in the storage device 114. Furthermore, a file provided by a user used for filtering can also be stored in the storage device 114. This user file can come from a variety of sources. For example, an e-mail utility that includes a filtering function can be used as a source. Files used to filter SPAM mail can be downloaded to the data processing system 100. Of course, any external file can be used as this file.

  Furthermore, the caption / video analysis unit 116 can modify the video to improve the readability of the caption. Such corrections can include color corrections that adjust the color of the screen portion where the subtitles appear or adjust the text display in the subtitles. For example, when the color of the text is similar to the background color, characters constituting the text can be outlined with a color different from the background. Furthermore, the background color of the area where the caption is displayed can be changed to provide contrast that makes the caption more readable.

  In these examples, the caption / video analysis unit 116 can be implemented in various forms. For example, the video unit can be implemented as a separate processing unit using a suitable application specific integrated circuit (ASIC) and instructions that perform the functions in the illustrative example of the invention. Alternatively, the caption / video analysis unit 116 can include instructions executed by the processing unit 104 to provide those functions.

  In these examples, the data processing system 100 takes the form of a PVR. This description is not intended to be construed as limiting with respect to the architecture in which the features of the present invention may be implemented. The data processing system 100 may also be implemented using a computer with software and an appropriate adapter card that enables the reception and manipulation of multimedia programs that use the functions found in PVR.

  In this way, the mechanism of the present invention provides the ability to filter parts of multimedia programs. Even if a multimedia program has a rating of being unpleasant throughout, it is possible to watch the program with the unpleasant part removed. The sound can be inaudible, the video invisible, or both mute and erase can be performed.

  Turning now to FIG. 2, there is shown a flowchart of a process for filtering multimedia programs according to a preferred embodiment of the present invention. The process shown in FIG. 2 can be implemented in a filtering system such as the caption / video analysis unit 116 of FIG.

  The process begins with the decoding of the multimedia program (step 200). In these examples, the video stream is received in a format such as MPEG2, MPEG3, or JPEG. In these multimedia files, the audio and video channels are divided into separate channels. A closed caption section including subtitles is assigned to a channel different from that for video and audio. If necessary, the closed caption unit is superimposed on the video to present subtitles.

  This data decoding can be performed using encoder / decoder processing in a component such as a processing unit such as processing unit 104 of FIG. Encoding and decoding may be performed in hardware, such as a logic circuit including encoding and decoding functions, as described in the examples, depending on the specific implementation.

  A segment of the decoded multimedia program data is selected (step 202). In these illustrative examples, a segment of data within multimedia program data is defined as a plurality of frames. Video data is typically presented in 30 frames per segment.

  Next, optical character recognition is performed on the segment of the multimedia program data in order to obtain text from the caption in the closed caption portion of the segment data (step 204). This text is supplied to the filtering algorithm (step 206). Next, a rating is acquired (step 208). The rating of this segment is compared to the selection criteria selected by the user (step 210). This selection criterion can be, for example, a movie rating such as PG-13 or R.

  A determination is made as to whether the segment is appropriate for the selection criteria selected by the user (step 212). For example, if the user has selected PG-13 as an appropriate rating, and the filtering has identified text rated R from the segment, the segment is identified as an inappropriate segment. If the segment is inappropriate, the video and audio are erased or muted appropriately in combination (step 214). This process is performed on the segment. Actually, however, in step 214, erasure or mute is performed for each frame in the segment. The corrected multimedia program data is saved (step 216).

  Next, a determination is made as to whether there are any unprocessed segments (step 218). If there are still unprocessed segments, the process returns to step 202. If there are no more unprocessed segments, the multimedia program data is re-encoded (step 220), the processed multimedia program is saved (step 222), and then the process is terminated.

  In the example shown in FIG. 2, processing occurs for the segment. Of course, processing can be generated in units of frames according to specific implementation. In addition, when encoding and decoding are implemented in hardware, other functions such as Bayesian filtering and the frame buffer can be located in the same hardware unit.

  Turning now to FIG. 3, there is shown a flowchart of a process for performing subtitle color correction according to a preferred embodiment of the present invention. The process shown in FIG. 3 may be implemented in a filtering system such as the caption / video analysis unit 116 of FIG.

  The process begins with the decoding of the multimedia program data (step 300). In this example, the video portion of the multimedia program is not changed. The decrypted data is stored (step 302). A segment of the decoded video data of the multimedia program is selected for processing (step 304). In order to improve the readability of the subtitles in the selected segment, a determination is made as to whether this segment requires color correction (step 306). Depending on the implementation, at step 306, a determination may be made as to whether the text in the subtitle should be obscured or unreadable. This step can be performed to cover up harmful or other unpleasant language. If correction is necessary, correction (eg, color) is performed (step 308). The specific type of correction to perform can vary depending on the implementation. For example, the background of the text or the text can be changed to enhance the contrast between the background and the text. In another example, the text can be bordered with a color that has a greater contrast with the background. In another example, the text font size and font type can be changed.

  Next, a determination is made as to whether there are any unprocessed segments in the video data (step 310). If there are still unprocessed segments, the process returns to step 304. If there are no more unprocessed segments, the data is re-encoded (step 312) and the processed multimedia program is saved for later playback (step 314), after which the process ends. Referring again to step 306, if no correction is required, processing proceeds to step 310 described above.

  Accordingly, the present invention provides an improved method, system, and computer program for filtering multimedia programs. The inventive mechanism in the illustrative example allows one part or segment of the multimedia program to be modified to meet the user's selection criteria, while the other part is left unmodified. In the example shown, such modifications may include erasing the video segment, muting the audio for the segment, or erasing the video and audio for the segment.

  Although the present invention has been described as a data processing system having full functionality, the processing of the present invention can be distributed in the form of instructions and computer readable media in various forms. Those skilled in the art will appreciate that the present invention applies equally regardless of the specific type of signal carrier medium used to implement the. It is important to keep this in mind. Examples of computer readable media include digital and digital media using recordable media such as floppy disks, hard disk drives, RAM, CD-ROMs, DVD-ROMs, and transmission forms such as radio frequency and light wave transmissions. Examples include transmission media such as analog communication links, wired or wireless communication links. The computer readable medium may take the form of an encoded form that is decoded when actually used in a specific data processing system.

  The description of the present invention has been presented for purposes of illustration and description, and is not intended to be exhaustive or limiting to the invention in the form disclosed. Many modifications and variations will be apparent to practitioners skilled in this art. The disclosed embodiments have been chosen and described in order to best explain the principles and practical applications of the invention, and specific uses contemplated by others skilled in the art upon understanding the invention. This is because various embodiments in which various modifications are made so as to be suitable for the above can be implemented.

1 is a block diagram of a data processing system in which the present invention can be implemented. 6 is a flowchart of a process for filtering multimedia programs according to a preferred embodiment of the present invention. 6 is a flowchart of a process for performing subtitle color correction according to a preferred embodiment of the present invention;

Explanation of symbols

100 Data Processing System 102 Bus System 104 Processing Unit 106 Memory 108 Audio Unit 110 Video Unit 112 Communication Unit 114 Storage Device 116 Subtitle / Video Analysis Unit 118 Display

Claims (22)

A method in a data processing system for processing multimedia program data comprising:
Identifying text in subtitles of the multimedia program data to generate a series of text;
Analyzing the series of texts to form analysis results;
Identifying a portion of the multimedia program data to be modified based on the analysis results to form an identification portion;
Modifying the identification portion. Generating the series of text comprises:
The method of claim 1, wherein optical character recognition is performed on subtitles of the multimedia program data to generate the series of texts.   The method according to claim 1, wherein the multimedia program data includes a video component and an audio component, and the identification portion is modified by deleting at least one of the video portion and the audio portion. Forming the analysis results,
The method of claim 1, comprising performing a Bayesian filter on the series of text. Decoding the multimedia program data, performed prior to generating the series of text;
The method of claim 1, further comprising re-encoding the multimedia program data performed after the step of modifying the identification portion.   The method of claim 1, wherein the multimedia program data is a frame or a group of frames.   The method of claim 1, wherein the multimedia program is a movie. A method in a data processing system for processing a multimedia program comprising:
Decoding the multimedia program to form decoded multimedia program data;
Analyzing a portion of the multimedia program data;
Determining whether the readability of subtitles in the portion of the multimedia program data requires improvement;
In response to the readability requiring improvement of the subtitles in the portion of the multimedia program data, performing correction on a portion of the multimedia program data including the subtitles, and Improving the readability of. A data processing system for processing multimedia program data,
Identifying means for identifying text in subtitles of the multimedia program data and generating a series of text;
Analyzing means for analyzing the series of texts to form an analysis result;
Identifying means for identifying a portion of the multimedia program data to be modified based on the analysis result to form an identification portion;
A data processing system including a modification unit that modifies the identification portion.   The data processing system according to claim 9, wherein the multimedia program data includes a video component and an audio component, and the modifying unit modifies by deleting at least one of the video portion and the audio portion of the identification portion. The analysis means is
The data processing system according to claim 9, comprising execution means for performing a Bayesian filter on the series of texts. Decoding means for decoding the multimedia program data before generating a series of text;
The data processing system according to claim 9, further comprising re-encoding means for re-encoding the multimedia program data after modifying the identification portion.   The data processing system according to claim 9, wherein the multimedia program data is a frame or a group of frames. A data processing system for processing multimedia programs,
Decoding means for decoding the multimedia program to form decoded multimedia program data;
Analyzing means for analyzing a portion of the decrypted multimedia program data;
Determining means for determining whether the readability of subtitles in the portion of the decoded multimedia program data requires improvement;
In response to the readability requiring improvement of the subtitles in the portion of the decoded multimedia program data, a correction is made to a portion of the decoded multimedia program data that includes the subtitles. A data processing system including execution means for improving readability of the subtitles. A computer program in a computer readable medium for processing multimedia program data comprising:
A first instruction for identifying text in subtitles of the multimedia program data and generating a series of text;
A second instruction that analyzes the series of texts to form an analysis result;
A third instruction for identifying a part of the multimedia program data to be modified based on the analysis result and forming an identification part;
And a fourth instruction for modifying the identification portion.   The computer program according to claim 15, wherein the multimedia program data includes a video component and an audio component, and the identification portion is modified by deleting at least one of the video portion and the audio portion. The second instruction is
The computer program product of claim 15, comprising sub-instructions that perform a Bayesian filter on the series of texts. A fifth instruction for decoding the multimedia program data before generating the sequence of text;
The computer program according to claim 15, further comprising: a sixth instruction for re-encoding the multimedia program data after modifying the identification part.   The computer program product of claim 15, wherein the multimedia program data is a frame or a group of frames. A computer program in a computer readable medium for processing a multimedia program comprising:
A first instruction for decoding the multimedia program to form decoded multimedia program data;
A second instruction for analyzing a portion of the decrypted multimedia program data;
A third instruction for determining whether the readability of subtitles in the portion of the decoded multimedia program data requires improvement;
In response to the readability requiring improvement of the subtitles in the portion of the decoded multimedia program data, a correction is made to a portion of the decoded multimedia program data that includes the subtitles. And a fourth instruction that executes to improve the readability of the subtitles. A bus system,
A communication unit connected to the bus system;
A memory including a plurality of instructions connected to the bus system;
A processing unit connected to the bus system, wherein the processing unit is made to identify text in subtitles of multimedia program data and generate a series of text, and to analyze the series of text A data processing system comprising: an instruction for forming a result; an instruction for identifying a part of the multimedia program data to be modified based on the analysis result; forming an identification part; and an instruction for modifying the identification part. A bus system,
A communication unit connected to the bus system;
A memory including a plurality of instructions connected to the bus system;
A processing unit connected to the bus system, instructions for causing the processing unit to decode multimedia programs and to form decoded multimedia program data, and instructions for analyzing a part of the multimedia program data And instructions for determining whether the readability of subtitles in the portion of the multimedia program data requires improvement, and improvement of the subtitles in the portion of the multimedia program data is required. In response to the readability, a data processing system including an instruction for executing correction on a part of the multimedia program data including the caption and improving the readability of the caption.

Images