WO2010069171A1

WO2010069171A1 - Mobile terminal and method for implementing television caption dynamic scaling

Info

Publication number: WO2010069171A1
Application number: PCT/CN2009/072563
Authority: WO
Inventors: 张晓勇
Original assignee: 中兴通讯股份有限公司
Priority date: 2008-12-19
Filing date: 2009-06-30
Publication date: 2010-06-24
Also published as: CN101437121A

Abstract

A mobile terminal for implementing television caption dynamic scaling is provided, the mobile terminal includes: a caption-image combining module, and the modules which connect to said caption-image combining module: a mobile terminal television function module, a caption processing module and a user operation analyzing module. The user operation analyzing module is used for analyzing the current caption size changing instruction launched by a user and sending this instruction to the caption-image combining module and the mobile terminal television function module respectively; the mobile terminal television function module is used for receiving mobile terminal television programs, decoding and gaining the mobile terminal television program image frames, once the mobile terminal television function module receives the instruction, then sends the mobile terminal television program image frames to the caption-image combining module; the caption processing module is used for sending the captions in the mobile terminal television programs to the caption-image combining module; the caption-image combining module is used for changing the size of the captions in the mobile terminal television programs according to the instruction and superposing the changed captions on the mobile terminal television program image frames.

Description

Mobile terminal and method for realizing dynamic subtitle of television subtitle

Technical field

The present invention relates to the field of television based on mobile terminals, and more particularly to a mobile terminal and method for realizing dynamic zooming of television subtitles.

Background technique

With the continuous popularization of the third generation of mobile communication technologies, multimedia applications based on mobile terminals are becoming more and more popular, especially the program information of mobile terminal televisions is becoming more and more abundant and diversified. Among them, most mobile TV programs need to display subtitles.

At present, the manner of dynamically displaying subtitles on a mobile terminal television program can be divided into two types. One method is that subtitles and video images are synthesized on the terminal side, which is referred to as a terminal synthesis subtitle. In this case, the network side and the terminal side need to cooperate together. The network side separately transmits the subtitle to be displayed and the video image of the mobile terminal television program to the terminal side, and is synthesized by the terminal side, wherein the network side transmits the UTF8 encoding of the subtitle to the terminal side in real time according to the progress of the television video image playing of the mobile terminal;

Alternatively, the subtitle to be displayed and the video image of the mobile terminal have been synthesized in advance on the network side, which is simply referred to as source synthesized subtitle, and the synthesized video image is transmitted as complete image data to the terminal side for display.

However, due to the limitation of the screen size of the mobile terminal, the subtitles of the mobile terminal television programs are relatively small, especially in foreign language programs, dialect programs, cultural programs, teaching programs, and the small subtitles cause the visibility of the mobile terminal television programs to be extremely poor. .

Therefore, there is an urgent need to propose a mobile terminal and method for realizing dynamic zooming of mobile terminal television subtitles.

Summary of the invention

The technical problem to be solved by the present invention is to provide a mobile terminal and method for dynamically zooming TV subtitles, thereby dynamically scaling subtitles displayed on the mobile terminal television to achieve excellent visibility. In order to solve the above problem, the present invention discloses a mobile terminal for realizing dynamic zooming of a television subtitle, wherein the mobile terminal comprises: a word map synthesis module, and a mobile terminal television function module and a subtitle connected to the word map synthesis module. a processing module and a user operation parsing module, wherein the user operation parsing module is configured to parse a subtitle size command currently initiated by the user, and send the user instruction to the word graph synthesizing module and the mobile terminal television function respectively Module

The mobile terminal television function module is configured to receive a mobile terminal television program, decode and obtain a mobile terminal television program image frame, and send the mobile terminal television program image frame to the word map when receiving the user instruction Synthesis module

The subtitle processing module is configured to send a subtitle in a mobile terminal television program to the word map synthesis module;

The word map synthesizing module is configured to change a subtitle size in the mobile terminal television program according to the user instruction, and superimpose the changed subtitle on the mobile terminal television program image frame.

Further, the mobile terminal television function module is connected to the subtitle processing module, wherein the mobile terminal television function module is further configured to send a message to query whether the network side separately transmits the subtitle of the mobile terminal television program, if And the mobile terminal television function module notifies the subtitle processing module to receive the subtitle;

The subtitle processing module is further configured to separately receive a subtitle of the mobile terminal television program upon receiving the notification of the mobile terminal television function module, and send the subtitle to the word map synthesis module.

Preferably, the subtitle processing module is further configured to separately receive subtitles of the mobile terminal television program, and send the subtitles to the word map synthesis module.

Further, after the mobile terminal television function module sends the message to the network side, if the network side confirms that the subtitle of the mobile terminal television program is not separately transmitted, the mobile terminal television function module is further used to Transmitting, by the mobile terminal, a television program image frame to the subtitle processing module;

The subtitle processing module is configured to identify a subtitle from a television program image frame of the mobile terminal, and send the subtitle to the word map synthesis module. Further, the subtitle processing module identifies the subtitle from the mobile terminal television program image frame by using neural network pattern recognition, genetic algorithm, wavelet analysis, or database-based feature comparison.

Preferably, the mobile terminal further includes:

a font module, configured to store a font to be displayed, when the font synthesis module changes the size of the subtitle in the mobile terminal television program; and

And a display module, configured to display, to the user, an image frame sent by the word map synthesis module or a mobile terminal television program image frame sent by the mobile terminal television function module to the sub-picture synthesis module.

The invention also discloses a method for realizing dynamic zooming of a mobile terminal television subtitle, wherein after the mobile terminal starts the mobile terminal television application service, if the user initiates a change of the subtitle size instruction, the mobile terminal changes according to the user's instruction. The subtitle size of the mobile terminal television program received by itself, and the changed subtitle is superimposed on the image frame of the mobile terminal television program received by itself.

Further, after the user initiates the change of the subtitle size instruction, the method further includes: the mobile terminal sending a message asking whether the network side separately transmits the subtitle of the mobile terminal television program, and if yes, the mobile terminal separately receives the mobile Subtitles of the terminal television program, and changing the size of the received subtitles according to the instructions of the user.

Further, after the user initiates the change of the caption size instruction, the method further includes: the mobile terminal determining whether the caption of the mobile terminal television program is separately received, and if yes, the mobile terminal changes according to the user instruction The size of the subtitle received separately.

Preferably, after the mobile terminal sends the message, if the network side confirms that the subtitle of the mobile terminal television program is not separately transmitted, the mobile terminal receives the mobile terminal television program from itself. After the subtitle is recognized in the image frame, the subtitle size is changed according to the user's instruction.

Preferably, the mobile terminal uses a neural network pattern recognition, a genetic algorithm, a wavelet analysis, or a database-based feature comparison to identify a subtitle from an image frame of the mobile terminal television program.

Further, when the mobile terminal changes the subtitle size of the mobile terminal television program received by itself, the mobile terminal also invokes the font to be displayed; After the changed subtitle is superimposed on the image frame of the mobile terminal television program received by itself, the image frame is displayed to the user.

The technical solution of the invention optimizes the visibility of the mobile terminal television, and solves the problem that the mobile terminal television subtitle is too small to be scaled.

BRIEF abstract

1 is a schematic structural diagram of a mobile terminal proposed in this embodiment;

2 is a schematic flow chart of a method for implementing dynamic zooming of a television subtitle by the mobile terminal shown in FIG. 1.

Preferred embodiment of the invention

The specific implementation of the present invention is further described in detail below with reference to the accompanying drawings and embodiments. A mobile terminal for realizing dynamic zooming of television subtitles, as shown in FIG. 1, includes a word map synthesizing module 500; and the synthesizing module The connected font module 100, the subtitle processing module 200, the mobile terminal television function module 300, the user operation parsing module 400, and the display module 600; the mobile terminal television function module 300 is also respectively associated with the subtitle processing module 200, the user operation parsing module 400, and the display module. 600 connected, where:

The font module 100 is configured to store a font to be displayed during the scaling process for the font map synthesis module 500 to call. For example, when the font module 100 uses the dot matrix font, it needs to store the bitmap image information of each character, wherein the size of the font is indicated by the number of the dot matrix, so that multiple sets of dot matrix fonts of different sizes need to be placed; When the module 100 uses the vector font, it needs to store the vector information of the character and the vector font engine API, wherein the vector outline of the character is outputted as a dot matrix after passing through the vector engine, and finally outputted on the screen through the dot matrix, the font module 100 When a plurality of fonts are supported, such as a black body, a Song, a Wei, a baby, etc., a plurality of outline description files are included. The subtitle processing module 200 is configured to process the subtitles of the mobile terminal television program and send them to the word map synthesis module 500. The subtitle processing module 200 further includes a font encoding conversion unit 202, and an image recognizing unit 201 and a subtitle receiving unit 203 connected to the font encoding conversion unit. The subtitle processing module 200 is connected to the mobile terminal television function module 300 through the image recognizing unit 201. The over font encoding conversion unit 202 is connected to the word map synthesis module 500, wherein

The image recognition unit 201 is configured to read an image frame of the mobile terminal television program in the mobile terminal television function module 300, identify the subtitle from the subtitle, convert the recognized subtitle code into UTF8 code and send it to the font encoding unit 202, and perform image recognition. Unit 201 can be implemented by existing mature technologies, such as neural network based pattern recognition, genetic algorithms, wavelet analysis, or traditional database-based feature alignment, etc., to effectively identify specific information, such as characters, from images. ;

In this embodiment, the image recognition unit 201 implements image recognition by using a neural network algorithm, and the implementation steps are: (1) normalizing the input image frame data; (2) inputting the normalized input layer neurons to the neural network. (3) performing matrix operations using a neural network weight matrix; (4) outputting the output layer neurons as a set of character codes;

The font encoding conversion unit 202 converts the subtitle encoded by the image recognizing unit 201 or the subtitle receiving unit 203 into an encoding used by the font module 100, and transmits the converted character encoding to the font synthesizing module 500, for example, a subtitle receiving unit. The character encoding sent by 203 is UTF8 format, and the character encoding recognized by image recognition unit 201 is also UTF8 format, and when font library module 100 stores font information in GBK encoding order, font encoding conversion unit 202 needs to display characters in UTF8 format. The encoding is converted to the GBK encoding format. In other embodiments, the font encoding conversion unit 202 also supports conversion between other encodings, such as UTF8 to UNICODE. Of course, in some embodiments, the unit may be omitted and recognized by the image. The unit 201 and the subtitle receiving unit 203 convert the character encoding;

The subtitle receiving unit 203 is configured to receive and parse the subtitle data packet sent by the network side, and send the subtitle encoding to the font encoding conversion unit 202, where the subtitle data sent by the network side includes the following field, the encoding format of the character (UNICODE, UTF8 or GBK), a set of character encoding, character encoding terminal special display effect (italic, bold, etc.), the start and end image frame number of the group of codes, in other embodiments, the caption receiving unit 203 may also receive according to whether a subtitle data packet, to determine whether the current mobile terminal television program supports the terminal to synthesize the subtitle, that is, when the subtitle receiving unit 203 receives the subtitle data packet, it is considered that the current mobile terminal television program supports the terminal to synthesize the subtitle, when the subtitle receiving unit 203 does not When receiving the subtitle data packet, it is considered that the current mobile terminal television program supports the source end to synthesize the subtitle, and notifies the image recognition unit in the subtitle processing module to read the image frame of the mobile terminal television program from the mobile terminal television function module 300, and recognizes from it subtitle. The mobile terminal television function module 300 is configured to receive and decode the mobile terminal television program sent by the network side, and send the decoded mobile terminal television program to the subtitle processing module 200 or the display module according to the user operation sent by the user operation parsing module 400. 600, wherein, when the user operates to enlarge or reduce the font, the mobile terminal television function module 300 transmits the decoded mobile terminal television program to the subtitle processing module 200, and when the user operates for other operations, the mobile terminal television function module 300 The decoded mobile terminal television program is directly sent to the display module 600;

In this embodiment, the mobile terminal television function module 300 may further send a message to the network side, and query whether the current mobile terminal television program on the network side separately transmits the subtitle to determine whether the current mobile terminal television program supports the terminal synthesized subtitle or the source side synthesized subtitle. . Here, when the subtitle is sent separately, the current mobile terminal television program support terminal synthesizes the subtitle, otherwise it indicates that the current mobile terminal television program supports the source end synthesis subtitle. When the current mobile terminal television program supports the source side to synthesize the subtitle, the mobile terminal television function module 300 further transmits the decoded mobile terminal television program image frame to the subtitle processing module 200 according to the call of the user operation parsing module 400 (the user performs subtitle enlargement or reduction). Operation) or display module 600 (the user selects to directly display the mobile terminal television program or does not perform the subtitle zoom operation); when the current mobile terminal television program supports the terminal to synthesize the subtitle, the mobile terminal television function module will decode the data regardless of any operation of the user. The mobile terminal television program image frame is sent to the word map synthesis module 500;

In other embodiments, the mobile terminal television function module 300 may also not send a message to the network side, but the subtitle processing module 200 determines whether the current mobile terminal television program separately transmits the subtitle, that is, the subtitle processing module determines whether the network side sends the message. Subtitle packet.

The user operation parsing module 400 is configured to parse the keyboard of the underlying driver, the GUI/GDI report, or touch the message, thereby obtaining whether the current user operation is to enlarge the font, reduce the font, or other operations, when the operation selected by the user is to enlarge or reduce the font. The user operation is simultaneously sent to the word map synthesizing module 500 and the mobile terminal television function module 300. When the current user selects an operation to directly display the mobile terminal television program or does not select to enlarge or reduce the font, the mobile terminal television function module 300 is directly decoded. The image frame of the obtained mobile terminal television program is sent to the display module 600;

After the user operation parsing module 400 parses the user operation, the user-selected operation may be converted into a character size value and stored in a global variable. At this time, when the user enters the mobile terminal television application service again, the subtitle is displayed according to the saved character size. , of course, in other embodiments, user operations The parsing module may also not save the character size value. At this time, when the user enters the mobile terminal television application service again, the subtitle is displayed according to the default character size.

The word map synthesis module 500 is configured to extract a subtitle of a corresponding size from the font module 100 according to an instruction of the user operation parsing module 400, and superimpose the subtitle on the image frame of the mobile terminal television program, and send the subtitle to the display module 600, where When the current mobile terminal television program supports the terminal to synthesize the subtitle, the word map synthesis module 500 calls the GUI to provide a correlation function, and directly outputs the font to the display buffer; when the current mobile terminal television program supports the source side to synthesize the subtitle, the word map synthesis module 500 directly outputs the font to a specific location of the HDC-SCREEN;

The display module 600 is configured to display an image sent by the word map synthesis module 500 or a mobile terminal television program image frame sent by the mobile terminal television function module 300.

The method for implementing the dynamic zooming of the subtitle of the mobile terminal by the mobile terminal includes the following steps: Step 201: The user opens the mobile terminal to start the mobile application service of the mobile terminal;

Step 202: The mobile terminal television function is initialized to receive the mobile terminal television program. Step 203: The mobile terminal determines the mobile terminal television program that is received by the mobile terminal, that is, whether the current mobile terminal television program supports the terminal to synthesize the subtitle. If yes, the step is performed. 211, otherwise, performing step 204;

In this step, the mobile terminal may send a message to the network side to query whether the network separately sends the subtitle of the current mobile terminal television program. If the network side returns the confirmation message, it considers that the current mobile terminal television program supports the terminal to synthesize the subtitle; When the side returns "no support" message or does not return a message, it is determined that the current mobile terminal television program does not support the terminal synthesized subtitle.

In other embodiments, the mobile terminal may also determine whether the current mobile terminal television program supports the terminal to synthesize the subtitle based on whether a separate subtitle file has been received. That is to say, when the mobile terminal has received a separate subtitle file, it is considered that the current mobile terminal television program supports the terminal to synthesize the subtitle, otherwise the current mobile terminal television program supports the source end to synthesize the subtitle.

Step 204: Parse the current operation of the user.

Step 205: Determine whether the current operation of the user changes the initial value of the subtitle size. If yes, execute step 206. Otherwise, the image synthesized by the source end, that is, the received mobile terminal television program is decoded. Show to the user, and return to step 204;

In this step, if the current operation of the user is to exit the mobile terminal television service, the process ends. Step 206: The mobile terminal decodes the image frame of the current mobile terminal television program, and saves the size of the subtitle in the current mobile terminal television program image frame, that is, the size of the enlarged or reduced subtitle selected by the user;

Step 207: The mobile terminal analyzes the decoded image frame, identifies subtitles in the image frame, that is, each character displayed, and encodes the characters according to UTF8;

In this step, the mobile terminal can identify subtitles in the image frame through existing mature technologies, such as neural network based pattern recognition, genetic algorithm, wavelet analysis, or traditional database-based feature comparison, etc., which can effectively Identify specific information, such as characters, from the image;

Step 208: The mobile terminal converts the character code into an encoding used by the font module. In this step, the mobile terminal uses a plurality of font modules, for example, a dot matrix font, a vector font, or the like.

Step 209: The mobile terminal extracts the corresponding font from the font module by using the converted character encoding as an index according to the font size set by the user operation, where the font mainly refers to the character size. In a preferred embodiment, The font also includes fonts, thicknesses, and other font display effects;

Step 210, the mobile terminal superimposes the extracted font on the decoded image frame, and displays it to the user, and returns to step 204;

Step 211, parsing a current operation of the user;

Step 212, determining whether the current operation of the user changes the initial value of the subtitle size, and if yes, executing step 213, otherwise returning to step 211;

In this step, if the current operation of the user is to exit the mobile terminal television service, the process ends. Step 213: After receiving the subtitle data packet from the network side, the mobile terminal converts the character encoding into an encoding used by the font module.

In this step, the subtitle data contains: the encoding format of the character, such as UNICODE, UTF8 or

GBK, etc.; a set of character encoding; character encoding terminal special display effects, such as italic, bold, etc.; and the start and stop image frame number of the group of codes. Step 214: The mobile terminal extracts the corresponding font from the font module according to the font size set by the user operation, and uses the converted character encoding as an index to extract the corresponding font from the font module. Step 215: The mobile terminal superimposes the extracted font onto the decoded font. On the image frame, it is displayed to the user, and returns to step 211.

It can be seen from the above embodiments that the technical solution of the present invention mainly uses the dynamic zooming technology of the subtitles according to user requirements on the terminal side, thereby optimizing the visibility of the mobile terminal television, and solving the problem that the mobile terminal television subtitles are too small to be scaled. The problem.

It is a matter of course that the invention can be embodied in various other forms without departing from the spirit and scope of the invention.

Industrial Applicability The present invention utilizes a technique of dynamically scaling subtitles according to user requirements on the terminal side, optimizes the visibility of the mobile terminal television, and solves the problem that the mobile terminal television subtitles are too small to be scaled, and thus has an industrial Practicality.

Claims

Claim

A mobile terminal for realizing dynamic zooming of a television subtitle, wherein the mobile terminal comprises: a word map synthesis module, and a mobile terminal television function module, a subtitle processing module, and a user operation analysis module both connected to the word map synthesis module , among them,

The user operation parsing module is configured to parse a change caption size instruction currently initiated by the user, and send the user instruction to the word graph synthesizing module and the mobile terminal television function module respectively;

2. The mobile terminal according to claim 1, wherein the mobile terminal television function module is connected to the subtitle processing module, wherein

The mobile terminal television function module is further configured to send a message to query whether the network side separately transmits the subtitle of the mobile terminal television program, and if yes, the mobile terminal television function module notifies the subtitle processing module to receive the subtitle;

3. The mobile terminal of claim 1, wherein

The subtitle processing module is further configured to separately receive subtitles of the mobile terminal television program, and send the subtitles to the word map synthesis module.

4. The mobile terminal of claim 2, wherein

After the mobile terminal television function module sends the message to the network side, if the network side is When the subtitle of the mobile terminal television program is not separately transmitted, the mobile terminal television function module is further configured to send the mobile terminal television program image frame to the subtitle processing module;

The subtitle processing module is configured to identify a subtitle from a television program image frame of the mobile terminal, and send the subtitle to the word map synthesis module.

5. The mobile terminal of claim 4, wherein

The caption processing module identifies the caption from the mobile terminal television program image frame by using neural network pattern recognition, genetic algorithm, wavelet analysis, or database-based feature comparison.

The mobile terminal of claim 1, wherein the mobile terminal further comprises:

7. A method for realizing dynamic zooming of a mobile terminal television subtitle, wherein

After the mobile terminal starts the mobile terminal television application service, if the user initiates the change of the subtitle size instruction, the mobile terminal changes the subtitle size of the mobile terminal television program received by the mobile terminal according to the instruction of the user, and changes the subtitle size. Superimposed on the image frame of the mobile terminal television program received by itself.

8. The method of claim 7, wherein after the user initiates the change of the caption size instruction, the method further comprises:

The mobile terminal sends a message to inquire whether the network side separately transmits the subtitle of the mobile terminal television program, and if so, the mobile terminal separately receives the subtitle of the mobile terminal television program, and changes the received according to the user's instruction. Subtitle size.

9. The method of claim 7, wherein after the user initiates the change of the caption size instruction, the method further comprises:

The mobile terminal determines whether the subtitle of the mobile terminal television program is separately received, and if so, the mobile terminal changes the separately received subtitle size according to an instruction of the user.

10. The method of claim 8 wherein After the mobile terminal sends the message, if the network side confirms that the subtitle of the mobile terminal television program is not separately transmitted, the mobile terminal identifies from the image frame of the mobile terminal television program received by itself. After the subtitles are output, the subtitle size is changed according to the user's instruction.

11. The method of claim 10, wherein

The mobile terminal identifies a subtitle from an image frame of the mobile terminal television program using neural network pattern recognition, genetic algorithm, wavelet analysis, or database based feature alignment.

The method of claim 7, wherein when the mobile terminal changes the subtitle size of the mobile terminal television program received by itself, the mobile terminal also invokes the font to be displayed;

After the changed subtitle is superimposed on the image frame of the mobile terminal television program received by itself, the image frame is displayed to the user.