JP5240833B2 - Subtitled video playback device, subtitled video playback method and program - Google Patents

Description

  The present invention relates to a captioned video playback apparatus, a captioned video playback method, and a program for playing back captioned video.

  There is a technique for separating subtitle information and video information of a video with subtitles and separately outputting them to a display unit.

In Patent Document 1, a caption text is extracted from a stream with a display time attached, and when the user selects a caption text, the selected caption text and a video corresponding to the caption text are synthesized. A closed caption display device for reproduction is disclosed.
JP 2003-18491 A

However, in the caption display device disclosed in Patent Literature 1, only the caption text before and after the selected caption is displayed in the caption display field. Therefore, the caption display device disclosed in Patent Document 1 cannot easily grasp information related to the reproduction time of the caption text.
For example, it has not been possible to grasp how much the time during which the selected subtitle text is reproduced accounts for the reproduction time of the entire program.
Further, for example, it has not been possible to grasp where in the playback time of the entire program the time from the start of playback of the program until the selected caption text is played back.

  The present invention has been made in view of the above problems, and an object thereof is to provide a captioned video playback apparatus, a captioned video playback method, and a program capable of easily grasping information regarding the time at which the caption text switches. To do.

  In order to achieve the above object, a video playback device with captions according to a first aspect of the present invention stores a plurality of caption texts and a plurality of videos in association with each other based on the playback order, Stores subtitle number information indicating the number of subtitles in a plurality of subtitle texts and total playback time information indicating the total playback time of content, and adds subtitle switching time information indicating subtitle switching times for switching to each subtitle sentence to each subtitle sentence. The subtitle sentence is reproduced according to the reproduction order from the subtitle sentence selected from the plurality of subtitle sentences stored in the storage means, and the subtitle sentence selected by the selection means. Subtitle playback means, video playback means for playing back video corresponding to the subtitle text being played back by the subtitle playback means, the number of subtitles indicated by the subtitle number information, and the total playback time indicated by the total playback time information , Each subtitle sentence Acquisition means for acquiring the caption switching time indicated by the added caption switching time information, a first image indicating the length of the acquired total reproduction time, and a second number corresponding to the number of the acquired captions An image generating means for generating an image, an image synthesizing means for generating an image by synthesizing the second image at a position corresponding to the caption switching time of the first image, and the image synthesizing means. Display means for displaying a composite image.

  For example, the image generation unit generates a third image and a fourth image different from the third image as the plurality of second images, and the image synthesis unit generates the first image. And synthesizing the third image at a position corresponding to the caption switching time corresponding to the caption text reproduced by the caption text reproducing means, and the fourth image at a position corresponding to the caption switching time other than the position. You may make it synthesize | combine the image of.

  For example, the third image may have a predetermined color, and the fourth image may have a color different from the predetermined color.

  Further, for example, the first image may be a line segment or a first bar, and one end of the line segment or the first bar may correspond to the top of the content.

  Further, for example, the third image may be a line segment having a predetermined thickness, and the fourth image may be a line segment having a thickness different from the predetermined thickness.

  Further, for example, the third image may be a circle having a predetermined diameter, and the fourth image may be a circle having a diameter different from the predetermined diameter.

  Further, for example, the image generation means has a brightness higher than the brightness of the second image by a predetermined brightness, and is reproduced by the caption sentence reproduction means of the first image from the one end. A second bar having a length up to a position corresponding to the subtitle switching time of the subtitle sentence is generated, and the image synthesizing unit generates one end of the second bar and one of the first bars. The image may be combined with the composite image by overlapping the end of the image.

  Further, for example, the image synthesizing unit may synthesize the second bar with the synthesized image so as not to overlap with the second image.

  Further, for example, video switching time information indicating video switching time for switching to each video is added to each video, and the acquisition unit is configured to add video switching time added to the video being played back by the video playback unit. The video switching time indicated by the information is further acquired, the image generation unit further generates a fifth image, and the image synthesis unit acquires the fifth image by the acquisition unit of the first image. You may make it synthesize | combine to the position corresponding to video switching time.

  Further, for example, the captioned video playback device further includes timer means for measuring a time from when the content is played back until the video being played back by the video playback means is played back, and the image generation means includes: A sixth image may be further generated, and the image synthesizing unit may synthesize the sixth image at a position corresponding to the length of time measured by the timer unit of the first image. .

In order to achieve the above object, a video playback method with captions according to a second aspect of the present invention is a video playback method with captions performed by a video playback device with captions, wherein the selection means defines a plurality of caption texts. Selecting a subtitle sentence from among a plurality of subtitle signals to be performed and a plurality of video signals defining a video associated with the subtitle signal based on the reproduction order, and subtitle reproduction from means, the selected selected caption text in step, in accordance with the reproduction order, the subtitle reproduction step of reproducing the caption text, the image reproducing means, an image corresponding to the caption sentence is being played in the subtitle reproduction step a video reproducing step of reproducing, the acquisition means, and subtitle number of said plurality of caption text, and the total playback time of the content, an acquisition step of acquiring and subtitle switching time to switch to the caption text, image generation Stage, and an image generation step of generating a first image showing the length of the acquired in acquisition step the total playback time, and a second image the subtitle fraction obtained by the obtaining step, image synthesizing means, and an image synthesizing step of generating a synthesized composite image at a position corresponding to said second image to the subtitle switching time of the first image, the display unit, generated by the image synthesizing step synthesis And a display step for displaying an image.

In order to achieve the above object, a program according to a third aspect of the present invention stores, in a computer, a plurality of subtitle sentences and a plurality of videos in association with each other based on the reproduction order. Subtitle number information indicating the number of subtitles in the subtitle text and total playback time information indicating the total playback time of the content are stored, and subtitle switching time information indicating the subtitle switching time for switching to each subtitle sentence is added to each subtitle sentence. A function to store, a function to select any subtitle sentence from a plurality of stored subtitle sentences, a function to reproduce a subtitle sentence from the selected subtitle sentence according to the reproduction order,
A function of playing a video corresponding to the subtitle text being played back, the number of subtitles indicated by the subtitle number information, the total playback time indicated by the total playback time information, and the subtitle switching time information added to each subtitle sentence A function of acquiring the subtitle switching time indicated by the function, a function of generating a first image indicating the length of the acquired total playback time, and a second image of the acquired number of subtitles, A function of generating a composite image in which a second image is combined at a position corresponding to the caption switching time of the first image and a function of displaying the generated composite image are realized. .

  According to the present invention, by visualizing information about the time when subtitles are switched, it becomes possible for the user to easily grasp information about the time when subtitles are switched, and usability is improved.

  Hereinafter, a captioned video reproduction apparatus according to an embodiment of the present invention will be described with reference to the drawings. In the present embodiment, a video playback device with captions is described as a mobile phone.

(First embodiment)
An appearance of a mobile phone according to this embodiment is shown in FIG.
The mobile phone 1 is, for example, a folding type that is divided into an upper housing 1a and a lower housing 1b.

FIG. 2 shows the configuration of the mobile phone 1 according to the present embodiment.
The cellular phone 1 according to the present embodiment includes an antenna 11, a communication unit 12, an audio microphone 13, an audio speaker 14, an antenna 15, a tuner 16, a decoding unit 17, a storage unit 18, a DAC ( Digital to Analog Converter) 19, speaker 20, display unit 21, operation unit 22, control unit 23, and bus 24.

  The mobile phone 1 is capable of displaying a caption text and a video and displaying a caption list, and is configured to be able to select a video based on the caption text selected in the caption list.

  The bus 24 transmits data between the units. The control unit 23, the storage unit 18, the DAC 19, and the display unit 21 are connected via a bus 24.

  The antenna 11 transmits and receives radio signals such as call voice and various data to and from a base station (not shown). As shown in FIG. 1, the antenna 11 is provided in the upper part of the upper housing 1 a of the mobile phone 1. The installation location of the antenna is not limited to the upper housing 1a, and may be installed in the lower housing 1b, for example.

  The communication unit 12 performs signal processing of radio signals transmitted and received by the antenna 11.

  As shown in FIG. 1, the audio microphone 13 is provided in the lower housing 1 b and collects audio and converts it into an audio signal during a call or the like, and supplies the converted audio signal to the communication unit 12. .

  The audio speaker 14 is provided in the upper housing 1a as shown in FIG. 1, and outputs audio based on an audio signal during a call or the like.

  The antenna 15 receives a radio signal of a broadcast signal of one-segment broadcasting and is built in the mobile phone 1. The broadcast signal of this one-segment broadcasting includes video data, audio data, and caption data as broadcast program content data. Note that the radio wave signal received by the antenna 15 has a format conforming to a data transfer method in one-segment broadcasting, for example.

  The tuner 16 and the decoding unit 17 analyze the broadcast signal received by the antenna 15 to obtain video data, audio data, and caption data. The tuner 16 demodulates the one-segment broadcasting signal from the radio signal received by the antenna 15, and supplies the demodulated demodulated signal to the decoding unit 17.

  The decoding unit 17 decodes the supplied demodulated signal and generates a TS (Transport Stream) as shown in FIG.

  TS is a stream in which TSP (Transport Stream Packet) is multiplexed, and video packet V for video data, audio packet A for audio data, and subtitle packet C for subtitles are included in TSP. is there.

  Each TSP is composed of a TS header and a payload, as shown in FIG. The TS header is an area for storing data indicating the start of a TS packet, a packet identifier, and the like.

  The payload is an area for storing data, and a PES (Packetized Elementary Stream) packet as shown in FIG. 3C is divided and stored. The PES is composed of data stored in each payload.

  The PES is composed of a PES header and a PES payload. The PES header is an area in which data indicating the start of PES, data such as a packet identifier and packet length, a presentation time PTS (Presentation Time Stamp) indicating the time when subtitles and video are switched and the subtitles and video are reproduced are stored. is there. The PES payload is an area in which an ES (Elementary Stream) is divided and stored, and video data, audio data, and caption data are compression-coded and stored in the PES payload.

  Based on the information stored in the TS header, the decoding unit 17 separates the video packet V, the audio packet A, and the caption packet C from the TS, and extracts the PES from the payload of each TSP.

  Further, based on the data stored in the PES header, the decoding unit 17 performs video PES including video data, audio PES including audio data, and subtitles including subtitle data, as shown in FIG. PES is generated.

  Note that the TS packet has a fixed length, and the PES has a variable length. For this reason, the TSP may include an adaptation field for making the length of the TSP constant.

  An example of video and caption television signals (hereinafter referred to as “TV signals”) SG displayed on the display unit 21 is shown in FIG. The cellular phone 1 converts the content recording data into the format of the TV signal SG, further converts it into displayable video data, and reproduces the video.

  Data in the one-segment broadcasting is data conforming to the predictive coding method, and the video PES includes an IDR (Instantaneous Decoder Refresh) frame and a P frame.

  In FIG. 4, Id_1, Id_2, Id_3, and Id_q indicate IDR frames, and P11, P12, P21, P22, and Pq1 indicate P frames. q is the number of IDR frames included in the TV signal SG.

  The IDR frames Id_1 to Id_q are frames that do not refer to other video frames and correspond to the independent video packet VI. The P frames P11 to Pq1 are frames that refer to the IDR frames Id_1 to Id_q that are reproduced before their own reproduction timing, and correspond to the reference video packet VP.

  The playback timing Tf is stored in the PES header of the video PES. The reproduction timing Tf is data indicating the reproduction time of the first video of the TV signal SG to be constructed. Further, the TSP header includes a seek point TR indicating the display time of the IDR frames Id_1 to Id_q.

  In FIG. 4, TR1, TR2, TR3, and TRq respectively indicate elapsed times of the IDR frames Id_1, Id_2, Id_3, and Id_q from the reproduction timing Tf.

  In the PES payload of the caption PES, compression-coded captions (data) M1 to Mp (p: 0 or a natural number) are stored. The caption sentences M1 to Mp are data indicating the contents of the caption added to the video.

  Each PES header of the caption PES includes caption switching times TM1 to TMp and control codes CS1 to CSp.

  The caption switching times TM1 to TMp are time information representing the timing at which the displayed caption text is switched to the caption text M1 to Mp, respectively. The control codes CS1 to CSp are information indicating delimiters of the caption texts M1 to Mp, respectively.

  In the TS header shown in FIG. 3B, reference time information called PCR (Program Clock Reference) is stored. A PTS is stored in the PES header. This PTS is time information indicating a time for reproducing an ES (Elementary Stream) in the PES, and is expressed based on a clock frequency of 90 kHz.

  The mobile phone 1 interprets the time at the moment of receiving the PCR as the time indicated by the PCR value, and synchronizes the STC (System Time Clock; reference time) of the mobile phone 1 with the PCR. Thereby, the time of all the receivers (cell-phone 1) synchronizes.

  Note that the reproduction timing Tf, caption switching times TM1 to TMp, and elapsed times TR1 to TRq are stored as data in units of PTS.

  The decoding unit 17 supplies the TS header and PES header information to the control unit 23 together with the decoded video PES, audio PES, and subtitle PES.

  Returning to FIG. 2, the storage unit 18 stores various data. The storage unit 18 includes a nonvolatile memory such as a flash memory, an internal memory such as a RAM (Random Access Memory), and an external recording medium.

  As shown in FIG. 5, the storage unit 18 includes a content recording data storage area 181, a caption management data storage area 182, a seek point management information storage area 183, and a display data storage area 184.

  The content recording data storage area 181 is an area for storing video PES, audio PES, and subtitle PES analyzed by the decoding unit 17 as data.

  The caption management data storage area 182 is an area for storing caption management data 182a as shown in FIG. The caption management data is table data that associates “caption switching time TMi” (1 = 1 to p) with “caption text Mi”, and is generated by the control unit 23.

  “Subtitle switching time TMi” indicates a time at which the subtitle text displayed on the display unit 21 is switched to the subtitle text Mi. In the present embodiment, the subtitle switching time TMi, that is, the reference for displaying the subtitle sentence Mi is set as the offset time that is the reproduction timing Tf at the start of reproduction of the content recording data (the reproduction time of the first video of the TV signal) .

  The seek point management information storage area 183 is an area for storing seek point management information 183a as shown in FIG. The seek point management information is table data that associates the seek point TRj (j = 1 to q) with the video position PRj, and is generated by the control unit 23.

  “Seek point TRj” indicates the time at which each video frame is displayed, and is expressed in units of PTS. “Video position PRj” is data indicating the position of the video frame in the content recording data, and functions as an index. In the case of one-segment broadcasting, “video position PRj” is usually a TSP number including the head of the IDR frame.

  The display data storage area 184 is an area for storing data such as video and subtitles displayed on the display unit 21. In the display data storage area 184, in addition to the video and subtitles extracted from the broadcast signal of the one-segment broadcasting, image data of an image representing information regarding the time at which the subtitle sentence is switched is also stored. This image will be described later.

  Further, the storage unit 18 stores data of an operation control program for the control unit 23 and also functions as a work memory for the control unit 23.

  A DAC (Digital to Analog Converter) 19 converts a digital audio signal (sound data) from the control unit 23 into an analog audio signal, and supplies the converted analog audio signal to the speaker 20.

  The speaker 20 outputs sound based on the analog audio signal supplied from the DAC 19. The speaker 20 is provided in the upper housing 1a as shown in FIG.

  The display unit 21 includes a dot matrix type LCD (Liquid Crystal Display) 31, a driver circuit (not shown), and the like, and displays subtitles reproduced together with video on the LCD 31. The LCD 31 is provided in the upper housing 1a. The display unit 21 reads data from the display data storage area 184 of the storage unit 18 under the control of the control unit 23 and displays the subtitles reproduced along with the video based on the read data on the LCD 31. The display unit 21 also displays an image representing the time at which the caption text is switched under the control of the control unit 23. In this embodiment, the display used for the display unit is an LCD, but the present invention is not limited to this. For example, an organic EL (Organic Electro-Luminescence) display may be used.

  The mobile phone 1 has two viewing modes, and there are two viewing modes of “viewing mode with caption list” and “normal viewing mode”. The display unit 21 displays a screen corresponding to the mode on the LCD 31.

  The “viewing mode with subtitle list” is a mode in which subtitles corresponding to the video are displayed and the subtitle list is displayed to enable selection of the video from the subtitle list. “Normal viewing mode” is a mode in which subtitles and video are displayed without displaying a subtitle list.

  When “viewing mode with subtitle list” is selected, the display unit 21 displays a screen D1 as shown in FIG.

  In the “viewing mode with subtitle list” screen, the video area RA is an area for displaying video. The display unit 21 displays a video frame corresponding to the subtitle sentence selected from the subtitle list in the video area RA.

  The caption area MA is an area for displaying captions. The display unit 21 displays the caption text selected by the user from the caption list in the caption area MA.

  The time bar area TA displays a switching time when switching to each caption text included in the content. For example, when the playback time of the entire content is 30 minutes, the playback time of the entire content is displayed as a bar, and the subtitle switching time TMi is at the position of 10 minutes and 15 minutes, the position of 1/3 of the bar and 1 / A vertical line is displayed at position 2.

  The subtitle list area LA is an area for displaying a plurality of subtitle sentences arranged in a list format. The display unit 21 displays, for example, five subtitle list branch areas LA1 to LA5 in the subtitle list area LA.

  The soft key area FA is an area for displaying soft keys Soft1, Soft2, and center. The soft keys Soft1, Soft2, center are used, for example, for switching the viewing mode. The soft keys Soft1, Soft2, and center are selected by operating the keyboard 32.

  The picto area PA is an area for displaying operation information of the mobile phone 1. The display unit 21 displays the radio wave reception status, the remaining battery level, the current date and time, etc. of the mobile phone 1 in this pictogram area PA.

  The “normal viewing mode” is a mode for displaying the subtitles reproduced together with the video without displaying the subtitle list. In the viewing mode with subtitle list, when any subtitle sentence is selected from the subtitle sentences displayed in the subtitle list, a video frame corresponding to the selected subtitle sentence is displayed in the video area RA. Then, when the operation information for instructing the reproduction of the video is supplied from the operation unit 22, the mode is switched to the “normal viewing mode”, and the video corresponding to the selected subtitle sentence is sequentially reproduced. When the “normal viewing mode” is set, the display unit 21 displays on the LCD 31 a screen D2 in which the caption list area LA as shown in FIG.

Returning to FIG. 2, for example, the operation unit 22 instructs to start / end the reception of television broadcasts, to instruct the start / end of content recording when the user records a TV program, and to enter the password for user authentication. It accepts user input such as input.
The operation unit 22 includes a keyboard 32 as shown in FIG. The keyboard 32 includes a power switch, a cursor key 33, a numeric keypad, and the like, and is used for turning on / off the power of the mobile phone 1, switching modes, inputting numbers, characters, symbols, selecting menus, and receiving one-segment broadcasting. .

  The control unit 23 includes a microprocessor unit and the like, and controls the operation of the entire mobile phone 1 according to operation control program data stored in the storage unit 18. Further, the control unit 23 controls the operation of the mobile phone 1 based on the operation information supplied from the operation unit 22.

  Next, the operation of the mobile phone 1 according to the present embodiment will be described.

  The control unit 23 turns on and off the power of the mobile phone 1 every time operation information indicating that the power switch of the keyboard 32 is pressed is supplied from the operation unit 22.

  When the operation information for selecting the menu for receiving the broadcast signal of the one-segment broadcast is supplied from the operation unit 22, the control unit 23 controls the antenna 15, the tuner 16, and the decoding unit 17 to start reception of the one-segment broadcast. .

  The antenna 15 receives a one-segment broadcast radio signal, and the tuner 16 demodulates the one-segment broadcast signal from the radio signal received by the antenna 15 and supplies the demodulated demodulated signal to the decoding unit 17.

  The decoding unit 17 separates video data, audio data, and caption data included in the supplied content data, decodes these data, and sends the decoded video PES, audio PES, and caption PES to the control unit 23. Supply.

  When the operation information indicating that the menu for recording the content of the 1Seg broadcast is selected from the operation unit 22, the control unit 23 records the 1Seg broadcast.

  The control unit 23 stores the video PES, audio PES, and subtitle PES supplied from the decoding unit 17 in the content recording data storage area 181 of the storage unit 18.

  When storing the video PES, the audio PES, and the caption PES in the content recording data storage area 181, the control unit 23 reads the program data of the caption management data generation process, and executes the caption management data generation process according to the flowchart shown in FIG. To do.

  First, the control unit 23 determines whether or not there is a caption PES in the content recording data (step S101). When it is determined that there is no caption PES (step S101; No), the control unit 23 ends the caption management data generation process.

  When it is determined that there is a caption PES (step S101; Yes), the control unit 23 extracts a caption PES (step S102).

  Next, the control unit 23 determines whether or not the caption PES has been successfully extracted (step S103). When it is determined that the caption PES has not been successfully extracted (step S103; No), the control unit 23 advances the process to step S108.

  When it is determined that the caption PES has been successfully extracted (step S103; Yes), the control unit 23 acquires the reproduction timing TMi of the caption sentence, and uses the reproduction timing Tf of the first video data of the TV signal SG from the reproduction timing TMi. Subtract (step S104).

  The control unit 23 converts the elapsed time (TMi−Tf) obtained in step S104 from the PTS unit to the hour / minute / second unit (step S105).

  The control unit 23 registers the elapsed time (hour / minute / second unit) converted in step S105 in the caption management data 182a (step S106). This elapsed time indicates the time from the beginning of the content until the caption text displayed in the caption display area MA is switched to the caption text Mi.

  Next, the control unit 23 extracts the caption sentence Mi from the caption PES, adds the elapsed time registered in step S106 to the caption sentence Mi as the caption switching time TMi (in hours, minutes, and seconds) and registers it in the caption management data 182a. (Step S107).

  Then, the control unit 23 determines whether or not the end of the content recording data has been reached (step S108). When it is determined that the end of the content recording data has been reached (step S108; Yes), the control unit 23 ends the caption management data generation process.

  If it is determined that the end of the content recording data has not been reached (step S108; No), the control unit 23 returns the process to step S101. Therefore, the control unit 23 repeatedly executes the processes in steps S101 to S108 until the end of the content recording data.

  When the subtitle management data generation process is executed in this way, the control unit 23 sequentially analyzes the TSP based on the TS header data supplied from the decoding unit 17. The control unit 23 registers the analyzed data in the seek point management information in the seek point management information storage area 183 of the storage unit 18 to generate seek point management information.

  Further, the control unit 23 analyzes the TV signal SG and stores the total reproduction time Ttotal, which is a time for reproducing the entire content, in the storage unit 18.

  Next, an operation of scrolling the selected subtitle sentence so that it is positioned at the center of the subtitle list area LA will be described with reference to the flowchart of FIG.

  When an application for reproducing video with captions is activated and operation information for setting the list-added caption viewing mode is supplied from the operation unit 22, the control unit 23 starts the caption scrolling process illustrated in the flowchart of FIG. .

  First, when the operation information for selecting the caption text Mn is supplied from the operation section 22, the control section 23 selects the caption text Mn (step S201). Here, the selected subtitle sentence Mn is the nth subtitle sentence (n: a positive integer from 1 to p) among the subtitle sentences registered in the subtitle management data 182a.

Next, the control unit 23 determines the value of n−1 (step S202).
When it is determined that n−1 is 2 or more, the control unit 23 displays the caption texts Mn−2, Mn−1, and Mn in the caption list areas LA1, 2, and 3, respectively (step S203).
If it is determined that n−1 is 1, the control unit 23 displays the caption texts M1 and M2 in the caption list areas LA2 and LA3 (step S204).
When determining that n−1 is 0, the control unit 23 displays the caption sentence M1 in the caption list area LA3 (step S205).

Next, the control unit 23 determines the value of p−n (step S206).
When it is determined that p−n is 2 or more, the control unit 23 displays the caption sentences Mn + 1 and Mn + 2 in the caption list areas LA4 and LA, respectively (Step S207), and ends the caption scrolling process.
If it is determined that p-n is 1, the control unit 23 displays the caption text MN in the caption list area LA4 (step S207), and ends the caption scrolling process.
When it is determined that p−n is 0, the control unit 23 ends the caption scroll process.

  When the user of the mobile terminal 1 operates the cursor key 33 to select a subtitle sentence by this subtitle scroll processing, the selected subtitle sentence is scrolled to the center of the subtitle list area LA and displayed. For example, in the viewing mode with a caption list, as shown in FIG. 7A, when five caption list branch areas LA1 to LA5 are displayed in the caption list area LA, the control unit 23 selects the selected caption sentence Mn. Is displayed in the caption list branch area LA3.

  Next, subtitle switching time display processing for displaying the display time at which subtitle text reproduction starts in the time bar area will be described with reference to the flowchart of FIG.

  The control unit 23 starts an application that reproduces video with captions, and starts subtitle switching time display processing when operation information for setting the viewing mode with caption list is supplied from the operation unit 22.

  First, the control unit 23 acquires a total reproduction time Ttotal that is a time for reproducing the entire content stored in the storage unit 18 (step S301).

  Next, the control unit 23 reads the caption management data 182a stored in the storage unit 18, and acquires the caption switching times TM1 to TMp of all the caption texts (step S302).

  The control unit 23 generates a time bar indicating the length of the total reproduction time Ttotal and a line segment corresponding to p subtitles of all subtitle sentences (step S303).

  And the control part 23 synthesize | combines p line segments in the position corresponding to the caption switching time of a time bar, and produces | generates an image (step S304). Then, the generated image is displayed in the time bar area TA (step S305).

  The control unit 23 selects a caption sentence Mn (n = 1 to p) based on the operation information supplied from the operation unit 22 (step S306).

  When the caption sentence Mn is selected, the control unit 23 emphasizes and displays the synthesized line segment at the position of the caption switching time TMn corresponding to the caption sentence Mn in the time bar (step S307). For example, the control unit 23 displays a line segment at a position corresponding to the caption switching time TMn in red and displays the other line segments in black.

  Subsequently, the control unit 23 executes the above-described subtitle scroll process, scrolls the subtitle sentence in the subtitle list area LA, and displays the selected subtitle sentence Mn to be displayed in the center of the subtitle list area LA. (Step S308).

  Then, the control unit 23 displays the video frame corresponding to the subtitle sentence Mn in the video area RA (step S309).

Next, when the keyboard 32 is operated by the user and operation information is supplied from the operation unit 22, the control unit 23 determines the content of the key input based on the operation information (step S310).
When it is determined that the key input is content for instructing to end the application (step S310; END), the control unit 23 ends the caption switching time display process.

When the key input indicates selection of a caption text and there is no corresponding caption text (step S310; NG), the control unit 23 returns the process to step S310 and prompts for a new key input.
Here, “when the key input instructs selection of a caption sentence and there is no corresponding caption sentence”, for example, a caption sentence corresponding to the first playback order of content is displayed in the caption area MA. When the video frame corresponding to the subtitle sentence is displayed in the video area RA, the up key of the cursor key 33 is pressed, and operation information for selecting the previous subtitle sentence is supplied. Is the case. In this case, since there is no caption text to be selected, the control unit 23 prompts a new key input.
At this time, the control unit 23 may display, for example, “there is no corresponding subtitle sentence” in the video area RA, or may emit a sound to call attention.

When the key input indicates selection of a caption text and there is a corresponding caption text (step S310; OK), the control unit 23 returns the process to step S306 and selects the corresponding caption text.
The “corresponding subtitle text” in this case is not limited to the subtitle text before or after the currently displayed subtitle text. For example, when the left direction key of the cursor key 33 is pressed, the control unit 23 selects a subtitle sentence one page before the currently displayed subtitle list, and when the right direction key is pressed, the control section 23 A subtitle sentence may be selected.
And the control part 23 repeats the process of step S306-step S310 until there exists key input which instruct | indicates completion | finish of an application.

  Here, FIG. 11 shows an example of an image displayed in the time bar area TA of the display unit 21 when the caption switching time display process is executed.

  When the control unit 23 selects the caption sentence Mn, the display unit 21 displays an image D3 as shown in FIG. A caption text Mn is displayed in the caption area MA, and a video corresponding to the caption text Mn is reproduced in the video area RA. Subtitle sentences Mn-2, Mn-1, Mn, Mn + 1, and Mn + 2 are displayed in the subtitle list area LA, and a rectangular time bar 301 is displayed in the time bar area TA.

  For example, when the total playback time of the content is 1 hour, the length of the time bar 301 is 300 pixels wide, and the caption switching time of the caption text Mn is 25 minutes, the control unit 23 is shown in FIG. Thus, the line segment 302n is displayed at a position of 25/60 × 300 = 125 pixels.

  The control unit 23 displays line segments in the same manner for the other subtitle sentences Mn-2, Mn-1, Mn + 1, and Mn + 2. The control unit 23 displays the line segment 302n of the currently selected subtitle sentence Mn in red, for example, and displays the other line segments in black, for example. The display color of the line corresponding to the subtitle sentence being reproduced is not limited to red, and may be different from the display color of the line corresponding to the subtitle sentence not being reproduced. Alternatively, the line segment corresponding to the subtitle sentence being reproduced may be displayed so that the line segment is different from the line segment corresponding to the other subtitle sentence.

  Further, the display form of the line segment 302n (n; 1 to p) corresponding to the subtitle sentence Mn being reproduced is not limited to this. For example, in the process of step S304, as shown in FIG. 11C, the line segment of the subtitle sentence Mn being played starting from the position corresponding to the beginning of the content (the left end of the time bar 301) on the time bar 301. A bar 303 having the end point 302n may be overlapped. In this case, the bar 303 is displayed with a higher brightness than the line segments 302n-2 to 302n + 2. Then, the user of the mobile phone 1 can recognize a line segment corresponding to a subtitle sentence other than the subtitle sentence being reproduced.

  Further, for example, as shown in FIG. 11 (d), by adjusting the size and display color of the bar 303 shown in FIG. 11 (c), the user of the mobile phone 1 corresponds to a subtitle sentence not selected. The subtitle plots 302n-2 and 302n-1 may be recognized.

  In addition to displaying the playback time of content with a bar, other display forms may be used.

  For example, in the process of step S307, an image D4 as shown in FIG. 12 may be displayed. In this case, the total playback time of the content is displayed with a line segment 304 instead of a bar. In this case, an image to be arranged at a position corresponding to the caption switching time TMi (i: 1 to p) of each caption sentence on the line segment 304 is, for example, a circle 305n as shown in FIGS. 12 (b) to (d). -2 to 305n + 2 may be displayed. Further, for example, the diameter of the circle corresponding to the subtitle sentence being reproduced may be displayed differently from the diameter of the circle corresponding to the other subtitle sentence.

  Also in this display form, as shown in FIGS. 12C and 12D, the subtitle switching time of the subtitle sentence being reproduced may be emphasized and displayed by superimposing the bar 306 on the line segment 304.

  For example, in the process of step S307, an image D5 as shown in FIG. 13 may be displayed. In this case, as shown in FIGS. 13B to 13D, the total playback time Ttotal of the entire content is displayed as a line segment 307, corresponding to the caption switching time TMi (i: 1 to p) of each caption sentence. Line segments arranged at positions are displayed as line segments 308n-2 to 308n + 2.

  Also in this display form, as shown in FIGS. 13C and 13D, the subtitle switching time of the subtitle sentence being reproduced may be emphasized and displayed by superimposing the bar 309 on the line segment 307.

  By this subtitle switching time display processing, the user of the mobile phone 1 can easily grasp where the time of switching to the subtitle sentence being played is located and the proportion of the total playback time of the content. it can. Therefore, the user of the mobile phone 1 can easily grasp information related to the caption switching time of the caption sentence being reproduced.

(Second Embodiment)
In the first embodiment, the mobile phone display process in the viewing mode with subtitle list has been described. As described above, the mobile phone 1 is provided with the “normal viewing mode” in addition to the “subtitle list viewing mode” as the viewing mode. In this normal viewing mode, no caption text is displayed in the caption list area LA. For this reason, the user of the mobile phone 1 cannot select the caption sentence by operating the cursor key 33. Therefore, as in the first embodiment, it is not possible to grasp information regarding the switching time corresponding to the subtitle sentence being reproduced.

  Therefore, the mobile phone 2 according to the present embodiment visualizes and displays the elapsed time from the top of the content of the subtitled video currently being reproduced in the time bar area TA in the normal viewing mode.

  A playback time display process for visualizing and displaying the elapsed time from the top of the content of the subtitled video being played back in the time bar area TA will be described with reference to the flowchart of FIG.

  The control unit 23 activates an application that reproduces video with captions from the operation unit 22, is supplied with operation information for setting the viewing mode with caption list, and further has operation information for setting the “normal viewing mode”. When supplied, the reproduction time display process is started.

  First, the control unit 23 executes processing from step S301 to step S304 shown in the flowchart of FIG. 10 (step S301 to step S304 of FIG. 10).

  When the composite image of the time bar and the line segment is generated (step S304), the control unit 23 generates a line segment for displaying the time of the video being reproduced (step S401).

  When the operation information for starting the reproduction of the content is supplied from the operation unit 22, the control unit 23 starts the reproduction of the content (step S402).

  The control unit 23 acquires the seek point TRj of the video currently being played back (step S403).

  The control unit 23 combines the line segment generated in step S401 at the position corresponding to the time indicated by the seek point TRj acquired in step S403 of the image combined in step S304 (step S404).

  The control unit 23 displays the image synthesized in step S404 in the time bar area TA (step S405).

  Then, the control unit 23 determines whether or not the content being reproduced has ended (step S406). If it is determined that the process has ended (step S406; Yes), the control unit 23 ends the reproduction time display process.

  On the other hand, if it is determined that the processing has not ended (step S406; No), the control unit 23 returns the process to step S403. Therefore, the control unit 23 repeats the processing from step S403 to step S406 until the content ends. In step S406, the control unit 23 may determine whether or not operation information indicating that the content reproduction is to be ended is supplied from the operation unit 22.

  In the playback time display process described above, the seek point TRj of the video being played back is acquired. However, the present invention is not limited to this. For example, the PTS added to the video frame may be acquired, and the line segment generated in step S401 may be combined at the position corresponding to the time indicated by the seek points TRj and PTS. . In this case, it is possible to display the line segment so as to move more smoothly than when only the seek point TRj is acquired.

  With this playback time display process, the user of the mobile phone 2 can determine whether the switching time of the video being played back and the switching time of the caption text even when the caption plot of the caption text corresponding to the video being played is not highlighted. The relationship can be easily grasped.

(Third embodiment)
Further, for example, the time from the beginning of the content to the time corresponding to the video being played back may be measured, and a line segment may be displayed at a position corresponding to the measured time.

  As shown in FIG. 15, mobile phone 2 according to the present embodiment has a configuration in which timer 25 is added to the configuration of mobile phone 1.

  The timer 25 is composed of a software timer or the like, measures the time from the beginning of the content to the time corresponding to the video being reproduced, and notifies the control unit 23 of the measured time.

  The reproduction time display process of the mobile phone 2 provided with the timer 25 will be described with reference to the flowchart of FIG.

  The control unit 23 activates an application that reproduces video with captions from the operation unit 22, is supplied with operation information for setting the viewing mode with caption list, and further has operation information for setting the “normal viewing mode”. When supplied, the reproduction time display process is started.

  First, the control unit 23 executes processing from step S301 to step S304 shown in the flowchart of FIG. 10 (step S301 to step S304 of FIG. 10).

  When the composite image of the time bar and the line segment is generated (step S304), the control unit 23 generates a line segment for displaying the time of the video being reproduced (step S401).

  When the operation information for starting the reproduction of the content is supplied from the operation unit 22, the control unit 23 starts the reproduction of the content (step S402).

  The control unit 23 controls the timer 25 to measure the time from the beginning of the content to the time corresponding to the video being played back (step S501).

  The control unit 23 synthesizes the line segment generated in step S401 at a position corresponding to the time measured in step S501 of the image synthesized in step S304 (step S404).

  The control unit 23 displays the image synthesized in step S404 in the time bar area TA (step S405).

  Then, the control unit 23 determines whether or not the content being reproduced has ended (step S406). If it is determined that the process has ended (step S406; Yes), the control unit 23 ends the reproduction time display process.

  On the other hand, if it is determined that the process has not been completed (step S406; No), the control unit 23 returns the process to step S501. Therefore, the control unit 23 repeats the processing from step S501 to step S406 until the content ends. In step S406, the control unit 23 may determine whether or not operation information indicating that the content reproduction is to be ended is supplied from the operation unit 22.

  Even in this playback time display process, the user of the mobile phone 2 can calculate the elapsed time of the video being played and the switching time of the caption text even when the caption plot of the caption text corresponding to the video being played is not highlighted. The relationship can be easily grasped.

  A display example of the screen of the display unit 21 when the reproduction time display process is executed is shown in FIG.

  When the reproduction time display process is executed, an image D6 as shown in FIG. 17A is displayed on the display unit 21, and no caption text is displayed in the caption list area LA. At this time, in the time bar area TA, as shown in FIGS. 17B to 17D, a line segment 307 and a line segment 308n-2 at a position corresponding to the caption switching time of each subtitle sentence in the line segment 307 are displayed. ~ 308n + 2 are displayed. Then, a line segment 308x is displayed at a position corresponding to the switching time of the video being reproduced.

  The line segment 308x is displayed so as to move toward the right end of the line segment 307 as time passes. Therefore, the user can easily grasp the relationship between the elapsed time from the beginning of the content of the currently reproduced video content and the subtitle sentence switching time.

  The display color of the line segment 308x may be different from the display color of the line segments 308n-2 to 308n + 2, for example.

  For example, as shown in FIGS. 17C and 17D, the bar 309 may be displayed with the left end of the line segment 307 as the start point and the position corresponding to the display time of the content as the end point.

  For example, the bar 309 may be displayed as a bar having a lightness higher than the lightness of the line segments 308n-2 to 308n + 2.

  In this embodiment, a line segment is used as an image indicating the previous playback time of content. However, for example, a bar may be displayed.

  In this embodiment, line segments are displayed at positions corresponding to the subtitle switching time of each subtitle sentence and at positions corresponding to the elapsed time of the video being played back, but for example, a figure such as a circle, square, or triangle is displayed. May be used.

  In the above embodiment, the video playback device with captions has been described as the mobile phone 1. However, the video playback device with captions is not limited to this. For example, the present invention can also be applied to PHS, PDA (Personal Digital Assistants), electronic cameras, electronic watches, notebook PCs, portable TVs, portable video recording / reproducing devices, and car navigation devices.

  In the above embodiment, the program is described as being recorded in advance in a memory or the like. However, a program for operating the subtitled video playback apparatus as all or a part of the entire apparatus or executing the above-described processing is stored in a flexible disk, a CD-ROM (Compact Disk Read-Only Memory), a DVD ( Store and distribute in a computer-readable recording medium such as Digital Versatile Disk (MO) or MO (Magneto Optical disk), install it on another computer, operate as the above-mentioned means, or execute the above-mentioned steps May be.

  Furthermore, the program may be stored in a disk device or the like included in a server device on the Internet, and may be downloaded onto a computer by being superimposed on a carrier wave, for example.

It is a figure which shows the external appearance of the mobile telephone which concerns on embodiment of this invention. It is a block diagram which shows the structural example of the mobile telephone which concerns on embodiment of this invention. It is a figure which shows the structure of the signal to receive, (a) shows TS, (b) shows TSP, (c) shows PES. It is a figure which shows the structure of the converted television signal (TV signal). It is a figure which shows each storage area of the memory | storage part shown in FIG. (A) shows an example of the data structure of caption management data, and (b) shows an example of the data structure of seek point management information. It is a figure which shows the screen which a display part displays on LCD, (a) shows an example of the screen in subtitle list viewing mode, (b) shows an example of the screen in normal viewing mode. It is a flowchart of the subtitle management data generation process which the control part of the mobile phone which concerns on 1st Embodiment performs. It is a flowchart of the subtitle sentence scroll process which the control part of the mobile telephone which concerns on 1st Embodiment performs. It is a flowchart of a subtitle switching time display process executed by the control unit of the mobile phone according to the first embodiment. (A), (b), (c), (d) is an example of a screen displayed on the LCD by the display unit when the caption switching time display process is executed. (A), (b), (c), (d) is an example of a screen displayed on the LCD by the display unit when the caption switching time display process is executed. (A), (b), (c), (d) is an example of a screen displayed on the LCD by the display unit when the caption switching time display process is executed. It is a flowchart of the reproduction | regeneration time display process which the control part of the mobile telephone which concerns on 2nd Embodiment performs. It is a block diagram which shows the structural example of the mobile telephone which concerns on 3rd Embodiment. It is a flowchart of the reproduction time display process which the control part of the mobile telephone which concerns on 3rd Embodiment performs. (A), (b), (c), (d) is an example of a screen displayed on the LCD by the display unit during execution of the reproduction time display process.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1, 2 ... Mobile phone, 18 ... Memory | storage part, 21 ... Display part, 22 ... Operation part, 23 ... Control part, 25 ... Timer

Claims (12)

A plurality of subtitle sentences and a plurality of videos are stored in association with each other based on the reproduction order, and the subtitle number information indicating the number of subtitles of the plurality of subtitle sentences and the total reproduction time information indicating the total reproduction time of the content Storage means for adding and storing subtitle switching time information indicating subtitle switching time for switching to each subtitle sentence to each subtitle sentence;
Selecting means for selecting any subtitle sentence from a plurality of subtitle sentences stored in the storage means;
Subtitle playback means for playing back a caption text from the caption text selected by the selection means according to the playback order;
Video playback means for playing back video corresponding to the caption text being played back by the caption playback means;
Obtaining means for obtaining the number of subtitles indicated by the subtitle number information, the total reproduction time indicated by the total reproduction time information, and the subtitle switching time indicated by the subtitle switching time information added to each subtitle sentence;
Image generation means for generating a first image indicating the length of the acquired total reproduction time and second images corresponding to the acquired number of subtitles;
Image synthesizing means for synthesizing the second image at a position corresponding to the caption switching time of the first image, and generating an image;
Display means for displaying a composite image generated by the image composition means;
A video playback apparatus with captions, comprising: The image generation means generates a third image and a fourth image different from the third image as the plurality of second images,
The image synthesizing unit synthesizes the third image at a position corresponding to the subtitle switching time corresponding to the subtitle sentence being reproduced by the subtitle sentence reproducing unit of the first image, and the image other than the position is synthesized. Synthesizing the fourth image at a position corresponding to the caption switching time;
The video reproduction apparatus with captions according to claim 1. The third image has a predetermined color,
The fourth image has a color different from the predetermined color.
The video reproduction apparatus with captions according to claim 2. The first image is a line segment or a first bar;
One end of the line segment or the first bar corresponds to the top of the content,
The video playback device with captions according to any one of claims 1 to 3. The third image is a line segment having a predetermined thickness,
The fourth image is a line segment having a thickness different from the predetermined thickness.
The video reproduction apparatus with captions according to any one of claims 2 to 4. The third image is a circle having a predetermined diameter;
The fourth image is a circle having a diameter different from the predetermined diameter.
The video reproduction apparatus with captions according to claim 2 or 3. The image generation means has a lightness higher than the lightness of the second image by a predetermined lightness, and the subtitle sentence being reproduced by the subtitle sentence reproduction means of the first image from the one end. Generate a second bar with a length to the position corresponding to the caption switching time,
The image composition means superimposes one end of the second bar and one end of the first bar to compose the composite image.
The video playback apparatus with captions according to any one of claims 4 to 6. The image synthesizing unit synthesizes the second bar with the synthesized image so as not to overlap the second image;
The video playback device with captions according to claim 7. Video switching time information indicating video switching time for switching to each video is added to each video,
The acquisition means further acquires a video switching time indicated by video switching time information added to the video being played back by the video playback means,
The image generation means further generates a fifth image,
The image synthesizing unit synthesizes the fifth image at a position corresponding to the video switching time acquired by the acquiring unit of the first image;
The video playback device with captions according to any one of claims 1 to 8. The subtitle-added video playback device further includes timer means for measuring a time from when the content is played back until the video being played back by the video playback means is played back,
The image generation means generates a sixth image;
The image synthesizing unit synthesizes the sixth image at a position corresponding to a length of time measured by the timer unit of the first image;
The video playback device with captions according to any one of claims 1 to 9. A video playback method with subtitles performed by a video playback device with subtitles,
One of the subtitles is selected from among a plurality of subtitle signals that define a plurality of subtitle sentences and a plurality of video signals that define the video associated with the subtitle signal based on the playback order. A selection step for selecting a sentence;
A subtitle playback means for playing back the subtitle text from the subtitle text selected in the selection step according to the playback order;
Video playback means for playing back video corresponding to the caption text being played back in the caption playback step;
An acquisition step in which the acquisition means acquires the number of subtitles of the plurality of subtitle sentences, the total playback time of the content, and the subtitle switching time at which each subtitle sentence is switched;
An image generation step in which the image generation means generates a first image indicating the length of the total reproduction time acquired in the acquisition step and second images corresponding to the number of subtitles acquired in the acquisition step. When,
An image synthesis step for generating a synthesized image in which the image synthesizing unit synthesizes the second image at a position corresponding to the caption switching time of the first image;
The display unit, and a display step of displaying the composite image generated in the image synthesizing step,
A video playback method with subtitles, comprising: On the computer,
A plurality of subtitle sentences and a plurality of videos are stored in association with each other based on the reproduction order, and the subtitle number information indicating the number of subtitles of the plurality of subtitle sentences and the total reproduction time information indicating the total reproduction time of the content And subtitle switching time information indicating the subtitle switching time for switching to each subtitle sentence is added to each subtitle sentence and stored, and
A function for selecting one of subtitle sentences from a plurality of stored subtitle sentences;
A function for reproducing subtitle sentences from the selected subtitle sentences in accordance with the reproduction order;
A function to play back the video corresponding to the subtitle text being played,
A function of acquiring the number of subtitles indicated by the subtitle number information, the total reproduction time indicated by the total reproduction time information, and the subtitle switching time indicated by the subtitle switching time information added to each subtitle sentence;
A function of generating a first image indicating the length of the acquired total playback time and a second image corresponding to the acquired number of subtitles;
A function of generating a composite image obtained by combining the second image at a position corresponding to the caption switching time of the first image;
A function of displaying the generated composite image;
A program characterized by realizing.

Images