JP2006140777A - Moving picture reproducing apparatus, moving picture reproducing method, and moving picture reproducing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a moving picture reproducing apparatus, a moving picture reproducing method, and a moving picture reproducing program capable of efficiently carrying out the reproduction, temporary stop and restarting or like of a moving picture file and synchronizing video with audio. <P>SOLUTION: A second control section 22 acquires the key frame information of video data closest to a reproduction start position and thereafter obtains an audio reproduction start position of audio data closest to the acquired reproduction start position around the key frame information in order to take synchronization with the video data. The second control section 22 transfers the video data to a display buffer 62 according to the key frame information, writes the audio data to a RAM 24 according to the audio reproduction start position, decodes the video data and the audio data, respectively, and reproduces the video data by a display section 6 and the audio data by speakers 7R, 7L. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a moving image reproducing apparatus, a moving image reproducing method, and a moving image reproducing program.

In recent years, a technique for compressing moving image data with high efficiency and transmitting the stream, and recording and reproducing the transmitted moving image data as a file has been widely used. When a moving image file is played back using such a technique, it is usually assumed that the moving image file is played from the beginning. Therefore, if playback is attempted from the middle, there may be a time lag between video and audio. In view of this, a technique has been proposed in which video and audio are synchronized when a moving image file compressed by the MPEG1 method is reproduced halfway (see, for example, Patent Document 1).
JP 07-170490 A

  Incidentally, an ASF (Advanced Systems Format) format, which is a streaming distribution data format, is known as a technology with a high compression rate and little image degradation. For example, a program (video file) recorded by a hard disk recorder is converted into an ASF format video file and stored in a recording medium such as an SD card, and the recording medium is stored in a portable terminal capable of wide display. A method of using such a program (video file) has been proposed.

  However, the ASF format has a problem that it cannot be played back or paused for a long time movie file, and it takes time to play back.

  Therefore, the present invention provides a moving image reproducing apparatus, a moving image reproducing method, and a moving image reproducing program capable of efficiently reproducing, pausing, and restarting a moving image file and synchronizing video and audio. With the goal.

  In order to achieve the above object, a video playback apparatus according to a first aspect of the present invention is a video playback apparatus that plays back a video file composed of a plurality of sets of video data and audio data so that the video file is played back halfway. A playback instruction means for instructing, and when playback is instructed by the playback instruction means, a key frame acquisition means for acquiring a key frame of video data closest to a playback start position instructed to be played halfway from a plurality of video data; Audio data acquisition means for acquiring audio data closest to the periphery of the key frame acquired by the key frame acquisition means from the plurality of audio data, and decoding of video data from the key frame acquired by the key frame acquisition means At the same time, the decoding of the audio data acquired by the audio data acquisition means is started and Characterized by comprising a reproduction control means for.

  Further, as a preferred aspect, for example, as in claim 2, in the moving image reproduction apparatus according to claim 1, the audio data acquisition means is configured to obtain a difference between a reproduction start position of the video data and a reproduction start position of the audio data. The audio data may be obtained so that is less than the allowable time difference.

  As a preferred mode, for example, as described in claim 3, in the video playback device according to claim 1 or 2, the video file may be in an ASF (Advanced Systems Format) format.

  According to another aspect of the present invention, there is provided a moving image reproducing method for reproducing a moving image file composed of a plurality of sets of video data and audio data, wherein the moving image file is reproduced halfway. A reproduction instruction step for instructing to perform the reproduction, and when reproduction is instructed in the reproduction instruction step, acquisition of a key frame for acquiring a key frame of the video data closest to the reproduction start position where the intermediate reproduction is instructed from a plurality of video data Audio data acquisition step for acquiring audio data closest to the periphery of the key frame acquired in the key frame acquisition step from the plurality of audio data, and video from the key frame acquired in the key frame acquisition step The decoding of the data is started, and the audio data acquired by the audio data acquisition step is Wherein a reproduction control step of reproducing the start of the decoding of the data, in that it consists of.

  In order to achieve the above object, a moving picture reproducing program according to the invention described in claim 5 is a moving picture reproducing program for reproducing a moving picture file composed of a plurality of sets of video data and audio data. A reproduction instruction step for instructing to perform the reproduction, and when reproduction is instructed in the reproduction instruction step, acquisition of a key frame for acquiring a key frame of the video data closest to the reproduction start position where the intermediate reproduction is instructed from a plurality of video data Audio data acquisition step for acquiring audio data closest to the periphery of the key frame acquired in the key frame acquisition step from the plurality of audio data, and video from the key frame acquired in the key frame acquisition step Data decoding starts and is acquired by the voice data acquisition step Characterized in that to execute a and a reproduction control step of reproducing the start of the decoding of the audio data to the computer.

  According to the first aspect of the present invention, when the playback instruction means instructs to play the moving image file halfway, the key frame acquisition means is closest to the playback start position where halfway playback is instructed from a plurality of video data. The key frame of the video data is acquired, and the audio data acquisition unit acquires the audio data closest to the periphery of the acquired key frame from the plurality of audio data. Then, the playback control means starts decoding the video data from the acquired key frame and starts decoding the acquired audio data and plays it back. And the advantage of being able to synchronize video and audio.

  According to a second aspect of the present invention, the audio data acquisition means obtains the audio data so that a difference between the reproduction start position of the video data and the reproduction start position of the audio data is equal to or less than an allowable time difference. Therefore, there is an advantage that the deviation between the reproduction of the video and the audio can be minimized.

  According to the third aspect of the present invention, since the moving image file is in the ASF (Advanced Systems Format) format, the moving image file efficiently reproduces, pauses, or resumes the moving image file in the streaming distribution data format. And video and audio can be synchronized.

  According to the fourth aspect of the present invention, when an instruction to play a moving image file from the middle is given, a key frame of the video data closest to the playback start position at which the halfway playback is instructed is obtained from a plurality of video data, The sound data closest to the obtained key frame periphery is acquired from the plurality of sound data. And since the decoding of the video data is started from the acquired key frame and the decoding of the acquired audio data is started and played back, the video file can be played, paused, resumed, etc. efficiently And the advantage that the video and the audio can be synchronized can be obtained.

  According to a fifth aspect of the present invention, a reproduction instruction step for instructing to reproduce a moving image file in the middle, and reproduction instructed to be reproduced from a plurality of video data when reproduction is instructed in the reproduction instruction step A key frame acquisition step for acquiring a key frame of video data closest to the start position; an audio data acquisition step for acquiring audio data closest to the periphery of the key frame acquired in the key frame acquisition step from a plurality of audio data; A reproduction control step of starting decoding of the video data from the key frame acquired in the key frame acquisition step and starting decoding of the audio data acquired in the audio data acquisition step To play, pause, resume, etc. Rate manner can be performed, and the advantage is obtained that it is possible to synchronize the video and audio.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

A. Configuration of Embodiment Next, FIG. 1 is an external view corresponding to a front view of a state in which a lid of a mobile phone 1 according to this embodiment is opened and closed. 1A shows a state in which the cover of the mobile phone 1 is opened, FIG. 1B shows a state in which the cover of the mobile phone 1 is closed (lid side), and FIG. The state (back side) where the cover of the telephone 1 is closed is shown.

  As shown in the figure, the lid 2 can be opened and closed with respect to the main body 3 with the first hinge 100 as a rotation axis (arrow α), and can be rotated with the second hinge 12 as a rotation axis (arrow). β). The lid 2 includes a first speaker 5, a display unit 6, and a built-in antenna (not shown) for data communication. The first speaker 5 is a dynamic speaker and is disposed on the inner surface of the lid portion 2. The display unit 6 is arranged on the inner surface of the lid unit 2 and is a 2.7 inch (240 × 400 dot) TFT liquid crystal (having a resolution equivalent to QVGA or HVGA) color liquid crystal capable of displaying an image with an aspect ratio of 16: 9. Consists of.

  The first hinge part 100 is connected to the lid part 2 and rotates integrally with the lid part 2. The first hinge part 100 is fixed to the main body part 3 side and is provided with a pair of left and right parts provided on both sides of the movable part 101. It is comprised by the fixing | fixed part 102,103.

  The main body 3 includes a key input unit 8 and a microphone 10. The key input unit 8 includes a cross key 81 for moving the focus during menu display, a determination key 82 for determination, an address book key 83 for browsing address book data, and a network connection key 84 for instructing network connection. , A movie playback key 85 for starting the movie playback mode, a mail key 86 for starting the mail mode (reception, creation, editing, transmission), an off-hook key 87, a clear key 88 (cancel detection), an on-hook key 89 ( All clear: Jump instruction to wait for incoming call), and a numeric keypad 810 for inputting a telephone number and a character symbol. In addition, on the back surface of the main body, an imaging lens 17 for photographing, and speakers 7R and 7L for reproducing ringtones, alarms, and melody are arranged.

  Next, FIG. 2 is a block diagram showing a circuit configuration of the mobile phone 1 according to the present embodiment. When the reception signal processing unit 18 receives an instruction to start the data communication mode from the switch circuit 201, the reception signal processing unit 18 starts negotiation (terminal authentication, line connection) with an external wireless base station, and temporarily occupies the communication bandwidth. I do. Then, a signal that is code-modulated to the own device is extracted from the radio signal received by the built-in antenna 16 and is output to the communication data processing unit 202 via the switch circuit 201.

  The communication signal processing unit 19 is always supplied with power both in the standby state and in the data communication mode, and includes a digital modulation signal (transmission voice data, mail data, network connection information) from the communication data processing unit 202. Packet data) is code-modulated and output from the main antenna 11. On the other hand, a signal which is transmitted from the radio base station and received by the main antenna 11 and code-modulated to its own device is extracted and output to the communication data processing unit 202 via the switch circuit 201. .

  Note that the built-in antenna 16 and the main antenna 11 are in a positional relationship effective for space diversity reception, regardless of whether the lid 2 is rotated or closed.

  The first control unit 20 performs processing centering on connection control with information resources such as a radio base station and a database connected via a network infrastructure such as a communication service provider or the Internet at the subsequent stage. In the present embodiment, the cellular phone 1 has two control units, and the communication processing in general is the first control unit 20, and other shooting processing, image processing, and video reproduction processing will be described later. This is performed by the second control unit 22.

  The switch circuit 201 receives a control signal from the communication data processing unit 202, and controls the supply of power to the reception signal processing unit 18 and the communication signal processing unit 19 and the end of processing operation. Specifically, in the normal power-on state or the reception ON mode state, power is supplied only to the communication signal processing unit 19 to operate, while the user operates the network connection key 84, or When the start of the data communication mode is instructed according to a preset schedule, the reception signal processing unit 18 and the communication signal processing unit 19 are operated.

  The communication data processing unit 202 modulates CELP digital audio data output from the audio processing unit 205 into a signal corresponding to the PSK system, and modulates the signal into a spread code, and also receives the received signal processing unit 18 and the communication signal processing unit 19. A process of decoding the code-modulated signal received by the digital signal of the PSK system is performed.

  The system ROM 203 stores a control program for communication control. As a feature of the present embodiment, an operation switching control program between the reception signal processing unit 18 and the communication signal processing unit 19 is stored in addition to a normal communication negotiation program. Thus, when the start of the data communication mode is instructed, space diversity reception is performed by the main antenna 11 and the built-in antenna 16 to temporarily occupy the communication bandwidth, and the reception signal processing unit 18, communication signal processing The code-modulated signals generated by both units 19 are decoded and synthesized into digital signals almost simultaneously, and a program output to the bus 200 via the interface processing unit 204 is stored.

  The audio processing unit 205 includes a CELP audio modulation / demodulation system and an MPEG audio audio demodulation system that accompanies moving image demodulation, and the analog audio output from the audio processing unit 205 is the first in a call state. Output from the speaker 5. In the movie playback mode, the switch 31 controls the second control unit 22 to send audio signals to both the second speaker 7R and the third speaker 7L when the audio data stored in the movie file is stereo-compatible. Output. The second speaker 7L is also driven by the notification driver 26, and reproduces and outputs an incoming notification sound.

  The second control unit 22 controls the movie file downloaded in the data communication mode in the movie playback mode, the still image / moving image playback display process in the camera mode, and the overall shooting process in the camera mode. In particular, in the present embodiment, the second control unit 22 obtains the playback start time of the video data and the playback start time of the audio data during the halfway playback of the ASF format moving image file, and synchronizes the video data and the audio data. Perform this process. More specifically, it is as follows.

  A moving image file in the ASF file format has an area called “Simple Index Object”, which contains video key frame information. By using the key frame information, the video and audio are synchronized, and the search method at the time of synchronization is taken into consideration, thereby increasing the speed. For example, in the case of playback from the middle, the playback start time is specified and playback is started, but when the given playback start position and the total playback time of the ASF format video file are encoded. The information of “Simple Index Object” is determined from the key frame (reference frame, I frame, I picture) insertion interval of the key frame closest to the designated reproduction start position. Then, the reproduction of the video starts from the key frame. Since playback is from a key frame, intra-frame decoding is performed on the first frame. For audio, data having an audio reproduction time closest to the reproduction start time around the key frame is determined. A predetermined range is determined around the key frame so that the difference between the playback time and the playback time of the audio is minimized. If the minimum value does not fall within the allowable range of deviation between video and audio, the predetermined range is expanded to perform a search so that it falls within the allowable range. By using the key frame information, it is possible to narrow down the search start position of the voice for synchronization, and further increase the search range step by step, so that there is no need to perform a useless search.

  In the present embodiment, parallel processing with the first control unit 20 described above can be performed at the time of an incoming call interruption by the mobile phone function. It is assumed that the processing unit 204 performs.

  The program ROM 23 stores processing programs related to the synchronization of video data and audio data during movie playback mode, camera mode, playback of moving image files, and playback during playback. The RAM 24 stores various data necessary for data communication processing and voice communication.

  Next, the compression / decompression processing unit 25 complies with MPEG-2 and -4, compresses still images / moving images shot in the camera mode, and downloads the movie in the data communication mode in the movie playback mode. Video data is decompressed from a file, a still image / moving image in the camera mode, and an ASF format moving image stored in a card medium 331 such as an SD card.

  The notification driver 26 operates the notification LED 13 and the third speaker 7L in order to sound a ringtone or an alarm notification sound. The switch processing unit 27 outputs control signals to the first control unit 20 and the second control unit 22 in response to operation detection from the key input unit 8, the first side key 90, the second side key 91, and the shutter key 92. To do.

  In the subsequent stage of the camera lens 17, an imaging device 171 including an image sensor such as a CCD or CMOS and a step motor for optical zoom, and an analog signal captured by the imaging device are converted into a digital signal, A DSP 172 for generating image data is provided. The effective pixel area of the image sensor in the present embodiment is fixed to an aspect ratio of 4: 3. The cradle connector 32 is for charging a battery (not shown) with electric power from the cradle when it is installed on a cradle, that is, an installation table that also serves as a charging table, and is connected to the bus via the connector interface 31. ing. A recording medium 331 such as an SD card is mounted in the memory card slot 330 (33).

  Here, FIG. 3 is a conceptual diagram showing a data structure of a moving image file in the ASF format. In the ASF format moving image file, the head is “Header Object”, and the next is “Data Object” in which actual data is stored. The “Data Object” is composed of a plurality of Data Packets, and each Data Packet is composed of a Video payload and an Audio payload. The last “Index Object” is a portion including information such as a key frame, and “Simple Index Object” is also one of “Index Objects”.

  FIG. 4 is a conceptual diagram showing the data structure of the “Simple Object”. “Object ID” is an identification code of “Simple Index Object”. “Object Size” indicates the size of “Simple Index Object”. “Index Entry Time Interval” indicates the key frame insertion interval, and “Maximum Packet Count” indicates the maximum number of “Data Packets” in which the key frame is configured. “Index Entries Count” is the total number of “Index Entries” that follow. Each element of “Index Entries” is key frame information, “Packet Number” is the number of “Data Packet” at which the key frame is started, and “Packet Count” is the number of “Data Packets”. Indicates whether the key frame is configured.

B. Operation of Embodiment Next, the operation of the above-described embodiment will be described.

  FIG. 5 is a flowchart for explaining the operation of the mobile phone 1 according to the present embodiment. First, when the user reads out an ASF format moving image file and playback is instructed from the middle (step S10), the second control unit 22 starts from the playback start position based on the information of “Simple Index Object”. Frame information is acquired (step S12). The acquisition method determines the element number of “Index Entries” by dividing the reproduction start position by “Index Entry Time Interval” of “Simple Index Object” which is the key frame insertion interval. The key frame information becomes the key frame information of the video data closest to the reproduction start position.

  Next, audio data at a timing at which no time difference from the searched video data is generated is searched (step S14). Since the ASF format moving image file has a data structure as shown in FIG. 3, a reproduction start Audio payload having the smallest time difference from the reproduction start time is obtained.

  In the data structure of the ASF moving image file, the Video payload and the Audio payload each have elapsed time information from the first position. The unit is msec. Here, FIG. 6 is a conceptual diagram showing a method of retrieving voice data. In FIG. 6, the long and narrow block corresponds to “Data Object”. Also, the arrow 100 is the position of “Data Packet” of “Packet Number” of the key frame closest to the previously determined start time.

  The “Data Packet” including the audio data closest to the reproduction start position is obtained around this “Data Packet”. Since each Audio payload has elapsed time information from the first position, “Data Packet” is set within a predetermined range of A and B in FIG. 6 (for example, a total of 200 Data Packets with 100 Data Packets before and after). Scan and find the playback start position from the Audio payload included in each “Data Packet”, and it will be less than the preset allowable minimum difference (for example, 50 msec), or otherwise all scans will be minimum The position of “Data Packet” is obtained. The audio data is transferred to the buffer from that position, that is, “Data Packet” of the “Packet Number”.

If the minimum value of the difference from the video data at the reproduction start position is 1000 msec, that is, 1 second or more in scanning between AB, it may not be said that the video data and the audio data are synchronized. Therefore, the range is further expanded, and “Data Packet” in the range of C and D is also scanned to obtain “Packet Number” which is the position of the Data Packet where the difference is the smallest within the range of A to D. . In this way, it is possible to cover even when the synchronization between the ABs is insufficient.

  In addition, when the search of the range between AB is insufficient, and the synchronization error is the smallest in the range of B, it is more likely that D closer to B has the smallest synchronization error. The range of D is scanned first. As a result, the allowable difference (for example, 50 msec) set in advance in the search of the range D is less than or equal to, and the search can be completed only by scanning within the range.

  Next, the second control unit 22 accesses the card medium 331 such as an SD card according to the key frame information indicating the key frame of the video data closest to the reproduction start position obtained in step S12, and displays “ “Packet Number” and “Packet Count” that are “Index Entry” are acquired, and for the video data, the “Data Packet” data of the “Packet Number” is accessed and transferred to the display buffer 62, while the audio data is transferred. Writes “Data Packet” including the audio payload including the audio data in, for example, the RAM 24 (step S16).

  Next, the compression / decompression processing unit 25 starts decoding from the video data in the display buffer 62, and the audio processing unit 105 decodes the audio data written in the RAM 24 to reproduce video and audio respectively. Reproduction is performed on the display unit 6 and the speakers 7R and 7L, which are blocks (step S18).

  According to the above-described embodiment, when playback from the middle of a video file is instructed, based on the playback start position, the total playback time when all video files are played back, and the key frame insertion interval during video encoding Since the key frame closest to the reproduction start position is obtained, and the reproduction position of the audio data closest to the reproduction start position is obtained as the audio reproduction start position based on the information of the key frame, the ASF format It is possible to efficiently reproduce, pause, resume, etc. a moving image file and to synchronize video and audio.

  In the above-described embodiment, the mobile phone has been described as the video playback device. However, the mobile phone is not limited to this, and may be a personal digital assistant (PDA) or a video playback device specialized for video and still image playback.

It is an external view equivalent to the front view of the state which opened and closed the cover of the mobile telephone 1 which concerns on embodiment of this invention. It is a block diagram which shows the circuit structure of the mobile telephone 1 by this embodiment. It is a conceptual diagram which shows the structure of the moving image file of the ASF format by this embodiment. It is a conceptual diagram which shows the data structure of "Simple Object" of the moving image file of ASF format. 4 is a flowchart for explaining the operation of the mobile phone 1 according to the present embodiment. It is a conceptual diagram for demonstrating operation | movement (search of an audio | voice) of the mobile telephone 1 by this embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Cellular phone 2 Cover part 3 Main-body part 5 1st speaker 6 Display part (display means)
7R 2nd speaker 7L 3rd speaker 8 Key input part (reproduction instruction means)
10 Microphone 11 Main antenna 12 Second hinge 13 LED for notification
16 built-in antenna 17 imaging lens 18 received signal processing unit 19 communication signal processing unit 20 first control unit 22 second control unit (key frame acquisition unit, audio data acquisition unit, reproduction control unit)
23 Program ROM
24 RAM
25 Compression / decompression processing unit 26 Notification driver 27 Switch processing unit 30 Audio output system driver 31 Switch 61 Display driver 62 Display buffer 90 First side key 91 Second side key 92 Shutter key 100 First hinge unit 201 Switch circuit 202 Communication data Processing unit 203 System ROM
205 Audio Processing Unit 206 Switch Circuit 207 Audio Amplifier 330 Memory Card Slot 331 Card Medium

Claims (5)

A video playback device for playing back a video file consisting of a plurality of sets of video data and audio data,
Replay instruction means for instructing to replay the video file from the middle;
When playback is instructed by the playback instruction means, key frame acquisition means for acquiring a key frame of video data closest to the playback start position for which halfway playback is instructed from a plurality of video data;
Audio data acquisition means for acquiring audio data closest to the periphery of the key frame acquired by the key frame acquisition means from the plurality of audio data;
Playback control means for starting decoding of video data from the key frame acquired by the key frame acquisition means and starting decoding of the audio data acquired by the audio data acquisition means and playing back A video playback device characterized by the above.   2. The moving image reproduction according to claim 1, wherein the audio data obtaining unit obtains the audio data so that a difference between the reproduction start position of the video data and the reproduction start position of the audio data is equal to or less than an allowable time difference. apparatus.   The moving image reproduction apparatus according to claim 1, wherein the moving image file is in an ASF (Advanced Systems Format) format. A video playback method for playing back a video file consisting of a plurality of sets of video data and audio data,
A playback instruction step for instructing to play the video file from the middle;
When playback is instructed in this playback instruction step, a key frame acquisition step of acquiring a key frame of video data closest to the playback start position for which halfway playback is instructed from a plurality of video data;
An audio data acquisition step for acquiring audio data closest to the periphery of the key frame acquired in the key frame acquisition step from the plurality of audio data;
A decoding control step of starting decoding of video data from the key frame acquired in the key frame acquisition step and starting decoding of the audio data acquired in the audio data acquisition step and playing back A reproduction control method characterized by the above. A video playback program for playing back a video file consisting of multiple sets of video data and audio data,
A playback instruction step for instructing to play the video file from the middle;
When playback is instructed in this playback instruction step, a key frame acquisition step of acquiring a key frame of video data closest to the playback start position for which halfway playback is instructed from a plurality of video data;
An audio data acquisition step for acquiring audio data closest to the periphery of the key frame acquired in the key frame acquisition step from the plurality of audio data;
A reproduction control step of starting decoding of video data from the key frame acquired in the key frame acquisition step and starting decoding of the audio data acquired in the audio data acquisition step and playing back to the computer. A video playback program characterized by being executed.

Images