JP2008160455A - Portable terminal device with streaming reproduction function - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To improve convenience and user-friendliness by making time to be traced back variable in accordance with the length of interruption time of streaming reproduction and restarting reproduction from scene switching. <P>SOLUTION: This portable terminal device with a streaming reproduction function is provided with a storing part 13 and a control part 12 for receiving streaming data and reproducing the streaming data. When reproducing the streaming data, the control part 12 stores reproduction time in the storing part 13 in accordance with the reproduction of the streaming data and monitors a data amount of each frame, and when there is a frame in which at least a data amount is equal to or more than a predetermined value, the control part 12 stores information specifying the frame in the storing part. When interrupt processing occurs during reproducing the streaming data to interrupt the reproduction and the interrupted reproduction of the streaming data is restarted, the control part 13 traces back more than the reproduction time of the interruption and performs control so as to make the frame in which a data amount specified on the basis of information stored in the storing part 13 is equal to or more than the predetermined value a reproduction restarting position. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a portable terminal device with a streaming reproduction function that receives and reproduces streaming transmitted from a server via a communication network.

  Streaming refers to a method of decoding and simultaneously reproducing encoded data while viewing multimedia such as images and sounds transmitted from a server via a communication network such as the Internet. For this reason, a terminal such as a mobile terminal device can implement real-time viewing of multimedia data by installing a streaming application program.

By the way, conventionally, a streaming data reproduction system is known in which, when an environment for reproducing streaming data changes, data reproduction can be continued in accordance with the change in the environment (see, for example, Patent Document 1). .
JP 2003-324444 A

According to the technique disclosed in Patent Document 1 described above, when streaming reproduction is resumed, it can be resumed from the interrupted position or after a predetermined time. However, in the case of the technique disclosed in Patent Document 1, depending on the position to be resumed, it is rare to resume from a well-delimited place such as a scene change, which is undesirable from the service receiving side. I can say that.
In streaming playback of mobile terminal devices such as mobile phones, generally, RTSP (Real Time Streaming Protocol) / RTP (Real-time Transport Protocol) protocol is used, and moving picture data is MPEG4 (Moving Picture Experts Group level-4). ) Format data is used, but in order to be able to resume from a well-delimited place during streaming playback, the method of accumulating all of the streaming data in the streaming buffer and determining the resume location is especially portable. It is not realistic for the phone due to memory limitations.

  The present invention has been made in order to solve the above-described problems. Even if streaming playback is interrupted without increasing the capacity of the streaming buffer, it is possible to restart from a well-delimited point such as scene switching. An object of the present invention is to provide a portable terminal device with a streaming reproduction function, which is improved in convenience and usability.

  In order to solve the above problems, a mobile terminal device with a streaming playback function according to the present invention includes a storage unit that stores information, a receiving unit that receives streaming data including a plurality of frames from a communication network, A controller that reproduces streaming data, and when the controller reproduces the streaming data, the controller stores the reproduction time in the storage unit according to the reproduction of the streaming data and monitors the data amount for each frame. If there is at least a frame whose data amount is greater than or equal to a predetermined value, information for identifying the frame is stored in the storage unit, interrupt processing occurs during playback of streaming data, playback is interrupted, and the interrupted streaming data When resuming the playback of the A frame data amount specified based on the information stored in the storage unit is equal to or greater than a predetermined value and reproduction resuming position.

  Further, in the mobile terminal device with streaming playback function of the present invention, a storage unit for storing information, a reference frame in which a reference image is encoded, and a difference from the reference frame is encoded, so that the reference frame A receiving unit that receives streaming data including a difference frame that forms a moving image by being reproduced from the communication network, and a control unit that reproduces the streaming data, the control unit including the streaming data When reproducing the streaming data, the reproduction time is stored in the storage unit according to the reproduction of the streaming data, and it is determined whether the frame to be reproduced is a reference frame or a difference frame. For example, information that identifies this is stored in the storage unit, and streaming data is being played When the interrupt processing occurs and the playback is interrupted, and the playback of the interrupted streaming data is resumed, the reference frame specified based on the information stored in the storage unit is played back from the playback time at the time of the interrupt. You may comprise so that it may be set as a restart position.

  Further, in the mobile terminal device with streaming playback function of the present invention, a storage unit for storing information, a reference frame in which a reference image is encoded, and a difference from the reference frame is encoded, and the reference frame is A receiving unit that receives streaming data including a difference frame that forms a moving image by being played back from a communication network; and a control unit that plays back the streaming data. When reproducing, the reproduction time corresponding to the reproduction of the streaming data is stored in the storage unit, and it is determined whether the frame to be reproduced is a reference frame or a difference frame, and if there is a frame to be a reference frame, Information for identifying this is stored in the storage unit, and if it is a difference frame, the amount of data is limited. If the frame is greater than or equal to the value, information for identifying the frame is stored in the storage unit, the interruption process occurs during the reproduction of the streaming data, the reproduction is interrupted, and when the reproduction of the interrupted streaming data is resumed, The reference frame specified based on the information stored in the storage unit and the difference frame where the amount of data is greater than or equal to a predetermined value is set as the playback restart position, which is earlier than the playback time at the time of interruption. Also good.

  Further, in the mobile terminal device with a streaming playback function according to the present invention, when the streaming playback is resumed, based on the time when the streaming playback is suspended, the control unit determines the playback time when the playback is resumed. You may comprise so that the amount of time to go back may be specified.

  The mobile terminal device with a streaming playback function according to the present invention further includes an operation unit for instructing fast forward and / or rewind, and the control unit starts a restart process when restarting the interrupted streaming playback. Then, when fast forward or rewind is instructed by the operation unit within a predetermined time, streaming time may be resumed by correcting the amount of time going back from the playback time at the time of interruption.

  In the mobile terminal device with a streaming playback function according to the present invention, the control unit stores the communication information when acquiring the streaming data in the storage unit as cache information, and resumes the playback of the interrupted streaming data. In some cases, the communication unit may be reconnected to the streaming data supply source on the basis of the cache information so that the streaming data acquisition is resumed.

  According to the present invention, even if streaming playback is interrupted without increasing the capacity of the streaming buffer, the retroactive time can be made variable according to the length of time, and scenes can be switched from a well-delimited location. It is possible to resume the operation, which provides convenience to the user and improves usability.

FIG. 1 is a block diagram showing an internal configuration of a mobile terminal device with a streaming playback function according to an embodiment of the present invention.
Here, a mobile phone 10 is illustrated as a mobile terminal device with a streaming playback function. As shown in FIG. 1, the mobile phone 10 includes a communication unit 11, a control unit 12, a storage unit 13, a voice processing unit 14, a speaker 15 (SP), a microphone 16 (MIC), and a display unit. 17 and the operation unit 18.

The communication unit 11 transmits and receives radio signals to and from the base station via a channel assigned by any one of base stations (not shown).
The control unit 12 comprehensively controls the overall operation of the mobile phone 10. That is, various processes of the mobile phone 10 (voice calls performed via a circuit switching network, creation and transmission / reception of e-mail, browsing of the Internet Web (World Wide Web) site, streaming playback, etc.) are performed by the operation unit 18. The operation of each block described above (transmission / reception of signals in the communication unit 11, input / output of audio in the audio processing unit 14, display of images in the display unit 17, etc.) is controlled so as to be executed according to an appropriate procedure. .

The control unit 12 includes a CPU (microprocessor) that executes processing based on a program (OS, application program, etc.) stored in the storage unit 13, and executes the above-described processing according to a procedure instructed in the program. To do. That is, instruction codes are sequentially read from programs such as an OS and application programs stored in the storage unit 13 and executed.
Particularly related to the present invention, the control unit 12 receives and buffers moving image data distributed from a server (not shown) connected via the Internet's Web (communication network), and stores stored data. Is decoded and reproduced, that is, when streaming reproduction is performed, the reproduction time is monitored, the information amount for each frame is monitored, and the reproduction time of a frame in which the information amount is a predetermined value or more is stored. Then, during streaming playback, it is interrupted by the occurrence of an interrupt, and after that, when streaming playback is resumed, it goes back from the playback time at the time of interruption and restarts from the most recent frame whose frame information amount is a predetermined value or more. Has a function to control. Details of the functional configuration at this time will be described later with reference to FIG.

The storage unit 13 stores various data used for processing in the control unit 12. For example, a program provided in the control unit 12, an address book for managing personal information such as a telephone number or an e-mail address of a communication partner, an audio file for playing a ring tone or an alarm sound, an image file for a standby screen, streaming data (Moving image data), various setting data, and temporary data used in the process of the program.
The above-described storage unit 13 is configured by, for example, a nonvolatile storage device (nonvolatile semiconductor memory, hard disk device, optical disk device, etc.), a randomly accessible storage device (eg, SRAM, DRAM), or the like.

  In addition, the audio processing unit 14 processes an audio signal output from the speaker 15 and an audio signal input from the microphone 16. That is, the voice input from the MIC 16 is amplified, analog-to-digital converted, further subjected to signal processing such as encoding, converted into digital voice data, and output to the control unit 12. Further, the audio data supplied from the control unit 12 is subjected to signal processing such as decoding, digital-analog conversion, amplification, etc., converted into an analog audio signal, and output to the SP 15. Here, an audio signal during streaming reproduction is output.

The display unit 17 is configured using a display device such as a liquid crystal display panel or an organic EL (Electro-Luminescence) panel, for example, and displays an image corresponding to the video signal supplied from the control unit 12. Here, the video signal at the time of streaming playback is displayed. In addition, various information and images such as the phone number of the callee at the time of outgoing call, the phone number of the other party at the time of incoming call, the contents of received mail and outgoing mail, date, time, remaining battery level, success / failure of outgoing call, standby screen Is also displayed.
The operation unit 18 includes keys to which various functions are assigned such as a power key, a call key, a numeric key, a character key, a direction key, a determination key, and a call key. Is operated, the signal corresponding to the operation content is generated and input to the control unit 12 as a user instruction. In relation to the present invention, for example, a button icon displayed on the display unit 17 is selected by the operation unit 18 and an operation for determining is performed, thereby instructing reproduction, fast-forwarding, rewinding, etc. of the streaming data.

FIG. 2 is a block diagram showing the functions of the internal configuration of the control unit 12 shown in FIG. 1, particularly focusing on streaming playback.
As shown in FIG. 2, the control unit 12 includes a main control unit 120, a streaming data receiving unit 121, a streaming buffer 122, a streaming playback control unit 123, a restart position table 124, and a scene switching time information table 125. A restart position calculation unit 126, a timer monitoring unit 127, an interruption time measurement processing unit 128, a table data storage processing unit 129, a threshold table 130, a table search unit 131, and a table correction processing unit 132. Is done.

The streaming data receiving unit 121 receives streaming data distributed from a server (not shown) via a communication network such as the Internet and stores the streaming data in the streaming buffer 122. Here, the streaming data reception unit 121 may be segmented such as scene switching. Time information of an I (intra) frame (reference frame) and a P (predictive) frame (difference frame) used for time determination of a place is supplied to the table data storage processing unit 129.
The streaming playback control unit 123 reads the video stored in the streaming buffer 122 based on playback, rewind, and fast-forward instructions operated via the operation unit 18, and displays a video signal obtained by decoding the video signal. The audio signal is supplied to the audio processing unit 14 and reproduced and output.

The restart position table 124 is a data table in which, for example, the relationship between the interruption time (seconds) and the restart position (rewind time) is set as a set by default, as shown in FIG. 3 as an example of the data structure. Yes, the resumption position (rewind time) when resuming after a reproduction interruption occurs is calculated and set by the resumption position calculation unit 126.
In FIG. 4, the resumption position (rewind time) when the interruption time is n seconds is expressed by the following equation so that the relationship between the interruption time (horizontal axis) and the resumption position (vertical axis) is displayed in a graph. It can be obtained by calculating (1). The relationship between the interruption time and the restart position (rewind time) is initially set to a default value, but is adjusted by the user performing a rewind or fast-forward operation during a certain period of time during the restart. Is done. Details will be described later.

(Equation 1)
m = m1 + (n−n1) × (m2−m1) / (n2−n1) (1)

On the other hand, in the scene switching time information table 125, as shown in FIG. 5 as an example of the data structure, the frame time information, here, all the I frames and the time information of the P frame exceeding the threshold (elapsed time). Time) is stored in association with each frame type.
In the portable terminal device with a streaming playback function according to the embodiment of the present invention, as described above, as a method for determining time information of a place where the segmentation is good, such as scene switching, an I frame (included in MPEG4 data) Using the time information of the reference image) and the P frame (difference from the immediately preceding frame), the elapsed time of the P frame whose information amount exceeds the threshold is held in the scene switching time information table 125. . This is based on the fact that it is possible to determine that the location where the difference in information amount is large is the scene switching position.

The table data storage processing unit 129 manages the processing for this purpose, and as shown in FIG. 5, the relationship between the elapsed time and the frame size (information amount) is shown. Elapsed time information is extracted and stored in the scene switching time information table 125. The process of storing scene switching time by the table data storage processing unit 129 will be described later with reference to the flowchart shown in FIG.
In FIG. 5, it is assumed that a bold line indicates an I frame, a thin line indicates a P frame, and a threshold relating to the amount of information is set in the threshold table 130.

On the other hand, the timer monitoring unit 127 monitors the current time, and the interruption time measurement processing unit 128 measures the streaming interruption time based on the timer value output from the timer monitoring unit 127. Details of the interruption time measurement processing by the interruption time measurement processing unit 128 will be described later with reference to the flowchart shown in FIG.
In addition, the table correction processing unit 132 is calculated by the restart position calculation unit 126 when a rewind instruction or fast-forward instruction is given via the operation unit 18, and the restart position (rewind time) held in the restart position table 124. ) Has a function to adjust information about. Details will be described later with reference to the flowchart shown in FIG.

  The main control unit 120 is responsible for the sequence control of each of the control blocks 123, 126, 128, 129, 131, and 132. Here, during streaming playback, voice reception, SMS (Short Message Service), mail reception When the interruption is received by the above or the like, the streaming reproduction is interrupted via the streaming reproduction control unit 123 for notification of incoming call or reception. In addition, when the interrupt event ends and the streaming playback is resumed, operations such as controlling the resume position calculation unit 126 and the table search unit 131 are performed in order to change the resume position according to the length of the playback interruption time. Further, when there is a rewind or fast-forward instruction via the operation unit 18 during a predetermined time during the calculation process for resumption, the table correction processing unit 132 is controlled and stored in the resumption position table 124. In addition, control for correcting the relationship between the interruption time and the restart position (rewind time) is also performed.

The functions of the respective blocks 120 to 132 described above are achieved by executing the respective programs stored in the storage unit 13 by the control unit 12. However, it does not indicate only the components that are separated from other blocks and built in, but is simply a representation of each processing unit separately for the sake of simplicity.
In addition, each of the above-described streaming buffer 122, restart position table 124, scene switching time information table 125, and threshold value table 130 is assumed to be allocated and stored in the storage unit 13.

Hereinafter, the operation of the mobile terminal device with the streaming playback function according to the embodiment of the present invention will be described in detail. Streaming playback means that continuous data such as video and audio is divided and downloaded sequentially from the desired server and buffered, and the previously buffered data is played back while unplayed data is downloaded simultaneously. And buffering technology.
FIG. 6 is a flowchart cited for explaining the basic operation of the mobile terminal device with the streaming playback function according to the embodiment of the present invention. The basic operation of the mobile terminal device with the streaming playback function according to the embodiment of the invention shown in FIG. 1 will be described below with reference to the flowchart shown in FIG.

In FIG. 6, first, the user starts the browser, connects the mobile phone 10 to a communication network such as the Internet, connects to a server that supplies moving pictures and the like, and is instructed to stream playback of desired data. Assume that streaming is started and the mobile phone 10 is currently in a streaming state (a state in which a series of processes of downloading, buffering, and decoding reproduction data is being performed) (S61). The browser is a program executed by the control unit 12, and communicates with a desired server or the like using a predetermined network protocol in the communication unit 11, and stores information used for communication in the storage unit 13 as a cache. Processing is also performed.
Here, when there is an interruption such as an incoming voice call, the control unit 12 interrupts the processing of the installed streaming playback application and switches to voice call processing. Here, it is assumed that the user talks with the other party for n seconds (S62). The control unit 12 waits for the end of the call to automatically start the streaming playback application again. At this time, the streaming playback application performs rewind playback by going back from the interruption position by m seconds according to the length of the interruption time. Perform (S63). Then, the unbuffered data is continuously reproduced by connecting to the server again using the cache of the storage unit 13 and downloading and buffering.

  As described above, during streaming, it is interrupted by incoming voice calls / SMS / mail, etc., and when restarting, it automatically changes the restart position according to the length of the interruption time (the longer the interruption time, the more The mobile terminal device with the streaming playback function can be obtained that is convenient and is easy to use.

On the other hand, when the user operates the operation unit 18 within a certain period of time after resumption and there is an instruction for fast forward and rewind, the control unit 12 holds the interruption held in the resume position table 124 described later. Data in which the relationship between the time and the restart position is maintained is corrected (S64). Here, the rewinding amount m ′ is m ′ = m−Δ in the case of fast-forwarding, and m ′ = m + Δ in the case of rewinding. Note that n and m described above correspond to n and m shown in FIGS. 3 and 4, and Δ is the adjustment time.
The relationship between the interruption time and the rewind time is initially set by default, but is adjusted by the user performing fast forward or rewind operation during a certain time at the time of restart. As a result, it is possible to provide a portable terminal device with a streaming playback function that has high flexibility and expandability.

Hereinafter, the processing procedure of each functional block constituting the control unit 12 will be described in detail with reference to the flowcharts shown in FIGS.
FIG. 7 is a flowchart showing a detailed flow of the interruption time measurement processing by the interruption time measurement processing unit 128. The operation of the interruption time measurement processing unit 128 shown in FIG. 2 will be described below with reference to the flowchart of FIG. Note that the processing of FIG. 7 is always executed during streaming playback.

  In FIG. 7, when the streaming is interrupted (S71 “Yes”), the interruption time measurement processing unit 128 controls the timer monitoring unit 127 to clear the interruption time measurement timer built in the timer monitoring unit 127 (initially). (S72). When streaming is in progress (S71 “No”), the end of streaming is awaited (S75 “Yes”), and the interruption time measurement processing by the interruption time measurement processing unit 128 is ended.

After initializing the interruption time measurement timer, the interruption time measurement processing unit 128 determines again whether the streaming is interrupted (S73). If the streaming is interrupted (S73 “Yes”), the interruption time measurement timer is determined. Is updated (updated) and the process returns to step S73 and subsequent steps (S74), and the streaming is completed (S75 “Yes”) or until the streaming is resumed (interrupt time measurement timer is updated). Do.
It is assumed that the streaming interruption / termination state is notified from the main control unit 120 to the interruption time measurement processing unit 128, and the value of the interruption time measurement timer measured here is output to the resume position calculation unit 126. Is done.

FIG. 8 is a flowchart showing a detailed flow of the scene switching time storage processing by the table data storage processing unit 129.
The operation of the table data storage processing unit 129 shown in FIG. 2 will be described below with reference to the flowchart of FIG.

  In FIG. 8, the table data storage processing unit 129 clears (initializes) the contents of the scene switching time information table 125 at the timing (S81 “Yes”) when streaming playback is started (S82). If the streaming playback start timing is not reached (S81 “No”), the scene switching time storage processing is terminated after the streaming playback is completed.

After clearing the contents of the scene switching time information table 125, the table data storage processing unit 129 determines whether streaming playback is in progress (S83). If it is determined that playback is not being performed (S83 “No”), the process returns to the determination process of step S81 described above. If it is determined that playback is being performed (S83 “Yes”), a streaming data receiving unit is further provided. It is determined whether or not the reproduction data received via 121 is an I frame (S84).
If it is determined that the frame is an I frame (S84 “Yes”), the frame time information is held in the scene switching time information table 125 as shown in FIG. (S85), the process returns to the determination process of step S83.

On the other hand, if it is determined in step S84 that the frame is not an I frame (S84 “No”), the table data storage processing unit 129 further determines whether the reproduction data is a P frame (S86). ).
When it is determined that the frame is a P frame (S86 “Yes”), the table data storage processing unit 129 compares the threshold with respect to the amount of information held in the threshold table 130 (S87), and the P frame is a threshold. (S87 “Yes”), the frame time information is held in the scene switching time information table 125 as shown in FIG. 5 as an example of the data structure (S88). repeat.

If it is determined in step S86 that the reproduction data is not a P frame (“No” in S86), the process returns to the determination process in step S83, and in the determination process in step S87, the P frame has a threshold value. Also when it is determined that it has not exceeded (S87 “No”), the process returns to the determination process of step S83.
The time information held in the scene switching time information table 125 in this way is referred to by the table search unit 131 under the control of the restart position calculation unit 126.

FIG. 9 is a flowchart showing a basic flow of the streaming restart process by the restart position calculation unit 126 and the table correction processing unit 132.
Hereinafter, basic operations of the restart position calculation unit 126 and the table correction processing unit 132 illustrated in FIG. 2 will be described with reference to the flowchart of FIG. 9.

In FIG. 9, when restarting the streaming interrupted by the occurrence of an interrupt such as an incoming voice call, the restart position calculation unit 126 executes a restart position calculation process to be described later (S91), and uses the calculated restart position information. Then, streaming is resumed (S92).
Then, the table correction processing unit 132 resumes the above-described streaming, and when a fast forward or rewind is instructed by the operation unit 18 within a predetermined time, a restart position table for correcting a restart position, which will be described later. Correction processing is executed (S93). Hereinafter, the restart position calculation process (S91) will be described in detail with reference to the flowchart of FIG. 10, and the restart position table correction process (S93) will be described in detail with reference to the flowchart of FIG.

FIG. 10 is a flowchart showing a detailed processing procedure of the restart position calculation process (S91) by the restart position calculation unit 126.
Hereinafter, the detailed operation of the restart position calculation unit 126 illustrated in FIG. 2 will be described with reference to the flowchart illustrated in FIG. 10.

In FIG. 10, when the restart position calculation unit 126 resumes the interrupted streaming due to the occurrence of the interrupt, as shown in FIG. 4, an example of the data structure is shown. A restart position (rewind time) is calculated from the interruption time information measured by the time measurement processing unit 128 (S911).
Subsequently, the resumption position calculation unit 126 obtains a reference resumption position (t1) obtained by subtracting the resumption position (rewinding time) calculated as described above from the interruption position (interruption time) (S912), and the reference resumption Based on the position (t1), the table search unit 131 is activated to search the scene switching time information table 125 (S912). Here, when the time information held in the scene switching time information table 125 is t2, the restart position calculation unit 126 searches for time information that minimizes (t2−t1), and uses the obtained search result as the restart position. (Time information) is set and output to the streaming playback control unit 123 (S913). As a result, the reproduction of the buffered streaming data is resumed, and the connection to the server is again performed using the browser cache, and the acquisition of the unbuffered streaming data is resumed.

  Specifically, an example of the data structure shown in FIG. 5 will be described as an example. If the resumption position (rewind time) calculated by the resumption position calculation unit 126 is 3 seconds, the table search unit 131 displays the scene. By searching the switching time information table 125, the latest time information of 3.5 seconds is set and output to the streaming playback control unit 123 as the restart position.

FIG. 11 is a flowchart showing a detailed processing procedure of the restart position table correction process (S93) by the table correction processing unit 132.
The detailed operation of the table correction processing unit 132 shown in FIG. 2 will be described below with reference to the flowchart shown in FIG.

  In FIG. 11, when there is a rewinding instruction from the operation unit 18 at the timing when the streaming is resumed before a predetermined time has elapsed (S <b> 931 “Yes”), the table correction processing unit 132 is held in the resume position table 124. Then, correction of the data in which the relationship between the interruption time and the restart position is held is performed (S934). Here, it is assumed that m ′ = m + Δ for a rewind instruction. On the other hand, if there is a fast-forward instruction from the operation unit 18 before the fixed time has elapsed (S932 “Yes”), the table correction processing unit 132 has a relationship between the interruption time and the restart position held in the restart position table 124. The stored data is corrected (S933). Here, since it is a fast-forward instruction, m ′ = m−Δ. Note that m described above corresponds to m described in FIGS. 3 and 4, and Δ is the adjustment time. The relationship between the interruption time and the rewind time is initially set using a default setting, but is adjusted by the user performing a fast forward or rewind operation during a certain time when the streaming is resumed.

Note that, as an operation mode of the mobile phone 10 having the streaming playback function described above, two modes of “manual” and “auto” are prepared as modes for restarting streaming.
Specifically, as shown in FIG. 12 (a), when the restart position is manually selected, during streaming playback, when there is an incoming voice call, the voice call screen is displayed, and when the call ends, the selection menu is displayed. Shall be made. Here, it is possible to select three items: “reproduce from the beginning”, “reproduce from one minute before the interruption”, and “reproduce from the interruption”. After the selection operation, the screen changes to a streaming playback screen.
If playback is desired to be traced back from the interrupted location, the time that is traced back can be input by pressing a numeric key assigned to the operation unit 18. It is also necessary to assign another key other than the numeric key to clear this retroactive time.

  On the other hand, as shown in FIG. 12B, when the restart position is selected automatically, the restart position is dynamically calculated according to the interruption time, as in the above-described embodiment. In addition, since the moving image of the streaming data uses MPEG4 format data, it is possible to determine a portion where the data difference is large as a scene switching position, and therefore the time information is stored during reproduction. Then, the time information of the scene change point is compared from the restart position corresponding to the interruption time, and the closest scene change point is set as the restart position.

  As described above, according to the mobile terminal device with the streaming playback function according to the embodiment of the present invention, since the storage of the contents becomes unclear if the interruption time becomes long, it is necessary according to the length of the interruption time. The user can be provided with convenience by automatically playing back from the point of interruption automatically, and this improves the usability. In addition, when playback is simply performed from an interrupted location, it may not be possible to grasp the contents in the middle of the scene, so multiple pieces of scene switching location information are stored, and the playback position is calculated from this information. Smooth scene reproduction is possible.

In the above-described embodiment of the present invention, only the mobile phone 10 is illustrated as a portable terminal device with a streaming playback function. Is possible.
In addition, the functions of the constituent blocks of the mobile phone described above may be realized entirely by software, or at least a part thereof may be realized by hardware. For example, the processing in the control unit 12 and the data processing in the communication unit 11 and the audio processing unit 14 may be realized on a computer by one or a plurality of programs, and at least a part thereof is realized by hardware. Also good.

It is a block diagram which shows the internal structure of the portable terminal device with a streaming reproduction function which concerns on embodiment of this invention. FIG. 2 is a block diagram showing the internal configuration of a control unit 12 shown in FIG. 1 with functions expanded, particularly focusing on streaming playback. It is a figure which shows an example of the data structure of the restart position table used in the portable terminal device with a streaming reproduction function concerning an embodiment of the invention. It is the figure quoted in order to demonstrate schematic operation | movement of the portable terminal device with a streaming reproduction function which concerns on embodiment of this invention. It is the figure quoted in order to demonstrate schematic operation | movement of the portable terminal device with a streaming reproduction function which concerns on embodiment of this invention. It is the flowchart quoted in order to demonstrate operation | movement of the portable terminal device with a streaming reproduction function which concerns on embodiment of this invention. It is the flowchart quoted in order to demonstrate operation | movement of the portable terminal device with a streaming reproduction function which concerns on embodiment of this invention. It is the flowchart quoted in order to demonstrate operation | movement of the portable terminal device with a streaming reproduction function which concerns on embodiment of this invention. It is the flowchart quoted in order to demonstrate operation | movement of the portable terminal device with a streaming reproduction function which concerns on embodiment of this invention. It is the flowchart quoted in order to demonstrate operation | movement of the portable terminal device with a streaming reproduction function which concerns on embodiment of this invention. It is the flowchart quoted in order to demonstrate operation | movement of the portable terminal device with a streaming reproduction function which concerns on embodiment of this invention. It is the flowchart quoted in order to demonstrate the operation | movement form of the portable terminal device with a streaming reproduction function which concerns on embodiment of this invention.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 ... Mobile phone, 11 ... Communication part, 12 ... Control part, 13 ... Memory | storage part, 14 ... Audio | voice processing part, 15 ... Speaker (SP), 16 ... Microphone (MIC), 17 ... Display part, 18 ... Operation part, DESCRIPTION OF SYMBOLS 120 ... Main control part 121 ... Streaming data receiving part 122 ... Streaming buffer 123 ... Streaming reproduction control part 124 ... Resumption position table 125 ... Scene switching time information table 126 ... Resumption position calculation part 127 ... Timer monitoring 128: Interruption time measurement processing unit, 129 ... Table data storage processing unit, 130 ... Threshold table, 131 ... Table search unit, 132 ... Table correction processing unit

Claims (6)

A storage unit for storing information;
A receiving unit for receiving streaming data including a plurality of frames from a communication network;
A controller for playing back the streaming data,
The controller is
When reproducing the streaming data, the reproduction time is stored in the storage unit according to the reproduction of the streaming data and the data amount for each frame is monitored.
If there is at least a frame whose data amount is a predetermined value or more, information for specifying this is stored in the storage unit,
When playback is interrupted due to an interrupt process that occurs during playback of streaming data, and the playback of the interrupted streaming data is resumed, it is specified based on information stored in the storage unit, going back from the playback time at the time of the interruption. A mobile terminal device with a streaming playback function, wherein a playback resume position is a frame in which the amount of data to be played is a predetermined value or more. A storage unit for storing information;
Streaming data including a reference frame in which an image serving as a reference is encoded, and a difference frame forming a moving image by encoding a difference from the reference frame and reproducing the reference frame, is transmitted over a communication network. A receiving unit for receiving from,
A controller for playing back the streaming data,
The controller is
When reproducing the streaming data, the reproduction time is stored in the storage unit according to the reproduction of the streaming data, and it is determined whether the frame to be reproduced is a reference frame or a difference frame,
If there is a frame to be a reference frame, information specifying this is stored in the storage unit,
When playback is interrupted due to an interrupt process that occurs during playback of streaming data, and the playback of the interrupted streaming data is resumed, it is specified based on information stored in the storage unit, going back from the playback time at the time of the interruption. A mobile terminal device with a streaming playback function, characterized in that a reference frame to be played is set as a playback resume position. A storage unit for storing information;
Streaming data including a reference frame in which a reference image is encoded and a difference frame forming a moving image by encoding a difference from the reference frame and reproducing the reference frame is transmitted from a communication network. A receiving unit for receiving;
A controller for playing back the streaming data,
The controller is
When reproducing the streaming data, the reproduction time corresponding to the reproduction of the streaming data is stored in the storage unit, and it is determined whether the frame to be reproduced is a reference frame or a difference frame,
If there is a frame to be a reference frame, information specifying this is stored in the storage unit,
If it is a difference frame, if it is a frame whose data amount is a predetermined value or more, information for specifying this is stored in the storage unit,
When the interrupting process occurs during the playback of the streaming data and the playback is interrupted, and the playback of the interrupted streaming data is resumed, it is specified based on the information stored in the storage unit, going back from the playback time at the time of the interruption. A mobile terminal device with a streaming playback function, characterized in that a reference frame to be played or a position of a difference frame at which the data amount is a predetermined value or more is set as a playback restart position. The said control part specifies the amount of time which goes back from the reproduction time at the time of interruption at the time of resuming reproduction | regeneration based on the time when reproduction | regeneration of streaming was interrupted when resuming the said streaming reproduction. 4. A mobile terminal device with a streaming playback function according to any one of items 1 to 3. An operation unit for instructing fast-forward and / or rewind;
The controller is
When resuming the interrupted streaming playback, if fast forward or rewind is instructed by the operation unit within a predetermined time after starting the restart processing, the amount of time going back from the playback time at the time of interruption is corrected. The mobile terminal device with a streaming reproduction function according to any one of claims 1 to 4, wherein the streaming reproduction is resumed. The controller is
Communication information at the time of acquiring streaming data is stored in the storage unit as cache information, and when the reproduction of the interrupted streaming data is resumed, the communication unit is connected to the streaming data supply source based on the cache information. The mobile terminal device with streaming playback function according to any one of claims 1 to 5, wherein the mobile terminal device is controlled to reconnect and resume acquisition of streaming data.

Images