JP4350627B2 - Streaming data reception / playback terminal - Google Patents

Description

  The present invention relates to a streaming data receiving / reproducing terminal that receives streaming data transmitted from a server via a network and reproduces it.

  There is known a streaming reproduction method in which content data transmitted from a server via a network is received by a streaming data receiving / reproducing terminal, and sequentially reproduced from the received data portion by the streaming data receiving / reproducing terminal. Such streaming playback methods are described in, for example, Patent Document 1 and Patent Document 2. Among the contents that are streamed and reproduced, there are contents composed of a combination of image information and sound information, contents only of image information, contents only of sound information, and the like. In addition, there are contents including other information such as character information and vibration information for vibrating a vibrator of a mobile phone as components. The streaming server individually establishes a connection for streaming communication (streaming session) for each type of content configuration information constituting the content, and distributes the various types of content configuration information through separate streaming sessions. In addition, as in the method described in Patent Document 1, the same type of content configuration information may be streamed through separate streaming sessions. Accordingly, when the content to be played is distributed through one session, only one streaming session is established between the streaming server and the streaming data receiving / playback terminal. Examples of such content include content that is composed of single content configuration information such as content of only image information and content of only sound information, and is distributed through one session. On the other hand, when the content to be played is distributed through two or more sessions, two or more streaming sessions between the streaming server and the streaming data reception / playback terminal are established. In such a case, the content is composed of a plurality of pieces of content configuration information such as content composed of a combination of image information and sound information, or the same type of content configuration as in the method described in Patent Document 1 above. There are cases where information is streamed through separate streaming sessions.

  FIG. 13 is a sequence flow diagram showing a conventional communication procedure when streaming playback of content on a mobile phone which is a streaming data reception / playback terminal. When content is to be reproduced, first, an acquisition request for SDP (Session Description Protocol) data (connection information data) is transmitted from the mobile phone to the streaming server, and SDP data is received by the mobile phone as a response. After storing the SDP data in the internal memory, the mobile phone refers to various setting items included in the SDP data, and transmits a streaming session establishment request (session establishment request) to the server according to the settings. Among the items included in the SDP data, there is an item for describing a streaming ID necessary for downloading the content configuration information. The streaming ID is information for specifying a communication port used by the streaming server to distribute the content configuration information. In accordance with the streaming ID in the SDP data, the mobile phone transmits a streaming session establishment request to the server for each ID. In the example shown in FIG. 13, since the streaming ID for image information and the streaming ID for sound information are described in the SDP data, two streaming sessions are established between the streaming server and the mobile phone. Will be. Then, the server starts transmitting streaming data through the established session in response to a content reproduction request from the mobile phone. Thereby, the mobile phone reproduces the streaming data of the image information and displays the image on the display means such as a liquid crystal display, or reproduces the streaming data of the sound information and outputs the sound from the sound output means such as the external speaker. Can be.

JP 2003-333579 A JP 2003-229843 A

However, conventionally, there has been no method for the user to easily know the type of streaming session to be established in order to appropriately reproduce the content to be reproduced. Therefore, various problems have arisen due to this.
For example, a part of the streaming session to be established is not established because the specification of the streaming data reception / playback terminal does not support a part of the streaming data distributed from the server through an individual session. Can happen. For example, when the content to be played is composed of image information and sound information, the streaming data reception / playback terminal is not compatible with the codec of the sound information, etc. There may be a situation where only the session is established and the streaming session for sound information cannot be established. In this case, when the server is configured to be able to transmit streaming data using only the probable session for image information, the streaming data reception / playback terminal displays an image based on the image information on the display means. However, the sound based on the sound information is not output from the sound output means. The user can know in advance that the content is composed of image information and sound information by receiving an explanation of the content. And if this is known, the user cannot establish a streaming session for sound information when an image is displayed on the display means but no sound is output. It is possible to guess whether this is the cause. As a result, it is possible to perform an appropriate operation of performing the reception / playback process again and reestablishing the streaming session.
However, the user often does not know that the content to be played is composed of image information and sound information. Therefore, when a situation occurs in which the image is displayed on the display means but no sound is output as described above, the user determines whether the content is only image information content or the content is image information. However, it is not possible to properly determine whether or not a streaming session for sound information has only been established. As a result, if the user mistakenly determines that the content to be reproduced is, for example, content of only image information, the user finishes the reproduction without listening to the sound without being aware of this. There was a problem.

Conventionally, there has been no way for a user to easily know even if a situation occurs in which a part of a streaming session to be established is not established as described above. Therefore, various problems have arisen due to this.
For example, a mobile phone as a conventional streaming data reception / playback terminal is provided with an operation mode called a manner mode in which operation is performed so as not to output a ringtone of a telephone call from an external speaker. Among such mobile phones, if the manner mode has already been set before the content reception / playback process is started, the sound of the content is not output from an external speaker. In this mobile phone, when the content reception / reproduction processing is started and the content is reproduced while the manner mode is set, even if the content configuration information of the content includes sound information, the sound is output from the external speaker. Not output. In this case, even if the user knows in advance that the content is composed of image information and sound information and is distributed through separate streaming sessions by receiving explanations about the content, etc. It was not possible to properly determine whether the content sound was not output because the manner mode was being set or whether the content sound was not output because the streaming session for sound information could not be established. Therefore, there is a problem that a user who erroneously determines that the cause is the latter is forced to perform an unnecessary operation of repeatedly performing an operation for reestablishing a streaming session.
In addition, not only when the reception / reproduction processing is started during the manner mode setting, but also in other cases such as when the output volume from the external speaker is set to a low level, the user similarly receives the content sound from the external speaker. The reason why the message is not output cannot be properly determined, and unnecessary work is performed.

  The above-described problem is not limited to the case where sound information is included in the content configuration information of the content to be played back and the sound is not output from an external speaker. For example, even if image information or text information is included as content configuration information and the image or text is not output to the display means, or vibration information is included as content configuration information Even when the vibrator does not vibrate based on the vibration information, it can occur similarly.

The present invention has been made in view of the above problems, and a first object is to receive streaming data that allows a user to easily know the type of connection to be established in order to appropriately reproduce the content to be reproduced. It is to provide a playback terminal.
A second object is to provide a streaming data receiving / reproducing terminal that allows a user to easily know that a part of the connection to be established is not established.

In order to achieve the first object, the invention of claim 1 is directed to display means, data communication means for performing data communication with a streaming server via a network, and receiving streaming data from the streaming server. When the connection information data for recognizing the establishment all-out two or more connections is received by the communication means, based on the connection information data, a connection establishment request for establishing the該確elevational all-out two or more connecting said Control means for transmitting to the streaming server by the data communication means and receiving / reproducing processing for receiving the streaming data transmitted from the streaming server through the connection by the data communication means and reproducing it. In the streaming data reception / playback terminal, the control means receives the received data. Based on the connection information data, with all of the established all-out two or more connections, is characterized in carrying out the process of displaying on the display means the necessary connection notification image corresponding individually.
In the streaming data receiving and reproducing terminal, is displayed on the required connection notification image display means corresponding individually to establish all-out two or more connections to receive streaming data. Therefore, the user can know the type of connection to be established in order to appropriately play back the content to be played back, simply by looking at the necessary connection notification image.

Further, according to the invention of claim 2, in the streaming data reception / playback terminal of claim 1, the control means determines, for each connection, whether or not one or more connections to be established have been established. When it is determined that the connection is established, instead of or together with the necessary connection notification image corresponding to the connection, another connection establishment notification image for notifying the user that the connection has been established is displayed. It is characterized by performing processing to be displayed on the means.
In this streaming data reception / playback terminal, when it is determined that a connection has been established, instead of or together with the necessary connection notification image corresponding to the connection, a connection establishment notification different from the necessary connection notification image is provided. An image is displayed on the display means. Therefore, the user sees the necessary connection notification image and the connection establishment notification image, and whether or not the connection to be established is actually established in addition to the connection to be established in order to appropriately reproduce the content to be reproduced. Can know.

In order to achieve the second object, the invention of claim 3 is directed to display means, data communication means for performing data communication with a streaming server via a network, and for receiving streaming data from the streaming server. When the connection information data for recognizing the establishment all-out two or more connections is received by the communication means, based on the connection information data, a connection establishment request for establishing the該確elevational all-out two or more connecting said Control means for transmitting to the streaming server by the data communication means and receiving / reproducing processing for receiving the streaming data transmitted from the streaming server through the connection by the data communication means and reproducing it. In the streaming data reception / playback terminal, the control means establishes the establishment. For all of the feeder 2 or more connections, the connection is determined whether established for each said connection when the connection is determined that has been established, the connection establishment notification to notify the subject that the connection has been established to the user A process of displaying an image on the display unit individually for each connection is performed.
In this streaming data reception / playback terminal, when it is determined that the connection has been established, a connection establishment notification image for notifying the user that the connection has been established is displayed on the display means. Therefore, if the user knows in advance the connection to be established in order to receive the streaming data, the user can determine the connection to be established in order to appropriately reproduce the content to be reproduced by viewing this connection establishment notification image. Even if a situation occurs in which some of the information is not established, the user can know this.

A fourth aspect of the present invention is the streaming data receiving / reproducing terminal of the first, second or third aspect, further comprising information notifying means for notifying a user of information and an operating means, wherein the control means receives the information. When it is determined that the number of connections to be established is two or more based on the connection information data, it is determined for each connection whether or not two or more connections to be established have been established within a predetermined time, and the establishment When it is determined that only a part of the two or more connections to be established has been established, the information notification is provided for prompting the user to select the operation means for determining whether to reproduce the streaming data. And when the selection operation for reproduction is performed on the operation means, the reception reproduction process is performed only through the established connection, and the selection operation for not reproduction is performed on the operation means. It is characterized in carrying out the process for cutting the established connection when it is.
When two or more connections are necessary to properly reproduce the content, when only a part of the connection is established, insufficient content is provided to the user. In order to prevent such insufficient content from being provided, it is conceivable to adopt a method of prohibiting the reproduction of content until all connections are established. However, it is possible to assume a user who desires insufficient content. For example, even if some of the connections that have not been established are content configuration information, if the content is live-type content, the state is inadequate when the content portion cannot be played back while reconnecting However, it is thought that there are many users who want to play content as it is. Also, if the content is very popular and it is difficult to establish a connection and there is a risk that it will not be possible to establish a connection once all of the connections have been made again, it will remain inadequate. Many users would like to play content.
Therefore, in the streaming data reception / playback terminal, an operation for selecting whether or not to play back the content to be played back when only a part of two or more connections to be established is established within a predetermined time. Information for prompting the user to be notified to the user. The user who has received this notification can select whether or not to perform reception / reproduction processing only through the established connection. When a selection operation for reproduction is performed on the operation means, the reception / reproduction process is performed only through the established connection, and the content is reproduced. On the other hand, when the selection operation not to reproduce is performed on the operation means, the established connection is disconnected, and the user can re-establish the connection.

The “streaming data reception / playback terminal” includes a mobile communication terminal, a portable information terminal (PDA: Personal Digital Assistance), a non-portable information terminal such as a desktop personal computer, and the like. The “mobile communication terminal” includes a mobile phone such as a PDC (Personal Digital Cellular) method, a GSM (Global System for Mobile Communication) method, a TIA (Telecommunications Industry Association) method, and an IMT (International Mobile Telecommunications) -2000. in standardized cellular phone, TD-SCDMA (Time Division S ynchronous Code Division Multiple Access) TD-SCDMA is one of the schemes (MC: Multi Carrier) scheme mobile phone, such as PHS (Personal Handyphone Service) and the like . The “mobile communication terminal” can receive and reproduce streaming data transmitted from a server via a network in addition to a telephone such as a PHS (Personal Handyphone System) and a car phone. Any mobile communication terminal is also included.
The control in the streaming data reception / playback terminal can also be realized by executing a predetermined program on a computer provided therein. Delivery of the program used in the computer may be performed using a recording medium such as an FD or a CD-ROM in which the program is recorded as digital information, or may be performed using a communication network such as a computer network.

As described above, according to the first and second aspects of the invention, there is an excellent effect that the user can easily know the type of connection to be established in order to appropriately reproduce the content to be reproduced. As a result, it is possible to solve various problems that have arisen in the past because there is no method for the user to easily know the type of connection to be established in order to appropriately reproduce the content to be reproduced.
In particular, according to the invention of claim 2, there is an excellent effect that it is possible to easily know whether or not a connection to be established in order to appropriately reproduce the content to be reproduced has been actually established.
According to the invention of claim 3, the user can easily know that a part of the connection to be established is not established on the condition that the user knows in advance the connection to be established. An effect is produced. As a result, it is possible to solve various problems that have conventionally occurred because there is no way for the user to easily know that a part of the connection to be established has not been established.
According to the invention of claim 4, there is an excellent effect that the content can be reproduced as it is by the user's selection even in an insufficient state where only a part of the connection is established.

Hereinafter, an embodiment in which the present invention is applied to a mobile phone which is a mobile communication terminal as a streaming data reception / playback terminal will be described with reference to the drawings.
FIG. 2 is an explanatory diagram for explaining the overall configuration of a mobile communication system in which the mobile phone according to the present embodiment can be used.
In this mobile communication system, a mobile phone 20 used by a user has a configuration capable of executing an application program registered by the user. In the present embodiment, the application program is developed by object-oriented programming or the like that does not depend on the platform. As such an application program, an application program written in JAVA (registered trademark, the same applies hereinafter), an application program written in the C ++ language, and an application execution environment of BREW (registered trademark, the same applies hereinafter) operate. Application programs are examples. The application program may be a structured program written in C language, assembly language, or the like. The cellular phone 20 can be connected to a cellular phone communication network 10 as a network. The mobile phone communication network 10 is connected to a streaming server 11 that distributes content data. The streaming server 11 transmits streaming data based on the content data to the mobile phone 20 in response to a request from the mobile phone 20.

  Content data distributed by the streaming server 11 is provided from a content developer such as a content developer. Specifically, when the content is storage-type content, for example, the content is uploaded from a personal computer on the content developer side to the streaming server 11 via a dedicated line or a public line and provided. Note that a recording medium such as an optical disk or a magnetic disk on which the storage-type content data is recorded is sent from the content developer to a communication carrier that manages and operates the streaming server 11, and the content data in the recording medium is sent to the streaming server 11. You may read and provide. The storage-type content data provided in this way is registered in the streaming server 11 in a state where streaming distribution is possible from the mobile phone 20 via the mobile phone communication network 10. On the other hand, for live content, data such as real-time images taken by the content developer is transmitted to the streaming server 11 via a dedicated line or a public line. The live content data provided in this way is received by the streaming server 11 and simultaneously delivered to the mobile phone 20 via the mobile phone communication network 10. Note that “content data” includes image content data such as a movie combining sound and image, content data consisting only of sound such as music, content data consisting only of an image, and the like.

FIG. 3 is a schematic configuration diagram showing a hardware configuration of the streaming server 11.
The streaming server 11 includes a system bus 100, a CPU 101, an internal storage device, an external storage device 104, an input device 105, and an output device 106. The internal storage device includes a RAM 102, a ROM 103, and the like. The external storage device 104 includes a hard disk drive (HDD), an optical disk drive, and the like. The input device 105 includes a mouse, a keyboard, and the like. The output device 106 includes a display, a printer, and the like. The streaming server 11 further includes a mobile phone communication device 107 as a communication means for communicating with each user's mobile phone 20 via the mobile phone communication network 10.

  The components such as the CPU 101 and the RAM 102 exchange data and program instructions with each other via the system bus 100. Various programs for operating the streaming server 11 according to a predetermined procedure are stored in the ROM 103 or the external storage device 104, and are called up and executed in the work area on the CPU 101 or the RAM 102 as necessary. The streaming server 11 stores content data to be transmitted to the mobile phone 20 in the external storage device 104. In response to a request from the mobile phone 20, the streaming server 11 cooperates with the CPU 101, the RAM 102, the mobile phone communication network communication device 107, and the like to transfer the content data stored in the external storage device 104 to the mobile phone communication network. 10 has a function of transmitting streaming data to the mobile phone 20 through the network 10. Further, it also has a function as a Web server that provides a Web site for introducing content data stored in the external storage device 104. In the present embodiment, when the user of the mobile phone 20 intends to stream the content data, the user first accesses the website and selects the content data to be played back, and then the streaming data based on the selected content data. It will be received and played back. The streaming server 11 may be configured as a dedicated device or may be configured using a general-purpose computer system. Further, it may be configured by a single computer, or may be configured by connecting a plurality of computers each having a plurality of functions via a network.

4 is a front view showing the appearance of the mobile phone 20, and FIG. 5 is a schematic configuration diagram showing the hardware configuration of the mobile phone 20. As shown in FIG.
The mobile phone 20 is a clamshell (folding) type mobile phone, and includes an internal control device including a system bus 200, a CPU 201, a RAM 202, a ROM 203, an input device 204, an output device 205, and a mobile phone communication device 206. ing. Components such as the CPU 201 and the RAM 202 exchange various data and instructions of application programs described later via the system bus 200. The input device 204 includes a data input key (ten key, * key, # key) 21, a call start key 22, an end key 23, a scroll key 24, a multi-function key 25, a microphone 26, and the like. The output device 205 includes a liquid crystal display (LCD) 27 as a display means, a speaker for call 28, an external speaker (not shown), and the like. The mobile phone communication device 206 is for communicating with other mobile phones and the streaming server 11 via the mobile phone communication network 10. Further, in the RAM 202, there are a platform storage area as a first storage means managed by a telephone platform described later and an application storage area as a second storage means managed in an application execution environment described later. Exists. In the present embodiment, the RAM 202 and the ROM 203 function as data storage means.

FIG. 6 is a block diagram showing an extracted main part of the mobile phone 20, and FIG. 7 is an explanatory diagram of the software structure in the mobile phone 20.
The cellular phone 20 includes a telephone communication unit 211, a data communication unit 212 as a data communication unit, an operation unit 213 as an operation unit, an application program execution management unit 214 as an application program execution unit, a main control unit 215, and an output unit 216. Etc.

The telephone communication unit 211 performs wireless communication with a base station of the mobile phone communication network 10 in order to perform telephone communication with other mobile phones or fixed phones. This corresponds to the device 206 and the like.
Similar to the telephone communication unit 211, the data communication unit 212 corresponds to the mobile phone communication device 206 having the above-described hardware configuration. The data communication unit 212 exchanges mail with other mobile phones via the mobile phone communication network 10 or connects to an external network such as the Internet from the mobile phone communication network 10 via a gateway server. This is for exchanging e-mails and browsing web pages. The data communication unit 212 is also used to download an application program provided by the streaming server 11 via the mobile phone communication network 10.

  The operation unit 213 includes the above-described numeric keypad 21, call start key 22, end call key 23, and the like operated by the user. By operating the various keys of the operation unit 213, the user inputs data such as a URL to the mobile phone 20, starts and ends a call when receiving a call, selects an application program, It can be started and stopped. In addition, the user can access the Web site provided by the streaming server 11 by operating various keys of the operation unit 213, receive streaming data based on the content data, and reproduce it.

  The application program execution management unit 214 includes the system bus 200, the CPU 201, a part of the RAM 202, and the like. The application program execution management unit 214 corresponds to the central “application execution environment” in the software structure of FIG. 7, and includes a class library, an execution environment management library, and an application program developed by object-oriented programming. Software such as application management is provided to manage the execution environment of application programs. This application execution environment is appropriately selected according to the application program to be executed. For example, if the application program to be executed is written in JAVA, the JAVA application execution environment is selected. If the application program to be executed is written in C language that operates in the BREW execution environment, the BREW application execution environment is selected. If the application program to be executed is written in JAVA, it can be executed by constructing a JAVA application execution environment on the BREW application execution environment.

  Here, the application program can be used by calling a class library such as a function in the application execution environment via a class library API (application interface). The call history of the class library such as this function is stored in the application storage area in the RAM 202 until the virtual execution environment (virtual machine: VM) of the application program is terminated. The application execution environment also saves various data used when executing the application program in the application storage area. When these various data are used, they are read from or written to the application storage area. The execution environment management library in the application execution environment can be used by calling a telephone platform library in the telephone platform described later via the telephone platform API.

  The main control unit 215 controls the telephone communication unit 211, the data communication unit 212, and the output unit 216, and includes the system bus 200, the CPU 201, the RAM 202, and the like. The main control unit 215 exchanges control commands and various data with the application program execution management unit 214, and functions as a control unit by performing control in cooperation with the control command and various data. The main control unit 215 corresponds to the lowermost “telephone platform” in the software structure of FIG. 7 and executes a control program and user interface for controlling the telephone communication unit 211 and the like, Or provide a library. The telephone platform executes various processes in an application program by sending an event to an execution environment management library in the application execution environment, or performs application management software in the application execution environment via an application management API. Can be called and used. When the application execution environment calls and uses the telephone platform library via the telephone platform API, the telephone platform executes processing according to the telephone platform library. For example, the telephone platform reads out data stored in the platform storage area managed by the telephone platform in the RAM 202 based on an instruction from the application execution environment using the telephone platform library, and migrates it to the application storage area. can do.

  The output unit 216 includes the output device 205 including the liquid crystal display 27, the call speaker 28, and an external speaker. The output unit 216 displays the Web page screen received by the data communication unit 212 on the liquid crystal display 27. The liquid crystal display 27 of the output unit 216 is used when the telephone communication unit 211 or the data communication unit 212 notifies the user that information has been received. Specifically, when the information is received, the main control unit 215 displays an incoming call notification image on the liquid crystal display 27 of the output unit 216 or outputs a ring tone from an external speaker. Further, the output unit 216 is also used for outputting images and sounds of reproduced content data during execution of a content reproduction application program (described later) executed in the application execution environment.

  A control program for constructing a telephone platform for operating the mobile telephone 20 according to a predetermined procedure is stored in the RAM 202 and the ROM 203. Further, a basic OS (operating system) program, a program for building the application execution environment, and an application program are also stored in the RAM 202 and the ROM 203. These programs are called up and executed in the work area in the CPU 201 or RAM 202 as necessary.

Next, content data streaming playback processing will be described.
FIG. 8 is a flowchart showing the flow of processing operations of the cellular phone 20 when streaming the content on the cellular phone 20. FIG. 9 is a sequence flow diagram illustrating a communication procedure when all the sessions to be established when the mobile phone 20 performs streaming reproduction of content can be established.
When trying to reproduce the content data, first, the user operates the key of the operation unit 213 to activate the browser (S1). Then, by operating the key of the operation unit 213, the URL of the content selection site of the streaming server 11 is input and accessed (S2). Accordingly, the main control unit 215 that executes the browser transmits a request to the streaming server 11 specified by the URL. Upon receiving this request, the streaming server 11 transmits the content selection site screen (content list screen) related to the request as a response to the mobile phone 20 that is the transmission source of the request. Thereafter, the user operates the keys of the operation unit 213 to select the desired content from the content list on the screen displayed on the LCD 27 (S3). In response, an instruction to execute the application program is input to the telephone platform shown in FIG. 7, that is, the main control unit 215 shown in FIG. As a result, the main control unit 215 reads the content reproduction application program and activates it (S4). When the application program is started, the application program operates on the application execution environment shown in FIG. 7, that is, the application program execution management unit 214 shown in FIG. The application program may be stored in advance in the ROM 203 of the mobile phone 20 or downloaded from the outside and stored in the RAM 202.

  When the application program is activated, the application program execution management unit 214 mainly sends a request for acquiring SDP data (connection information data) to the content distribution site that distributes the content data to the streaming server 11 for the content selected in S3. It transmits via the control unit 215. In the present embodiment, the mobile phone 20 and the streaming server 11 perform data communication mainly using RTSP (Real Time Streaming Protocol) and RTP (Real-time Transport Protocol). Therefore, here, a DESCRIBE command including a distribution destination URL (Uniform Resource Locator) of streaming data is transmitted as an SDP data acquisition request. Upon receiving the SDP data request, the streaming server 11 transmits SDP data to the mobile phone 20 as a response. The application program execution management unit 214 of the mobile phone 20 that has received the SDP data stores the SDP data in the RAM 202 (S5).

  The SDP data includes various terminal setting information indicating setting items and setting contents to be set on the mobile phone side in order to play the streaming data of the corresponding content on the mobile phone 20, the name (title) of the content, Content information such as the total playback time of the content is described by item. In this SDP data, as an item for terminal setting, an item for describing a streaming ID, which is connection information for recognizing a streaming session to be established in order to receive streaming data of the content, is provided for each session. Is provided. For example, when the content is composed of video information (image information) and audio information (sound information), the SDP data includes a streaming ID of the video streaming session and a streaming ID of the audio streaming session. Are described in terms of terms. The streaming ID is information for specifying a communication port of the streaming server 11 used for transmitting corresponding information (video information, audio information, etc.). The application program execution management unit 214 storing the SDP data in S5 performs a session establishment notification process based on the streaming ID described in the SDP data (S20).

  In the session establishment notification process, the application program execution management unit 214 sends a session establishment request (SETUP command) including each streaming ID to the streaming server via the main control unit 215 for all streaming IDs described in the SDP data. 11 to send. As a result, the streaming server 11 establishes a streaming session related to the session establishment request with the mobile phone 20 and transmits a response to the effect that the streaming session is established to the mobile phone 20. In this embodiment, the content selection site and the content distribution site are provided by the same streaming server 11, but may be provided by different servers. Other details of the session establishment notification process will be described later.

  On the other hand, when the content reproduction application program is started, a content reproduction screen as shown in FIG. 10 is displayed on the LCD 27. When the user performs a content reproduction operation on the key of the operation unit 213 (S6), the application program execution management unit 214 sends a content reproduction request (PLAY command) to the streaming server 11 via the main control unit 215. Send. In response to this request, the streaming server 11 transmits a response indicating that the request has been accepted to the mobile phone 20. In addition, the CPU 101 of the streaming server 11 reads the requested content data from the external storage device 104, and starts transmitting the streaming data to the mobile phone 20 through the established streaming session (S7).

  The application program execution management unit 214 of the mobile phone 20 sequentially stores the received streaming data in a predetermined buffering area in the RAM 202. When the prescribed amount of streaming data is stored in the buffering area (S8), the application program execution management unit 214 sequentially reads the streaming data from the buffering area in the RAM 202 and starts reproduction (S9). Specifically, the application program execution management unit 214 outputs a streaming data reproduction command to the output unit 216 via the main control unit 215. As a result, an image based on the streaming data distributed through the video streaming session is displayed on the LCD 25 of the output unit 216 as shown in FIG. The external output speaker 29 of the output unit 216 outputs a sound based on the streaming data distributed through the audio streaming session. Thereby, the user can view the content.

  When the user performs an operation for stopping the reproduction of the content with respect to the key of the operation unit 213 during the content reproduction (S10), the application program execution management unit 214 outputs the output unit via the main control unit 215. A command to stop the output of content is output to 216. Further, the application program execution management unit 214 transmits a playback stop request (TEARDOWN command) (not shown) to the streaming server 11 via the main control unit 215. In response to this request, the streaming server 11 disconnects the streaming session with the mobile phone 20 and transmits a response to the mobile phone 20 indicating that the streaming session has been disconnected. The same applies to the case where the content is reproduced to the end and the reproduction is finished (S11).

[Operation example 1]
Next, an example of session establishment notification processing (hereinafter, this example is referred to as “operation example 1”), which is a characteristic part of the present invention, will be described.
FIG. 1 is a flowchart showing a flow of session establishment notification processing according to the first operation example. The application program execution management unit 214 storing the SDP data in S5 first refers to the SDP data (S21), and reads the streaming ID described therein. Then, a display command for displaying the pictograms in gray on the liquid crystal display 27 for all of the read streaming IDs is output to the output unit 216 (S22). Here, it is assumed that the streaming ID of the video streaming session and the streaming ID of the audio streaming session are described in the SDP data. In this case, as shown in FIG. 10A, pictograms (necessary connection notification images) B ′ and C ′ respectively corresponding to these streaming IDs are individually displayed in the pictogram display area A. At this stage, since a session establishment request has not yet been transmitted to the streaming server 11, a streaming session corresponding to each streaming ID has not been established. Accordingly, as shown in FIG. 10A, the pictograms B ′ and C ′ are both displayed in gray.

  After that, as shown in FIG. 9, the application program execution management unit 214 first sends a session establishment request (SETUP command) for establishing an audio streaming session among the streaming IDs described in the SDP data to the main control. The data is transmitted to the streaming server 11 via the unit 215 (S23). When the session establishment request is received by the streaming server 11 and the streaming server 11 establishes an audio streaming session with the mobile phone 20, the streaming server 11 transmits a response to that effect to the mobile phone 20. . Receiving this, the cellular phone 20 determines that the audio streaming session related to the transmitted session establishment request has been established (S24), and outputs a display command for replacing the gray audio pictograph B 'with color display. The data is output to the unit 216 (S25). Accordingly, as shown in FIG. 10B, instead of the audio pictogram B 'displayed in gray, an audio pictogram (connection establishment notification image) B in color display is displayed in the pictogram display area A.

  At this stage, since the session establishment request has not been transmitted for all the streaming IDs described in the SDP data (S26), the application program execution management unit 214 establishes a video streaming session that has not yet been transmitted. A session establishment request (SETUP command) is transmitted to the streaming server 11 via the main control unit 215 (S23). When the session establishment request is received by the streaming server 11 and the streaming server 11 establishes a video streaming session with the mobile phone 20, the streaming server 11 transmits a response to that effect to the mobile phone 20. . Receiving this, the mobile phone 20 determines that the video streaming session related to the transmitted session establishment request has been established (S24), and outputs a display command for replacing the gray video pict C 'with color display. The data is output to the unit 216 (S25). Accordingly, as shown in FIG. 10C, instead of the video pictogram C ′ displayed in gray, a video pictogram (connection establishment notification image) C in color display is displayed in the pictogram display area A.

  When all the streaming sessions are established in this way (S27), the session establishment notification process ends. Thereafter, when the user performs a content reproduction operation on the keys of the operation unit 213 (S6), the video information and audio information of the content are downloaded through the corresponding streaming sessions (S7). When a prescribed amount of streaming data is stored in the buffering area of the RAM 202 (S8), as shown in FIG. 10D, display of an image based on the streaming data of the video information is started, and the audio information Sound output based on the streaming data is started (S9).

  Here, there may occur a situation in which the streaming session to be established is not established because the specification of the mobile phone 20 does not correspond to a part of the streaming data distributed from the streaming server 11 through an individual session. When the response (RTSP: reply) to the session establishment request of the audio streaming session is not received within the specified time, the application program execution management unit 214 determines that the session has not been established (S24). In this case, the application program execution management unit 214 transmits a session establishment request for the next video streaming session as it is (S23). At this time, if the response (RTSP: reply) to the session establishment request of the video streaming session is not received within the specified time, since no session is established (S27), the content cannot be reproduced. . Therefore, in this case, the application program execution management unit 214 displays a warning message such as “The session could not be established. Select content again” on the liquid crystal display 27 via the output unit 216. Is displayed (S28), and the session establishment notification process is terminated.

When the audio streaming session is established but the video streaming session is not established (S27), the application program execution management unit 214 displays the information on the liquid crystal display 27 via the output unit 216. A warning message such as “The video session could not be established. Do you want to play it as it is?” Is displayed (S28). When the user who sees the warning message performs a content reproduction operation on the key of the operation unit 213 described above (S6), the application program execution management unit 214 makes a content reproduction request (PLAY command) to the streaming server 11. Is transmitted via the main control unit 215. The CPU 101 of the streaming server 11 of the present embodiment performs a process of transmitting streaming data for the session only through the probable streaming session, even in a state where a part of the probable streaming session is not probable. Therefore, when the content reproduction request is received, the CPU 101 of the streaming server 11 reads the content data related to the request from the external storage device 104, and sends the streaming data of the audio information to the mobile phone 20 only through the established audio streaming session. Transmission is started (S7). As a result, the mobile phone 20 starts outputting sound based on the streaming data of the audio information without displaying an image based on the streaming data of the video information on the liquid crystal display 27 (S9). On the other hand, when the user who sees the warning message performs an operation of canceling the reproduction of the content with respect to the key of the operation unit 213, the application program execution management unit 214 requests the streaming server 11 to stop the reproduction. (TEARDOWN command) is transmitted via the main control unit 215, and then the main control unit 215 terminates the application program for content reproduction.
The same applies when the video streaming session is established but the audio streaming session is not established.

As described above, according to the first operation example, the pictograms B ′ and C ′ for informing the user of the type of the streaming session to be established for receiving the streaming data are displayed in gray on the liquid crystal display 27 for each session individually. Is done. Therefore, the user can easily know the session to be established in order to appropriately reproduce the content to be reproduced, by simply looking at the pictograms B ′ and C ′. As a result, even if the content is played back without establishing the audio streaming session and an image is displayed on the liquid crystal display 27 but no sound is output, the user can By looking at Pict B ′, it can be recognized that the content also includes audio information. Therefore, the user does not mistakenly determine that the sound is not output even when the content is reproduced because the content does not include audio information. As a result, it is possible to prevent a situation in which the content continues to be reproduced without listening to sound without noticing that an audio streaming session has not been established. The same applies to the case where the content is played back without establishing the video streaming session and the sound is output but the image is not displayed on the liquid crystal display 27.
On the other hand, simply notifying the user by pictographic display of a streaming session to be established cannot easily determine whether the cause is due to session non-establishment when no sound is produced during content playback. In the first operation example, when a streaming session is established, the pictograms B ′ and C ′ displayed in gray corresponding to the session are changed to pictograms B and C in color display. Thus, the user can individually know whether the streaming session of video information and audio information has actually been established by watching the pictogram change from gray display to color display. Therefore, when the audio streaming session is not actually established, the user can easily recognize this. As a result, if the reason why no sound is not output when the content is played back is that the session is not established, the user incorrectly determines that the audio streaming session has not been established, and reestablishes the streaming session. For this reason, it is possible to prevent a situation where an unnecessary operation of repeatedly performing the operation is performed. The same applies when the reason why the image is not displayed on the liquid crystal display 27 even if the content is reproduced is not the establishment of the session.
In the first operation example, when only a part of the streaming sessions can be established, a warning message for prompting the user to select whether or not to reproduce the content is displayed on the liquid crystal display 27. Thereby, the user who has received the warning message can select whether or not to reproduce the content only through the established session. Therefore, even in an inadequate state where only a part of the sessions are established, the user can decide whether or not to play the content at his / her own will.

[Operation example 2]
Next, another example of session establishment notification processing (hereinafter, this example is referred to as “operation example 2”), which is a characteristic part of the present invention, will be described.
FIG. 11 is a flowchart showing a flow of session establishment notification processing according to the second operation example. The application program execution management unit 214 storing the SDP data in S5 first refers to the SDP data (S31), and reads the streaming ID described therein. Note that the communication sequence of the second operation example is the same as that shown in FIG. 9 except that no pictograph is displayed at this stage. As shown in FIG. 9, the application program execution management unit 214 first sends a session establishment request (SETUP command) for establishing an audio streaming session among the streaming IDs described in the SDP data to the main control unit 215. To the streaming server 11 (S32). When the session establishment request is received by the streaming server 11 and the streaming server 11 establishes an audio streaming session with the mobile phone 20, the streaming server 11 transmits a response to that effect to the mobile phone 20. . Receiving this, the mobile phone 20 determines that the audio streaming session related to the transmitted session establishment request has been established (S33), and outputs a display command for displaying the audio pictogram B to the output unit 216 ( S34). As a result, as shown in FIG. 12B, the audio pictogram (connection establishment notification image) B is displayed in color in the pictogram display area A.

  At this stage, since the session establishment request has not been transmitted for all the streaming IDs described in the SDP data (S35), the application program execution management unit 214 establishes a video streaming session that has not yet been transmitted. A session establishment request (SETUP command) is transmitted to the streaming server 11 via the main control unit 215 (S32). When the session establishment request is received by the streaming server 11 and the streaming server 11 establishes a video streaming session with the mobile phone 20, the streaming server 11 transmits a response to that effect to the mobile phone 20. . Receiving this, the mobile phone 20 determines that the video streaming session related to the transmitted session establishment request has been established (S33), and outputs a display command for displaying the video picture C to the output unit 216 ( S34). As a result, as shown in FIG. 12C, a video pict (connection establishment notification image) C is displayed in color in the pict display area A.

When all streaming sessions are established in this way (S36), the session establishment notification process ends.
Note that when both the audio and video streaming sessions cannot be established, and when only one of them cannot be established, the same operation process as in the first operation example is performed.

As described above, according to the second operation example, when it is determined that a streaming session has been established, pictograms B and C for informing the user that the session has been established are displayed on the liquid crystal display 27. Therefore, the user can know whether or not all the sessions that should be established in order to appropriately reproduce the content have been established by viewing the pictograms B and C. As a result, even if a content is played back without establishing an audio streaming session and an image is displayed on the liquid crystal display 27 but no sound is output, the user is not By looking at Pict B, it is possible to prevent a situation in which it is erroneously determined that an audio streaming session has not been established, and an unnecessary operation of repeatedly performing an operation for reestablishing the streaming session is performed. The same applies to the case where the content is played back without establishing the video streaming session and the sound is output but the image is not displayed on the liquid crystal display 27.
In particular, if the user knows in advance that the content to be reproduced contains images and sound, the user can confirm that the audio pictograph B is not displayed, and the cause of the sound not being output is that for audio. It is possible to accurately grasp that the streaming session is not established. In this case, as in the case of the operation example 1 described above, the user does not output sound even when the content is played back when the session is not established (for example, when the mute mode is set or the manner mode setting menu is used). In some cases, it is possible to prevent a situation in which it is erroneously determined that an audio streaming session has not been established, and an unnecessary operation of repeatedly performing an operation for reestablishing the streaming session is performed. The same applies when the reason why the image is not displayed on the liquid crystal display 27 even if the content is reproduced is not the establishment of the session.

In the above-described embodiment, the case where the necessary connection notification image and the connection establishment notification image are pictically displayed has been described. However, other display methods may be used.
In the above-described embodiment, the case where the streaming session to be established is two sessions of audio information and video information has been described. However, for streaming distribution of other content configuration information such as text information (character information) The same applies to sessions.
Further, the present invention is useful not only for the mobile phone 20 described above but also for other streaming reception / playback terminals capable of streaming playback.

The flowchart which shows the flow of the session establishment alerting | reporting process which concerns on the operation example 1 of the mobile telephone in embodiment. Explanatory drawing for demonstrating the whole structure of the mobile communication system which can use the mobile phone. The schematic block diagram which shows the hardware constitutions of the streaming server which comprises the mobile communication system. The front view which shows the external appearance of the mobile phone. The schematic block diagram which shows the hardware constitutions of the mobile phone. The block diagram which extracted and showed the principal part of the mobile phone. Explanatory drawing of the software structure in the mobile phone. The flowchart which shows the flow of the processing operation at the time of carrying out streaming reproduction of the content with the mobile phone. The sequence flow figure which shows the communication procedure when all the sessions which should be established at the time of carrying out the streaming reproduction | regeneration of the content by the mobile phone have been established. (A) thru | or (d) is explanatory drawing of the screen for content reproduction | regeneration displayed on the liquid crystal display of the mobile telephone in the example 1 of operation. The flowchart which shows the flow of the session establishment alerting | reporting process which concerns on the operation example 2 of a mobile telephone. (A) thru | or (d) is explanatory drawing of the screen for content reproduction | regeneration displayed on the liquid crystal display of the mobile phone in the example 2 of operation. The sequence flow figure which shows the communication procedure at the time of carrying out streaming reproduction | regeneration of the content with the conventional mobile phone.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Mobile phone communication network 11 Streaming server 20 Mobile phone 21 Numeric keypad 27 Liquid crystal display 213 Operation part 214 Application program execution management part 215 Main control part 216 Output part A Pictograph display area B 'Pictograph for audio (gray display)
B Picto for audio (color display)
C 'Pict for video (gray display)
C Picto for video (color display)

Claims (4)

Display means;
Data communication means for performing data communication with a streaming server via a network;
When the connection information data for recognizing the establishment all-out two or more connections to receive streaming data from the streaming server and received by said data communication means, based on the connection information data, more-out該確elevational all A connection establishment request for establishing a connection is transmitted to the streaming server by the data communication means, and streaming data transmitted from the streaming server through the connection is received by the data communication means and reproduced. In a streaming data reception / playback terminal comprising a control means for performing reception / playback processing for
The control means, based on the connection information data received, with all of the established all-out two or more connections, and characterized by performing a process of displaying on the display means the necessary connection notification image corresponding individually Streaming data reception / playback terminal. The streaming data reception / playback terminal according to claim 1,
Said control means, whether or not established the established all-out two or more connections is determined for each said connection when the connection is determined that is established, or in place of the required coupling notification image corresponding to the connection At the same time, a streaming data receiving / reproducing terminal characterized by performing a process of displaying another connection establishment notification image for notifying the user that the connection has been established on the display means. Display means;
Data communication means for performing data communication with a streaming server via a network;
When the connection information data for recognizing the establishment all-out two or more connections to receive streaming data from the streaming server and received by said data communication means, based on the connection information data, more-out該確elevational all A connection establishment request for establishing a connection is transmitted to the streaming server by the data communication means, and streaming data transmitted from the streaming server through the connection is received by the data communication means and reproduced. In a streaming data reception / playback terminal comprising a control means for performing reception / playback processing for
Said control means, for all of the established all-out two or more connections, whether or not the connection is established is determined for each said connection when the connection is determined that has been established, the user of the effect that the connection has been established A streaming data receiving / reproducing terminal characterized in that a connection establishment notification image for notification is displayed on the display means individually for each connection. In the streaming data reception / playback terminal according to claim 1, 2, or 3,
Information notifying means for notifying the user of information;
Operating means,
When it is determined that the number of connections to be established is two or more based on the received connection information data, the control means determines whether or not two or more connections to be established have been established within a predetermined time. And when it is determined that only a part of the two or more connections to be established has been established, the user is prompted to select whether or not to perform streaming data reproduction. Information is notified by the information notification means, and when the selection operation for reproduction is performed on the operation means, the reception / reproduction processing is performed only through the established connection, and the operation means is not reproduced. A streaming data receiving / reproducing terminal that performs processing for disconnecting an established connection when the selection operation is performed.

Images