JP2005348144A - Information terminal device, method and program for shared media data exhibition - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an information terminal device, a shared media data exhibition method, and a shared media data exhibition program for appropriately setting an exhibition method for media data to be shared, according to the number and/or properties of the media data to be shared, as well as unifying the ways how the media data to be shared are shown. <P>SOLUTION: A template storage 304 stores a plurality of template data which define media data to be exhibited and exhibition methods suitable for the media data to be exhibited. A template selecting table storage 320 stores a template selecting table which defines each correspondence between the template data and the numbers and/or properties of media data to be shared. A template selector 305 refers to the template selecting table when shared media data are changed over, and specifies template data corresponding to the number and/or properties of media data to be shared after the changeover. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an information terminal device, a shared media data presentation method, and a shared media data presentation program, and more particularly to a desirable presentation technique for shared media data.

  2. Description of the Related Art Conventionally, a system for performing a teleconference or cooperative work using a plurality of information terminal devices has been proposed. For example, Patent Document 1 discloses a system capable of exchanging multimedia information in real time with a plurality of information terminal devices. With such a system, multimedia information can be shared and used between a plurality of terminal devices.

  Further, in Patent Document 2, when the sizes of the display devices of a plurality of information terminal devices are different, according to the display size, in order to prevent the appearance of shared multimedia data from being different between terminals. A method for converting and displaying the size of multimedia data to be shared is disclosed. By using such a method, the user of each information terminal device can see the same part of the shared multimedia, so that smooth communication can be performed.

Patent Document 3 discloses a plurality of information by modifying a Web page to be transmitted to each information terminal device in accordance with designation of a style sheet that defines a window size, a table / list size, a font size, and a margin. The terminal device is configured to unify the appearance of the Web page.
JP-A-7-87210 JP-A-8-263257 JP 2001-243165 A

  By the way, the appropriate presentation method of media data differs depending on the number of media data to be shared and / or its characteristics between the information terminal devices. When the number and / or characteristics of media data to be changed changes, the method of presenting media data cannot be changed.

  For example, when the media data of the shared video is a video of your face and the other party's face, such as a videophone, the other party's face image is displayed larger and your own face image is smaller. Displaying is considered to be one of the effective presentation methods. Therefore, when both terminals use such a presentation method, it is possible to unify the appearance of the shared media and set the appearance appropriately.

  On the other hand, during a videophone call, when one of the terminals further transmits the material video in the disc, the number of media data of the video to be shared increases to three. When the number of media to be shared increases to three in this way, only the two images can be displayed by the above-described presentation method, so that the position and size of the media data to be presented are changed without permission. Communication may not be successful.

  Therefore, an object of the present invention is not only to unify the appearance of the media data shared between the own terminal and the communication partner side terminal, but also to share according to the number and / or characteristics of the shared media data. To provide an information information terminal device, a shared media data presentation method, and a shared media data presentation program capable of appropriately setting a media data presentation method.

  In order to solve the above problems, an information terminal device according to the present invention is an information terminal device that shares a method of presenting media data shared with a communication partner side device, and presents media data, The first storage means for storing a plurality of media control information that defines the method of presenting the media data to be presented, the number and / or characteristics of the media data shared between the communication counterpart devices, and the own terminal When switching the media data shared between the second storage means for storing the table that defines the correspondence with the media control information for use and the communication partner apparatus, refer to the table in the second storage means Then, the media control information for the own terminal corresponding to the number and / or characteristics of the media data to be shared after switching is specified, and the specified media control information is read from the first storage means. And be presented method determining means, based on the read medium control information, and a presentation means for presenting media data to be shared after switching.

  Preferably, the table in the second storage means further defines the correspondence between the number and / or characteristics of the media data and the media control information for the communication partner apparatus, and the presentation method determining means When the data is switched, the table in the second storage means is further referenced to specify the number of media data to be shared after switching and / or the media control information for the communication partner side device corresponding to the characteristics. Then, the specified media control information is read from the first storage means, and the information terminal device further includes means for transmitting the read media control information to the communication counterpart device.

  More preferably, the characteristics of the shared media data in the table are defined by at least one of the type of media data, the codec method of the media data, the size of the media data, the bit rate of the media data, and the source of the media .

  More preferably, the media control information for the own terminal and the media control information for the device on the communication partner side are used to present the media data received from the device on the communication partner side via the network by the device that uses the media control information. It includes information defining presentation of media data acquired by the photographing means of the apparatus.

  More preferably, the media control information for the own terminal and the media control information for the device on the communication partner side include information for determining the presentation timing of the media data to be presented.

  More preferably, the media control information for the own terminal and the media control information for the device on the communication partner side include information that defines the layout of the media data to be presented.

  More preferably, the media control information for the own terminal and the media control information for the device on the communication partner side include information for determining the volume and tone of the media data of the audio to be presented.

  More preferably, the media control information for the own terminal and the media control information for the device on the communication partner side include information defining presentation of media data stored in the storage means of the device using the media control information.

  More preferably, the media control information for the own terminal and the media control information for the device on the communication partner side are stored in a device other than the device on the communication partner side connected to the terminal using the network via the network. Contains information that defines the presentation of data.

  More preferably, the media control information includes information that defines presentation of character data and / or graphic data generated by a drawing unit of a terminal that uses the media control information.

  More preferably, the second storage unit stores a table for each of the plurality of devices connected to the own device and the network, and the presentation information determination unit selects the table corresponding to the communication partner side device. Referring to the selected table, the media control information for the own terminal and the media control information for the device on the communication partner side are specified.

  Preferably, the presenting method determining means has no media control information for the terminal corresponding to the number and / or characteristics of the media data shared with the communication partner side device in the table, or When the media control information specified in the first storage means does not exist, the media control information distribution request is transmitted to the communication partner side device together with the number and / or characteristics of the media data to be shared after switching, and the communication partner side The media control information transmitted from the device is received, and the presenting means presents the media data to be shared after switching based on the received media control information.

  The information terminal device according to the present invention is an information terminal device that shares a method of presenting media data shared with a communication partner side device, and is a media to be presented transmitted from the communication partner side device. Means for receiving media control information defining data and a method for presenting media data to be presented, and means for presenting shared media data based on the received media control information.

  Preferably, the information terminal device further includes means for processing the received media control information according to the function and processing capability of the own device, and the means for presenting is a media to be shared based on the processed media control information. Present the data.

  Further, the media data presentation method according to the present invention includes a first storage unit that stores a plurality of media control information that defines media data to be presented and a media data presentation method, and a communication partner side device. And a second storage means for storing a table that defines the correspondence between the number of media data to be shared in the network and / or its characteristics, the media control information for the own terminal, and the media control information of the communication partner device. A shared media data presentation method in which a shared media data presentation method is shared between a first information terminal device and a second information device, wherein the first information terminal device includes a second information When switching the media data shared with the terminal device, the table in the second storage means is referred to, and the number of media data shared after the switching and / or the characteristics thereof are determined. Identifying the media control information for the terminal to be read, reading the identified media control information from the first storage means, and the first information terminal device sharing after switching based on the read media control information A step of presenting media data, and a second information terminal corresponding to the number and / or characteristics of the media data shared by the first information terminal device after switching by referring to the table in the second storage means Identifying media control information for the device, reading the identified media control information from the first storage means, and the first information terminal device transmitting the read media control information to the second information terminal device A step in which the second information terminal apparatus receives media control information from the first information terminal apparatus; and a media control information received by the second information terminal apparatus. Based on the information, and presenting the media data to be shared.

  In addition, the shared media data presentation program according to the present invention provides a computer with media data to be presented and a media data presentation to be presented in order to share a media data presentation method shared with a communication partner apparatus. First storage means for storing a plurality of media control information defining a method, the number and / or characteristics of media data shared between the communication partner side device, media control information for the terminal itself, When switching media data to be shared between the second storage means for storing the table that defines the correspondence between the device and the communication partner side device, refer to the table in the second storage means and share it after the switching. Identifying the media control information for the terminal corresponding to the number and / or characteristics of the media data to be read, and reading the identified media control information from the first storage means. And it is presented method determining means, based on the read medium control information, to function as presentation means for presenting media data to be shared after switching.

  According to the information information terminal device, the shared media data presenting method, and the shared media data presenting program according to the present invention, not only the appearance of the media data shared between the own terminal and the communication partner terminal is shared, but also shared. The presentation method of the media data to be shared can be appropriately set according to the number and / or characteristics of the media data to be shared.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following description, the same components are given the same reference numerals, and the names and functions thereof are the same. Therefore, the description of the components given the same reference numerals will not be repeated.

(Configuration of multimedia data sharing system)
FIG. 1 shows a configuration of a multimedia data sharing system according to an embodiment of the present invention. Referring to FIG. 1, videophone connection server 1 provides a videophone connection service. The videophone connection server 1 has a SIP (Session Initiation Protocol) proxy function used in IP (Internet Protocol) telephones and the like. The content server 4 stores multimedia contents such as video images and music.

  When the user of the information terminal device 2 makes a videophone call to the user of the information terminal device 3, when the user makes a call to the information terminal device 3 via the videophone connection server 1, the information terminal device 3 receives the call. In the information terminal device 3, the incoming call of the videophone is notified by a ringing tone or the like. When the user of the information terminal device 3 performs an operation of receiving a videophone, the videophone is started between the information terminal device 2 and the information terminal device 3, and the user's video and audio are transmitted bidirectionally.

  When the user of the information terminal device 2 designates desired multimedia content to the content server 4 and requests distribution thereof, the content server 4 distributes the designated multimedia content to the information terminal device 2 on demand. To do.

(Communication function in multimedia data sharing system)
Next, communication functions provided in the multimedia data sharing system will be described. FIG. 2 shows a communication function in the multimedia data sharing system according to the embodiment of the present invention. Referring to FIG. 2, this multimedia data sharing system performs the following communication.

(1) Videophone The multimedia data sharing system has a videophone function. In order to make a videophone call, first, establishment control of a videophone session (SE1) is performed by SIP between the information terminal device 2 and the information terminal device 3 via the telephone connection server 1 (1). -A). After the establishment of the videophone session (SE1), the audio and video data of the videophone is transmitted by RTP (Real time Transport Protocol) (1-b) through this session (SE1).

(2) Sharing of local content The user of the information terminal device 2 displays local content such as a video image stored in the hard disk of the information terminal device 2 on the own device during conversation with the other party of the videophone, The local content can be transmitted to the communication partner side and presented on the communication partner side. This is called local content sharing. In order to share this local content, establishment control of a session (SE3) for local content communication is performed between the information terminal device 2 and the information terminal device 3 (2-a). After the session (SE3) for local content communication is established, local content communication is performed between the information terminal device 2 and the information terminal device 3 using this session (SE3) (2-b).

(3) Sharing remote content The user of the information terminal device 2 acquires and displays remote content such as video images stored in the hard disk of the content server 4 with his / her device during a conversation with a videophone partner. At the same time, the remote content can be acquired and presented to the communication partner.

  When the content is provided as a file such as a still image, the remote content is acquired by designating a URL (Uniform Resource Locator) of the remote content and making a transmission request (3-a), and HTTP (HyperText Transfer). This is performed by receiving remote content transmitted by (Protocol) (3-b). On the other hand, when the remote content is stream data such as video, the session establishment control is performed by SIP, and after the session is established, the remote content transmitted by RTP is received by the session. It is.

(4) Communication of template data Template data is data that specifies a method of presenting shared media data, and is a type of media control information. For example, when the shared media data is a videophone, the template data for the videophone is used. When the shared media data is the videophone and the local content, the template data for making a videophone while viewing the local content together is used. Is done.

  In order for the information terminal device 2 and the information terminal device 3 to share a shared media data presentation method, it is necessary for both terminal devices to use template data that provides a common presentation method. .

  When the information terminal device 2 presents media data to be shared using certain template data, the information terminal device 2 provides template data that defines a common presentation method and a template data presentation method used by the information terminal device 2. It transmits to the information terminal device 3. In order to perform communication of template data, first, establishment control of a session (SE2) for template communication is performed by SIP between the information terminal device 2 and the information terminal device 3 (4-a). After the template communication session (SE2) is established, the template data is communicated by HTTP using this session (SE2) (4-b).

(Configuration of information terminal device)
FIG. 3 is a functional block diagram showing a configuration of the information terminal device 2. The same applies to the information terminal device 3. With reference to the figure, the information terminal device 2 includes a main body 301 and a remote controller 302. The main body unit 301 includes a light receiving unit 311, a camera 312, a microphone 313, a light 315, a speaker 316, a display 317, and a communication interface 318.

  The light receiving unit 311 receives a signal from the remote controller 302.

  The camera 312 photographs the user and acquires the user's video.

  The microphone 313 captures the user's voice.

  The light 315 is lit to notify the user when an incoming call is received.

  The speaker 316 outputs the received voice from another information terminal device.

  The display 317 displays a video of a user on the other side of the videophone, a video of a user taken by a camera, and the like.

  The communication interface 318 is an interface for communicating with other information terminal devices through the Internet network.

  The information terminal device further includes a data processing unit 303, a template storage unit 304, a template selection unit 305, a data processing change unit 306, a data processing change information transmission unit 307, and a data processing change information reception unit 308. The communication negotiation unit 309 and the data communication unit 310 are provided. The functions of these components are realized by a computer executing a media data sharing program. That is, the CPU executes a media data sharing program that defines the function of each component expanded on the memory.

  The communication negotiation unit 309 negotiates communication using SIP, and controls and manages session establishment.

  The data communication unit 310 performs data communication using the session established by the communication negotiation unit 309. The data processing unit 303 directly transmits data for communication that does not require establishment of a session, such as HTTP.

  The local content storage unit 319 stores local content such as video images, still images such as photographs, or wallpaper (background screen) data.

  The shared media information storage unit 321 stores shared media information that is information about media data shared with other information terminal devices. Sharing media data means that both terminal devices have the same media data. The possession includes not only the state stored in the disk but also the state received by the receiving means. In this shared media information, the shared media information of the session is added after the session is established, and the shared media information of the session is deleted after the session ends.

  FIG. 4 shows an example of shared media information. Referring to the figure, the session number, the communication partner, the session establishment time, the media number, the media type, the media codec, the size of the image when the media data is an image or video, and the media data Consisting of a source.

  The shared media information in FIG. 4 indicates that sessions with session numbers 1, 2, and 3 have been established.

  In the shared media information, the session with the session number 1 is the communication terminal is the information terminal device A, the session establishment time is 10:22:23, and the media 1, 2, 3, and 4 Media 1 is an MPEG4 codec, 640x480, local camera source video, media 2 is a PCMU codec, local microphone is the source Audio, media 3 is MPEG4 codec, 640x480 size, source camera is the source camera, media 4 is PCMU codec, destination microphone is source audio Represents.

  In this example, each device has data. However, it is also possible to specify data held by another device on the local network as a source by using a URL.

  In the shared media information, the session with the session number 2 is that the communication partner is the information terminal device A, the session establishment time is 10:25:24, and the media data of the media 5 is communicated. The media 5 indicates that the data on the local disk having the size of 320 × 240 of the MPEG2 codec, that is, the data held in the local content storage unit 319 is designated as the source video.

  In the shared media information, the session with the session number 3 is that the communication partner is the information terminal device A, the session establishment time is 10:26:27, and the media data of the media 6 is communicated. , Media 6 indicates that a JPEG codec, 160 × 160 size still image on a communication partner disk is designated as source data.

  The template storage unit 304 stores a plurality of template data. The template data defines media data to be presented and a method for presenting the media data to be presented. The media data to be presented is selected from the media data to be shared. For example, the video of the communication partner's camera, the video or still image (local content) of the communication partner's disk, the video of the camera of its own terminal, the content server The video or still image (remote content) of the internal disc, the video or still image (local content) of the disc of the terminal, characters or graphics generated by the drawing means of the terminal are designated.

  In addition, for example, the media data layout (position, size, enlargement / reduction ratio, display amount designation, etc.), presentation timing, or volume and tone in the case of voice are defined as the presentation method. A specific example of the template data will be described later.

  The template selection table storage unit 320 stores a template selection table. FIG. 5 shows an example of a template selection table. As shown in the figure, conditions for selecting template data specified by template numbers 1, 2, or 3 are defined. This condition is specified by the combination of the communication partner, the number of media data, the type of media data, the codec of the media data, the size of the image when the media data is a still image or video, and the source of the media data. The

  For example, in the template data with the template number 1, the communication partner is unspecified (not limited), the number of shared media data is four media A, B, C, and D. MPEG4 codec, 640x480, local camera source video, media B is PCMU codec, local microphone is source audio, media C is MPEG4 codec 640x This is selected when the counterpart camera with the size of 480 is the source video, the media D is the PCMU codec, and the counterpart microphone is the source audio.

  The template selection unit 305 updates the shared media information updated from the shared media information storage unit 321 when the shared media information storage unit 321 is updated, that is, when the shared media is switched due to establishment or termination of a session. Read. Then, the template selection unit 305 specifies the number of shared media data, the type of each media data, the codec, the size in the case of an image, and the source. The template selection unit 305 reads the template selection table from the template selection table storage unit 320, and selects the template data for the own device that satisfies the specified conditions and the template data for the counterpart terminal.

  In the case of the shared media information shown in FIG. 4 and the template selection table shown in FIG. 5, the template selection unit 305 selects the template data of template number 3 for the own device and the template number 4 for the other terminal. Select template data.

  The data processing unit 303 performs main processing of information processing of the main body unit 301. For example, the data processing unit controls establishment of a session by the communication negotiation unit 309. In addition, the data processing unit 303 reads the template data for the own device selected by the template selection unit 305 from the template storage unit 304 and sets it as template data used by the own device. Further, the data processing unit 303 reads the template data for the partner device selected by the template selection unit 305 from the template storage unit 304 and transmits the template data to the device on the communication partner side through the data communication unit 310.

  The data processing unit 303 outputs the shared media data to the display 317 and the speaker 316 according to the set template data of the own device.

  When the template data selected by the template selection unit 305 does not exist in the template storage unit 304, the data processing change unit 306 selects a rule that satisfies the condition with the read shared media information from the stored rule set. Search and identify process change information. A rule is a type of media control information, similar to template data. FIG. 6 is an example of a rule set.

  As shown in the figure, rule 1 is applied when the condition that the number of media data is 1 is satisfied, and the process change information is “display media X in full screen”.

  Rule 2 is applied when the condition that the number of media data is 2, the source of the media X is the camera on the other side, and the source of the media Y is the local camera is satisfied. “Media X is displayed in full screen & Media Y is displayed in the lower right corner 1/9”.

  Rule 3 is applied when the condition that the number of media data is 2, the source of the media X is the partner's camera, and the source of the media Y is the partner's disk is satisfied. “Media X is displayed on the left half & Media Y is displayed on the right half”.

  Rule 4 is applied when the number of media data is 3, and the process change information is “two media data are randomly selected and the selected one is displayed as default”.

  The data process change information transmission unit 307 transmits the process change information specified by the data process change unit 306.

  The data processing change information receiving unit 308 receives the processing change information specified by the data processing changing unit of another information terminal device.

  FIG. 7 shows a detailed configuration of the remote controller 301. Referring to FIG. 3, remote control 302 includes utterance button 401, end call button 402, input button 403, operation button 404, share button 405, control button 406, and switch button 407.

  The utterance button 401 is a button for instructing the start of a call when a videophone call is made or received. The utterance button 401 functions as a toggle. When the utterance button 401 is pressed once, a voice call is made. When the utterance button 401 is pressed again, a videophone call is made.

  The end call button 402 is a button for instructing the end of the call.

  The input button 403 includes 13 buttons, and is a button for inputting numbers and characters such as a telephone number or URL.

  The operation button 404 is a trackball, and moves the cursor or the like by moving it up, down, left, or right, and a menu at the cursor position is selected by pressing down the center.

  The share button 405 is a button for sharing the content of the own device (local content) or the content of the content server 4 (remote content) with another information terminal device during a videophone call. By pressing once, a local content selection window is displayed, and a remote content selection window is displayed.

  The control button 406 is a button for controlling playback, pause, and stop of the video when the local content including the video is shared with other information terminal devices.

  The switch button 407 is a button for switching between performing communication between the remote controller and the main body using infrared rays or using a control line.

(Communication procedure and screen transition)
Next, the communication sequence of this multimedia data sharing system and the transition of displayed screens will be described. FIG. 8 is a diagram illustrating an example of a communication sequence of the multimedia data sharing system. FIG. 9 shows a screen displayed in the communication sequence of the multimedia data sharing system.

  Referring to FIG. 8, the network address of information terminal device 2 is “c1.videophone.com”, the telephone number is “050-123-5678”, and the network address of information terminal device 3 is “c2.videophone.com”. The telephone number is “050-123-4567”, and the network address of the videophone connection server 1 is “srv.videophone.com”.

  First, the information terminal device 2 sends an “INVITE” message 501, which is a SIP call signal, including the telephone number of the information terminal device 3 as a destination to the videophone connection server 1.

  The videophone connection server 1 receives the “INVITE” message 501 and extracts the telephone number of the information terminal device 3 included in the “INVITE” message 501. The videophone connection server 1 resolves the network address of the information terminal device 3 based on the telephone number of the information terminal device 3. Then, the videophone connection server 1 transmits an “INVITE” message 502 obtained by correcting the “INVITE” message 501.

  When the information terminal device 3 receives the “INVITE” message 502, the information terminal device 3 notifies the user that the videophone call request has been received. When the user selects to accept an incoming call, the information terminal device 3 sends a “200 OK” response message 503 to the videophone connection server 1. Upon receiving the “200 OK” response message 503, the videophone connection server 1 transmits a 200 OK ”response message 504 to the information terminal device 3.

  In this way, the videophone session SE1 is established. Thereafter, between the information terminal device 2 and the information terminal device 3, the videophone audio and video communication can be performed by the videophone call session SE1.

  After the videophone call session SE1 is established, the information terminal device 2 makes a template communication call using SIP to the information terminal device 3 as follows. Since the template data communication is independent of the videophone service, it is performed directly between the information terminal devices without using the videophone connection server 1.

  The information terminal device 2 transmits an “INVITE” message 505 to the information terminal device 3. The information terminal apparatus 3 receives the “INVITE” message 505, and transmits a “200 OK” response message 506 if the apparatus is compatible with reception of template data.

  In this way, the session SE2 for template data communication is established. Thereafter, template data can be communicated between the information terminal device 2 and the information terminal device 3 through the template communication session SE2. In addition, the session SE2 maintains the connection even after the template data communication is completed, so that it is possible to communicate the changed template data not only at the start of the videophone call but also during the call. It can be changed.

  The information terminal device 2 transmits the template data for videophone to the information terminal device 3 using the session SE2 for template communication, and the information terminal device 3 receives this. Further, the information terminal device 2 and the information terminal device 3 perform videophone video and audio communication using the videophone session SE1.

  The information terminal device 2 and the information terminal device 3 display videophone video on the display 317 and output videophone audio to the speaker 316 using videophone template data.

  FIG. 9A shows an example of a screen displayed by template data for videophone. As shown in the figure, this template data displays the entire received videophone video (video from the other party's camera) and the transmitted videophone video (local camera video) in the lower right corner. Yes.

  In this state, the user of the information terminal device 2 selects local content to be shared.

  FIG. 9B shows an example of a screen when local content sharing is selected. As shown in the figure, the user of the information terminal device 2 displays a content list, and selects one of the displayed local contents from the remote controller.

  The information terminal device 2 makes a call for local content communication using SIP to the information terminal device 3 as follows. Since communication of local contents is independent of the videophone service, it is directly performed between the information terminal devices without using the videophone connection server 1.

  The information terminal device 2 transmits an “INVITE” message 509 specifying the local content to be shared to the information terminal device 3. The information terminal device 3 receives the “INVITE” message 509 and specifies the content to be shared from the “INVITE” message 509. When the user of the information terminal device 3 designates acceptance of sharing of the content, the information terminal device 3 transmits a “200 OK” response message 510.

  In this manner, the session SE3 for local content communication is established. Thereafter, communication of local content is possible between the information terminal device 2 and the information terminal device 3 through the session SE3 for communication of local content.

  The information terminal device 2 transmits template data for videophone + local content to the information terminal device 3 through the session SE2 for template communication, and the information terminal device 3 receives this.

  The information terminal device 2 transmits the local content to the information terminal device 3 through the session SE3 for communication of local content, and the information terminal device 3 receives this.

  The information terminal device 2 and the information terminal device 3 display the videophone video and local content on the display 317 using the videophone + local content template data, and output the videophone audio to the speaker 316.

  FIG. 9C shows an example of a screen displayed by template data for videophone + local content. As shown in the figure, with this template data, the local content is displayed in the upper left corner, the received videophone video (the video of the other party's camera) is displayed in the lower right corner, and the transmitted videophone video (local Is displayed in the lower right corner.

(Processing when making a videophone call)
FIG. 10 is a flowchart showing a processing procedure when a videophone call is made. With reference to the figure, first, the data processing unit 303 accepts the transition to the call preparation mode input through the utterance button 401 of the remote control 302 (S701).

  Next, the data processing unit 303 receives the telephone number of the information terminal device of the communication partner input through the input button 403 of the remote controller 302 (S702).

  Next, the data processing unit 303 sends a request for establishing a videophone session (SE1) with the information terminal device having the received telephone number to the communication negotiation unit 309. The communication negotiation unit 309 transmits an INVITE message, which is a videophone call signal, including the received telephone number as a destination (S703).

  Next, the communication negotiation unit 309 receives a response message which is a response signal for a videophone call (S704).

  Next, the communication negotiation unit 309 determines whether a videophone session (SE1) has been established based on the received response message (S705).

  When the videophone session (SE1) is not established, that is, when the response message is other than “200 OK” response message, for example, “603 Decline”, the communication negotiation unit 309 causes the data processing unit 303 to Notify that. The data processing unit 303 displays on the display 317 that the videophone session (SE1) could not be established (S706).

  On the other hand, when the videophone session (SE1) is established, that is, when the response message is a “200 OK” response message, the communication negotiation unit 309 notifies the template selection unit 305 to that effect. The template selection unit 305 refers to the shared media information stored in the shared media information storage unit 321 and the template selection table stored in the template selection table storage unit 319, and determines the template data to be used in its own device. The data is specified and output to the data processing unit 303 (S707).

  The data processing unit 303 checks whether the specified template data exists in the template storage unit 304 (S708).

  When the specified template data does not exist in the template storage unit 304, the data processing unit 303 sends the shared media information to the data processing change unit 306. The data processing change unit 306 creates data processing change information based on the shared media information and the rule set, and sets it as data processing change information of the own device (S709).

  The data processing unit 303 sends a request for establishment of a session (SE4) for communication of data processing change information to the communication negotiation unit 309. The communication negotiation unit 309 transmits an INVITE message that is a signal for requesting establishment of a communication session (SE4) for data processing change information (S710).

  Next, the communication negotiation unit 309 receives a response message that is a response signal of the request for establishing the session (SE4) for communication of the data processing change information (S711).

  The communication negotiation unit 309 transmits the data processing change information through the data processing change information communication session (SE4) when the data processing change information communication session (SE4) is established. On the other hand, when the communication session (SE4) for the data processing change information is not established, the communication negotiation unit 309 leaves the processing to the partner apparatus (S712, S713).

  On the other hand, when the specified template data exists in the template storage unit 304, the data processing unit 303 reads out the template data and sets it as template data to be used in its own device (S714).

  The data processing unit 303 sends a request for establishing a template communication session (SE2) to the communication negotiation unit 309. The communication negotiation unit 309 transmits an INVITE message, which is a signal for requesting establishment of a template communication session (SE2) (S715).

  Next, the communication negotiation unit 309 receives a response message that is a response signal of a request for establishing a template communication session (SE2) (S716).

  Next, the communication negotiation unit 309 determines whether a template communication session (SE2) has been established based on the received response message (S717).

  When the template communication session (SE2) is established, that is, when the response message is a “200 OK” response, the communication negotiation unit 309 notifies the template selection unit 305 to that effect. The template selection unit 310 refers to the template selection table stored in the template selection table storage unit 319, specifies the template data used in the information terminal device on the communication partner side, and uses the specified template data as the template storage unit The data is read from 304 and output to the data communication unit 310. On the other hand, when the template communication session (SE2) is not established, the communication negotiation unit 309 leaves the processing to the partner apparatus (S718).

  Next, the data communication unit 310 transmits the template data received from the data communication unit 310 through the template communication session (SE2) (S719).

  Next, the data communication unit 310 transmits the video from the camera 312 of the own device and the sound from the microphone 313 of the own device through the videophone session (SE1), and the video from the camera of the other device and the other device. The voice from the microphone is received (S720).

  Next, in accordance with the set template data or data processing change information of the own device, the data processing unit 303 receives the video input from the camera 312, the audio input from the microphone 313, and the partner device received by the data communication unit 310. The video from the camera and the sound from the microphone are output to the display 317 and the speaker 316 (S721).

(Communication content switching process during a videophone call)
FIG. 11 is a flowchart showing a processing procedure for switching communication contents during a videophone call. With reference to the figure, first, the data processing unit 303 accepts sharing of local content input via the share button 401, the operation button 404, and the input button 401 of the remote control 302 and the specification of the local content to be shared (S801). .

  Next, the data processing unit 303 sends a request for establishing a session (SE3) for communication of local content to the communication negotiation unit 309. The communication negotiation unit 309 transmits an INVITE message, which is a signal for requesting establishment of a session (SE3) for local content communication (S802).

  Next, the communication negotiation unit 309 receives a response message that is a response signal of a request for establishing a local content communication session (SE3) (S803).

  Next, the communication negotiation unit 309 determines whether a local content communication session (SE3) has been established based on the received response message (S804).

  When the local content communication session (SE3) is not established, that is, when the response message is other than the “200 OK” response message, the communication negotiation unit 309 notifies the data processing unit 303 to that effect. The data processing unit 303 displays on the display 317 that the local content communication session (SE3) could not be established (S822).

  On the other hand, when a session (SE3) for local content communication is established, that is, when the response message is a “200 OK” response message, the communication negotiation unit 309 notifies the template selection unit 305 to that effect. The template selection unit 305 refers to the template selection table stored in the template selection table storage unit 319, specifies template data to be used by the own device in the local content communication session (SE3), and performs data processing. The data is output to the unit 303 (S805).

  The data processing unit 303 checks whether the specified template data exists in the template storage unit 304 (S806).

  When the specified template data exists in the template storage unit 304, the data processing unit 303 reads the template data and sets it as template data to be used in the own device (S807).

  The data processing unit 303 checks whether a template communication session (SE2) has already been established (S808).

  If the template communication session (SE2) has not been established yet, the data processing unit 303 sends a template communication session (SE2) establishment request to the communication negotiation unit 309. The communication negotiation unit 309 transmits an INVITE message, which is a signal for requesting establishment of a template communication session (SE2) (S809).

  Next, the communication negotiation unit 309 receives a response message that is a response signal of a request for establishing a template communication session (SE2) (S810).

  Next, the communication negotiation unit 309 determines whether a template communication session (SE2) has been established based on the received response message (S811).

  When the template communication session (SE2) is established, that is, when the response message is a “200 OK” response message, or in step S808, the communication negotiation unit 309 determines that the template communication session (SE2) has already been established. If so, the template selection unit 305 is notified of this. The template selection unit 310 refers to the template selection table stored in the template selection table storage unit 319, specifies the template data used in the information terminal device on the communication partner side, and uses the specified template data as the template storage unit The data is read from 304 and output to the data communication unit 310 (S812).

  Next, the data communication unit 310 transmits the template data through the template communication session (SE2) (S813).

  On the other hand, when the specified template data does not exist in the template storage unit 304, the data processing unit 303 sends the shared media information to the data processing change unit 306. The data processing change unit 306 creates data processing change information based on the shared media information and the rule set, and sets it as the data processing change information of the own device (S814).

  The data processing unit 303 checks whether a communication session (SE4) for data processing change information has already been established (S815).

  When the data processing change information communication session (SE4) has not yet been established, the data processing portion 303 sends a request for establishment of the data processing change information communication session (SE4) to the communication negotiation unit 309. . The communication negotiation unit 309 transmits an INVITE message that is a request signal for establishing a session (SE4) for communication of data processing change information (S816).

  Next, the communication negotiation unit 309 receives a response message that is a response signal of the establishment request for the session (SE4) for communication of the data processing change information (S817).

  The communication negotiation unit 309 determines that the data processing change information communication session (SE4) has been established (S818) or the data processing change information communication session (SE4) has already been established in step S815. Then, the data processing change information is transmitted through the session (SE4) (S819).

  Next, the data processing unit 303 reads the local content stored in the media data storage unit 320 and sends it to the data communication unit 310. The data communication unit 310 communicates local content through the local content communication session (SE3) (S820).

  Next, the data processing unit 303 receives the video input from the camera 312, the audio input from the microphone 313, the local content, and the data communication unit 310 according to the set template data or data processing change information of the own device. The video from the camera of the partner device and the sound from the microphone are output to the display 317 and the speaker 316 (S821).

(Processing of the terminal on the incoming side of a videophone)
FIG. 12 is a flowchart showing the processing procedure of the terminal on the incoming side of the videophone. Referring to the figure, first, upon receiving an INVITE message, which is a session establishment request signal, via communication interface 318, communication negotiation unit 309 sends the INVITE message to data processing unit 303 (S901).

  Next, the data processing unit 303 displays the type of the received INVITE message, the user name of the transmission source information terminal device, and the like on the display 317 and outputs them to the speaker 316 (S902).

  Next, the data processing unit 303 accepts a response to the session establishment request input through the input button 403 of the remote controller 302 (S903).

  The data processing unit 303 creates a response message based on the received response (S904).

  The communication negotiation unit 309 transmits the created response message (S905).

  When receiving the template data through the template communication session (SE2), the data communication unit 310 sends the template data to the data processing unit 303. The data processing unit 303 stores the received template data in the template storage unit 304. Then, the data processing unit 303 sets the received template data as template data to be used by the own device (S906, S907).

  When receiving the data processing change information through the data processing change information communication session (SE4), the data receiving unit 310 sends the data processing change information to the data processing unit 303. The data processing unit 310 sets the received data processing change information as the data processing change information of its own device (S908, S909).

  When receiving the videophone video and audio through the videophone session (SE1), the data receiving unit 310 sends the videophone video and audio to the data processing unit 303 (S910, S911).

  When receiving the local content through the local content communication session (SE3), the data receiving unit 310 sends the local content to the data processing unit 303 (S912, S913).

  Next, in accordance with the set template data or data processing change information of the own device, the data processing unit 303 receives the video input from the camera 312, the audio input from the microphone 313, and the partner device received by the data communication unit 310. The video from the camera, the sound from the microphone, and the local content of the disc are output to the display 317 and the speaker 316 (S914).

(Example of message for establishing a videophone session)
FIG. 13 shows an example of a message communicated when the videophone session SE1 is established. The establishment of the videophone session SE1 conforms to SIP (IETF RFC 3261). In the example of the message in FIG. 13, portions that are not directly related to the present embodiment are omitted for the sake of brevity.

  FIG. 13A is a diagram showing an example of an INVITE message 501 which is a videophone call signal transmitted from the information terminal device 2. The first line in the figure shows that this message is an “INVITE message. In the second line, this message is“ SIP / 2.0 / UDP c1.videophone.com:5060 ”. It is shown that the request is for a videophone connection via the videophone connection server 1. In addition, in the To field, the destination of this message is “050-123-4567@videophone.com”, that is, an information terminal. The device 3 is shown. Also, the From field indicates that the sender of this message is “050-123-5678@videophnoe.com”, that is, the information terminal device 2. It is also shown that the media data communicated by the videophone is PCMU audio and MP4 video.

  FIG. 13B is a diagram showing an example of an “INVITE” message 502 which is a videophone call signal transmitted from the videophone connection server 1. In the first line of the figure, it is indicated that this message is an “INVITE” message. The second line indicates that this message is “SIP / 2.0 / UDP srv.videophone.com:5060”, that is, a videophone connection request via the videophone connection server 1. . In addition, the To field indicates that the transmission destination of this message is “050-123-4567@videophone.com”, that is, the information terminal device 3. Further, the From field indicates that the transmission source of this message is “050-123-5678@videophnoe.com”, that is, the information terminal device 2.

  FIG. 13C is a diagram showing an example of a “200 OK” response message 503 which is a response signal for a videophone connection request transmitted from the information terminal device 3. The first line of FIG. 6 indicates that this message is a “200 OK” response message. The To field indicates that the transmission destination of the original message responding by this response message is “050-123-4567@videophone.com”, that is, the information terminal device 3. The From field indicates that the source of the original message to which this response message is responding is “050-123-5678@videophnoe.com”, that is, the information terminal device 2.

  FIG. 13D is a diagram showing an example of a “200 OK” response message 504 which is a response signal for a videophone connection request transmitted from the videophone connection server 1. The first line of FIG. 6 indicates that this message is a “200 OK” response message. The To field indicates that the transmission destination of the original message to which this response message is responding is “050-123-4567@videophone.com”, that is, the information terminal device 3. The From field indicates that the source of the original message to which this response message is responding is “050-123-5678@videophnoe.com”, that is, the information terminal device 2.

  As a result of the exchange of the messages of FIGS. 13A, 13B, 13C, and 13D, session SE1, which is a logical communication connection of the videophone, is established.

(Example of message for establishing a session for template communication)
FIG. 14 shows an example of a message communicated when the template communication session SE2 is established.

  FIG. 14A is a diagram illustrating an example of an INVITE message 505 which is a signal for requesting establishment of a template communication session (SE2) transmitted from the information terminal device 2. FIG. The first line in the figure shows that this message is an “INVITE” message. In addition, the To field indicates that the transmission destination of this message is “050-123-4567@videophone.com”, that is, the information terminal device 3. The From field indicates that the sender of this message is “050-123-5678@videophnoe.com”, that is, the information terminal device 3. Also, “m = application 6071 HTTP / VCTMPLT” on the eighth line indicates that this message is a connection request for template communication by HTTP.

  FIG. 14B is a diagram illustrating an example of a “200 OK” response message 506 transmitted from the information terminal device 3. The first line of FIG. 6 indicates that this message is a “200 OK” response message. The To field indicates that the transmission destination of the original message to which this response message is responding is “050-123-4567@videophone.com”, that is, the information terminal device 3. The From field indicates that the source of the original message to which this response message is responding is “050-123-5678@videophnoe.com”, that is, the information terminal device 2. Further, “m = application 6071 HTTP / VCTMPLT” on the 8th line indicates that this response message is a response to the connection request for template communication by HTTP.

  Since the message 505 in FIG. 14A and the message 506 in FIG. 14B are independent of the videophone service, they do not need to be transmitted via the videophone connection server 1. Further, the information terminal device 3 receives the network address “c1. Of the information terminal device 2 from“ Contact: <sip: 050-123-5678@c1.videophone.com> ”included in the INVITE message 501 of FIG. videophone.com ”. In addition, the information terminal device 2 receives the network address “c2.” Of the information terminal device 3 from “Contact: <sip: 050-123-4567@c2.videophone.com>” included in the response message 504 in FIG. videophone.com ”. Therefore, the information terminal device 2 and the information terminal device 3 can directly communicate messages using the acquired network address without going through the videophone connection server 1.

  The same applies when establishing a session for communication of data processing change information, and “m = application 6071 HTTP / VCINFO” is used instead of “m = application 6071 HTTP / VCTMPLT”.

(Example of template data of own device)
FIG. 15 is an example of template data used by the own device (information terminal device 2). In the following template data example, information is described by specifying the type by “tag” surrounded by “<” and “>”. Further, information surrounded by “tags” is called “fields” of information corresponding to the tags. For example, “<templateType> VideoPhone </ templateType>” is called a <templateType> field. Referring to FIG. 8, a <templateType> field 1202 indicates the type of template data, and any one of “VideoPhone”, “VideoPhoneWithLocalContent”, and “VideoPhoneWithExternalContent” is designated.

  As shown in FIG. 15, "VideoPhone" indicates that this template data is template data used for videophone video output. “VideoPhoneWithLocalContent” indicates that the template data is template data used for videophone video and local content output. “VideoPhoneWithExternalContent” indicates that this template data is template data used for videophone video and remote content output.

  A <for> field 1203 indicates an information terminal device that uses this template data.

  The <content_List> field 1204 lists information for each content that defines the processing with this template data.

  Content information consists of the <c_id> field that is the content identifier, the <c_type> field that represents the content type, the <c_source> field that represents the content source, and the <c_action> field that represents the content processing method. Become.

  In this template data, the content of the video content of the local camera whose identifier is "my_camera", the content of the video content of the other camera whose identifier is "your_camera", and the content information of the wallpaper whose identifier is "wallpaper" Is stipulated.

  The content with the identifier "my_camera" is of type "Video", that is, video, the acquisition source is "local: VideoDevice1", that is, its own camera, and the processing method is "Display", that is, display on the display It is prescribed.

  The content whose identifier is “your_camera” is the type “Video”, that is, video, the acquisition destination is “network_session: VideoPhone”, that is, the video received in the videophone session SE1, and the processing method is “Display”. In other words, it is specified that it is displayed on the display.

  For the content with the identifier “wallpaper”, the local data specified by “local: content / image / wallpaper / wp1.jpg” is used up to 600 seconds, and “http: // wallpaper” is used after 600 seconds. The processing method using data outside the apparatus specified by “.com / wallpaper1.jpg” stipulates that data is displayed on the display for up to 1200 seconds. Therefore, the wallpaper is not displayed after 1200 seconds.

  In other words, the acquisition destination of 600 seconds or less is a storage unit such as a hard disk of its own device, and the acquisition destination of 600 seconds or less is a device other than the communication partner terminal connected to the network (that is, the content server “wallpaper.com”). It is stipulated that The information terminal device that uses the template data reads the data from its own hard disk, specifies the data by the URL, accesses the content server “wallpaper.com”, and acquires the data.

  In addition, by using the <time> tag in this way, it is possible to define the presentation timing of content, and to perform animation display and the like.

  A <layout> field 1205 defines the spatial arrangement or time arrangement (timing) of each content. A <position> field in the <layout> field 1205 indicates the upper left position where the content is displayed. The <size> field indicates the width and height of the content. Option information can be described in the <option> field, and “over (your_camera)” in this case indicates that this data is overwritten and displayed on your_camera content.

(Example of template data of the partner device)
FIG. 16 is an example of template data used by the counterpart device (information terminal device 3). The template data in the figure is almost the same as the template data in FIG. Differences from the template data of FIG. 15 will be described.

  A <for> field 1303 indicates an information terminal device that uses the template data.

  For the content with the identifier “wallpaper”, the wallpaper data to be used until 600 seconds have passed becomes the data specified by the information terminal device 2 at http://c1.videophone.com/image/wallpaper/wp1.jpg ing.

  Further, since the size of the display of the information terminal device on the other side is unknown for the <layout> field 1305, the <position> field and the <size> field are specified in the actual size in the template data shown in FIG. Unlike, it is determined by the ratio of the screen.

(Example of template data of own device)
FIG. 17 is another example of template data used by the own device (information terminal device 2). This template data is template data for displaying videophone video and local content.

  The content whose identifier is “your_camera” is defined to have a processing method defined as “ToSkin (videoOut1)”, that is, a videoOut1 item of layout data called skin.

  The content with the identifier “localContent” is the type “Video”, that is, the video, and the source is “local: content / video / video1.mpg”, that is, the video stored on the disk of the device, and processed. It is specified that the method is defined as “ToSkin (videoOut2)”, that is, the item of videoOut2 of the file that defines the layout called skin.

  Further, it is specified that a file defining a layout called skin is stored in a file “local: content / skin / video / videocomm1.skn”.

(Example of skin)
FIG. 18 is a diagram illustrating an example of a skin. This figure shows the contents of the file specified by “local: content / skin / video / videocomm1.skn” in FIG. Referring to FIG. 17, the position and size of “your camera” in the template data of FIG. 17 are defined in the <videoOut1> field. In the <videoOut2> field, the position and size of “localContent” used in the template data of FIG. 17 are defined.

As is clear from the above description, the multimedia sharing system according to the present embodiment is
By providing a template selection table, not only the appearance of the media data shared between the local terminal and the communication partner's terminal, but also sharing according to changes in the number and / or characteristics of the shared media data The presentation method of media data to be performed can be appropriately changed.

<Modification>
The present invention is not limited to the above embodiment, and includes, for example, the following modifications.

(1) Other Examples of Template Data FIG. 19 shows an example of a screen displayed based on certain template data. The template data of the screen example in the figure has two media data to be shared. One media data of the shared media data (in this example, a swimming video) is displayed on the left screen, and the other media data (this In the example, it is defined that the athletic meet video) is displayed on the right screen, and it is possible to display a plurality of media in the same arrangement in both communicating parties.

(2) Other Examples of Template Data FIG. 20 shows an example of a screen displayed based on another template data. The template data in the screen example of FIG. 20A is template data corresponding to the user A. This template data defines that wallpaper data having a pattern as shown in FIG. In this way, template data used for each user can be set and displayed for each user.

  The template data in the screen example of FIG. 20B is template data corresponding to the user B. This template data defines that wallpaper data having a pattern as shown in FIG.

(3) Other examples of template data FIG. 21 shows an example of a screen displayed based on another template data. The template data of the screen example in FIG. 21 can be used by selecting one of them in the terminal by describing a plurality of layout data (skins) in the template data. Also, the selection method can be described in the template data. For example, this figure is an example in which candidates are presented to the user and the skin selected by the user is used.

(4) Example of Change of Template Data FIG. 22 shows an example in which the template data changes and the screen presentation method changes accordingly. In the state shown in the upper part of the figure, since the number of media data to be shared is one, template data corresponding to the media data is used. This template data defines that the other party's camera video is displayed.

  In the center of FIG. 22, as a result of the change in the number of media to be communicated, shared media data is selected and used as template data for shared communication corresponding to local content (video video) and the camera video of the other party. Has been. This template data defines that local content and the other party's video are displayed at the positions shown in the figure.

  The state in the lower part of FIG. 22 is that the media to be communicated has changed, and the shared media data has become local content 1 (video image), local content 2 (still image), and the other party's camera image. This is the result of using the corresponding one. This template data defines that the local content 1, the local content 2, and the video of the other party are displayed at positions as shown in FIG.

  In this way, the template data to be used changes according to the number and type of media to be communicated, and the same content can be seen by using the same template data.

(5) Example of Template Data Processing FIG. 23 shows an example in which the information processing terminal that has received the template data processes the template data according to the function and processing capability of the own device. The left side of FIG. 23 shows a case where the display of the information processing terminal that has received the template data is a 20-inch television having a resolution of XGA (1024 × 768). The data processing unit 303 in the information processing terminal specifies that the received template data defines that the local content and the other party's camera video are to be displayed. Then, the data processing unit 303 determines that these can be displayed simultaneously on the display of the own terminal, stores the received template data in the template storage unit 304, and displays the received template data as the received template data.

  On the other hand, the right side of FIG. 23 shows a case where the display of the information processing terminal that has received the template data is a liquid crystal TV with a resolution of VGA (640 × 480). The data processing unit in the information processing terminal specifies that the received template data is determined to display local content and the other party's camera video. Then, the data processing unit 303 determines that these cannot be displayed simultaneously on the display of its own terminal, and processes the received template data so that only the display of the local content is determined. Is stored in the template storage unit 304, and the processed template data is displayed on the display 317.

(6) Another Example of Rule Set FIG. 24 is another example of the rule set. This rule set defines which of the media X and the media Y is displayed with priority.

  As shown in the figure, the rule 21 is applied when the condition “the number of media is 2 and the resolution of the media X> the resolution of the media Y” is satisfied, and the processing change information indicates that “media X is displayed in full screen”. & Media Y is displayed in the lower left corner by 1/4. That is, the medium X has priority.

  The rule 22 is applied when the condition “the number of media is 2, the resolution of the media X = the resolution of the media Y, and the codec of the media X> the codec of the media Y” is satisfied. “Full screen display & media Y is displayed at 1/4 in the lower left corner”. That is, the medium X has priority. Here, the codec means MPEG2, H.264. H.264, MPEG4, and other media data encoding methods. In this example, it is assumed that the order relationship of “MPEG2> H.264> MPEG4> other codecs” is established.

  Rule 23 indicates that the condition “the number of media is 2, the resolution of media X = the resolution of media Y, the codec of media X = the codec of media Y, and the bit rate of media X> the bit rate of media Y” is satisfied. And the processing change information becomes “Media X is displayed in full screen & Media Y is displayed in 1/4 in the lower left corner”. That is, the medium X has priority.

  Rule 24 indicates that the condition “the number of media is 2, the resolution of media X = the resolution of media Y, the codec of media X = the codec of media Y, and the bit rate of media X = the bit rate of media Y” is satisfied. And the processing change information is “display media X on left half & display media Y on right half”. That is, the media X and the media Y are treated equally.

(7) Another Example of Rule Set FIG. 25 shows an example of a screen displayed according to another rule. Applied when the condition “number of media is 3” is satisfied, the processing change information is “randomly select two media, display one of the selected media (video 1) in full screen, and the other (video 2) is displayed in the lower right corner at 1/9, a dialog box is displayed, and the selection of whether or not to display other media (video 3) is accepted.

(8) Another example of rule set The rule “video data priority” may be used. According to this rule, video data is reduced and displayed at the same ratio so that all of the video data of the shared media data can be displayed, and other shared media data is displayed in the margin area.

  Similarly, the rule “content priority” may be used. In this rule, local content and remote content are displayed with priority, and other shared media data is displayed in the margin area.

(9) Other specific examples of template data In order to control the presentation of audio media data with template data, for example, <c_type> in the <content_list> field is set to sound, <c_source> is set to local or network_session, <c_action> is SPEAKER, <tone> in the <layout> field is used to specify the tone, and <volume> is used to specify the volume.

(10) Other specific examples of template data In order to control the display of character media data with template data, for example, <c_type> in the <content_list> field is set to font, and <c_source> is set to the character content to be displayed. <c_action> is set to DISLAY, <type> in the <layout> field specifies the font type, <position> specifies the position, and <size> specifies the character size.

  To control the display of graphic media data with template data, for example, <c_type> in the <content_list> field is set to figure, <c_source> is set to the graphic type (straight line, circle, polygon, etc.), and <c_action > Is set to DISLAY, the position may be specified by <position> in the <layout> field, and the size of the figure may be specified by <size>.

  An information terminal device using such template data displays media data of characters and figures using means (such as Draw software) for drawing characters and figures in the device itself.

  In this way, by allowing characters and figures to be drawn on the terminal or the partner terminal, the following becomes possible. For example, by describing a graphic representing the pointer in the template data, the pointer (representing the focus position) can be displayed on the displayed video or still image by means of drawing characters and graphics.

  (11) In S706, when the response message is other than “200 OK” response, for example, “603 Decline”, the display 317 indicates that the videophone session (SE1) could not be established. However, the present invention is not limited to this. For example, when it is unknown whether a session can be established from a response message, the reception of the next response message may be awaited. Further, when the user presses the end button 402 of the remote control 302, the communication negotiation unit 305 may transmit a message for canceling the calling process.

(12) Terminal device of transmission source of template data, local content, etc. In the embodiment of the present invention, an information terminal device on the calling side of a videophone calls template communication, transmission of template data, transmission of local content communication. Although the call and the transmission of the local content are performed, the present invention is not limited to this, and the information terminal device on the called side of the videophone may perform the call.

(13) Proper use of template data and rules In the embodiment of the present invention, when template data is held, output of media data is controlled using template data, and when template data is not held, Although the output of media data is controlled using a rule, the present invention is not limited to this. For example, the output of media data may be controlled using only template data or only rules if the information terminal device to be used is commonly used.

(14) Local content, remote content, and template data Local content and remote content are not limited to video and still images, but may be music or web pages. The template data may include an avatar or BGM.

(15) Selection of template data Conditions for selecting template data include the number of media data, the type of media data, the codec of the media data, the size of the image when the media data is a still image or video, and the media It is not limited to all combinations of data sources, and may be at least one of these.

(16) Template selection table The template selection table shown in FIG. 5 includes items of communication partners, and this item is also checked in the same manner as other items to check whether the template data to be selected The number is specified, but is not limited thereto. As a template selection table is provided for each information terminal device, when selecting template data, a template selection table for a communication partner is selected, and whether or not other items match using the selected template selection table. The template data may be selected by the above.

(17) Request for transmission of template data In S707 of the flowchart of FIG. 10, the description is omitted when there is no template data corresponding to the template selection table and the template data cannot be selected.

  In other words, after the session for template communication is established, the number of shared media data, the type of each media data, the codec, the size in the case of an image, and the source, which are template data selection conditions, are communicated The data is transmitted to the terminal device on the other side, and the terminal device on the communication partner side identifies the template data that satisfies the received selection condition, and returns the identified template data. The terminal device that has received the template data can set the template data as use template data.

  The same applies to the case where the template data selected in S708 is not in the template storage unit 304 and the corresponding rule is not owned in S709. The steps S805, S806, and S814 in the flowchart of FIG.

(18) Transmission of template data to communication partner In the embodiment of the present invention, for example, by S715 to S719 of the flowchart of FIG. Although template data was transmitted, it is not limited to this. If the information terminal device 3 on the other side also has the same template data, shared media information, and template selection table owned by the information terminal device 2, the information transmitted from the information terminal device 2 to the information terminal device 3 The same template data as the template data for the terminal device 3 can be set as its own use template data.

  The embodiment disclosed this time should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

It is a figure which shows the structure of the multimedia data sharing system which concerns on embodiment of this invention. It is a figure which shows the communication function in the multimedia data sharing system which concerns on embodiment of this invention. 3 is a functional block diagram showing a configuration of an information terminal device 2. FIG. It is a figure which shows the example of shared media information. It is a figure which shows the example of a template selection table. It is a figure which shows the example of a rule set. It is a figure which shows the detailed structure of the remote control. It is a figure which shows an example of the communication sequence of a multimedia data sharing system. It is a figure which shows the example of the screen displayed in the communication sequence of a multimedia data sharing system. It is a flowchart which shows the procedure of the process at the time of the videophone call. It is a flowchart which shows the procedure of the process which switches communication content during the telephone call of a videophone. It is a flowchart which shows the procedure of the process of the terminal of the incoming call side of a video telephone. (A) is a diagram illustrating an example of an INVITE message 501, (b) is a diagram illustrating an example of an “INVITE” message 502, and (c) is an example of a “200 OK” response message 503. FIG. 6D is a diagram illustrating an example of a “200 OK” response message 504. (A) is a diagram illustrating an example of an INVITE message 505, and (b) is a diagram illustrating an example of a “200 OK” response message 506. It is a figure which shows an example of the template data which an own apparatus (information terminal device 2) uses. It is a figure which shows an example of the template data which the other party apparatus (information terminal device 3) uses. It is a figure which shows another example of the template data which an own apparatus (information terminal device 2) uses. It is a figure which shows the example of a skin. It is a figure which shows the example of the screen displayed based on a certain template data. It is a figure which shows the example of the screen displayed based on another template data. It is a figure which shows the example of the screen displayed based on another template data. It is a figure which shows the example from which template data changes and the presentation method of a screen changes by it. It is a figure which shows the example which the information processing terminal which received template data processes template data according to the function and processing capability with an own apparatus. It is a figure which shows another example of a rule set. It is a figure which shows the example of the screen displayed by another rule.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 Videophone connection server, 2, 3 Information terminal device, 4 Content server, 301 Main body part, 302 Remote control, 303 Data processing part, 304 Template memory | storage part, 305 Template selection part, 306 Data processing change part, 307 Data processing change information Transmission unit, 308 Data processing change information reception unit, 309 Communication negotiation unit, 310 Data communication unit, 311 Light receiving unit, 312 Camera, 313 Microphone, 315 Light, 316 Speaker, 317 Display, 401 Speak button, 402 End button, 403 Input button, 404 operation button, 405 share button, 406 control button, 407 switching button.

Claims (16)

An information terminal device sharing a media data presentation method shared with a communication partner device,
First storage means for storing a plurality of media control information defining media data to be presented and a method for presenting the media data to be presented;
A second storage means for storing a table that defines the correspondence between the number of media data to be shared with the communication counterpart device and / or its characteristics and the media control information for the own terminal;
When switching media data to be shared with a communication partner side device, the table corresponding to the number of media data to be shared after switching and / or its characteristics is referred to by referring to the table in the second storage means. A presentation method determining means for identifying the media control information for the terminal and reading the identified media control information from the first storage means;
An information terminal device comprising presentation means for presenting media data to be shared after switching based on the read media control information. The table in the second storage means further defines the correspondence between the number and / or characteristics of the media data and the media control information for the device on the communication partner side,
When the shared media data is switched, the presenting method determining unit further refers to the table in the second storage unit and corresponds to the number of media data shared after the switching and / or the characteristics thereof. Identifying the media control information for the communication counterpart device to read out the identified media control information from the first storage means,
The information terminal device further includes:
2. The information terminal device according to claim 1, further comprising means for transmitting the read media control information to the communication counterpart device.   The shared media data characteristic in the table is defined by at least one of a media data type, a media data codec method, a media data size, a media data bit rate, and a media source. 2. The information terminal device according to 2.   The media control information for the own terminal and the media control information for the device on the communication partner side are the media data received by the device using the device from the device on the communication partner side over the network, and the device using the media control information The information terminal device according to claim 3, comprising information defining presentation of media data acquired by the photographing means.   The information terminal device according to claim 4, wherein the media control information for the own terminal and the media control information for the device on the communication partner side include information for determining a presentation timing of the media data to be presented.   6. The information terminal device according to claim 5, wherein the media control information for the own terminal and the media control information for the communication partner side device include information defining a layout of the media data to be presented.   6. The information terminal device according to claim 5, wherein the media control information for the own terminal and the media control information for the communication partner side device include information for determining a volume and a tone of the media data to be presented.   4. The media control information for the own terminal and the media control information for the device on the communication partner side include information defining presentation of media data stored in storage means of a device that uses the media control information. Information terminal device.   The media control information for the own terminal and the media control information for the device on the communication partner side are the media data stored in a device other than the device on the communication partner side connected to the terminal using the network via the network. The information terminal device according to claim 3, comprising information defining presentation.   The information terminal device according to claim 3, wherein the media control information includes information defining presentation of character data and / or graphic data generated by a drawing unit of a terminal that uses the media control information. The second storage means stores the table for each of a plurality of devices connected to the device and the network,
The presentation information determining means selects a table corresponding to the communication partner side device, refers to the selected table, and refers to the media control information for the own terminal and the media control information for the communication partner side device. The information terminal device according to claim 2, wherein the information terminal device is specified. The presenting method determining means, when the media control information for the terminal corresponding to the number and / or characteristics of the media data shared with the communication partner side device is not defined in the table, or When the specified media control information does not exist in the first storage means,
A media control information distribution request is transmitted to the communication counterpart device together with the number and / or characteristics of media data shared after the switching, and the media control information transmitted from the communication counterpart device is received.
The information terminal device according to claim 1, wherein the presenting unit presents media data to be shared after the switching based on the received media control information. An information terminal device sharing a media data presentation method shared with a communication partner device,
Means for receiving media control information defining media data to be presented from a communication counterpart device and a method for presenting media data to be presented;
An information terminal device comprising: means for presenting the shared media data based on the received media control information. The information terminal device
Furthermore, it comprises means for processing the received media control information according to the function and processing capability of its own device,
The information terminal device according to claim 13, wherein the presenting means presents the shared media data based on the processed media control information. First storage means for storing a plurality of media control information defining media data to be presented and a method for presenting the media data;
Stores a table that defines the correspondence between the number and / or characteristics of media data shared with the communication partner side device, the media control information for the terminal itself, and the media control information of the communication partner side device A shared media data presentation method for sharing a shared media data presentation method between a first information terminal device comprising a second storage means and a second information device,
When the first information terminal device switches the media data shared with the second information terminal device, the number of media data to be shared after the switching by referring to the table in the second storage means And / or identifying the media control information for the terminal corresponding to the characteristic, and reading the identified media control information from the first storage means;
A first information terminal device presenting media data to be shared after switching based on the read media control information;
Media control for the second information terminal device corresponding to the number and / or characteristics of the media data shared after the switching by referring to the table in the second storage means by the first information terminal device Identifying information and reading the identified media control information from the first storage means;
A first information terminal device transmitting the read media control information to a second information terminal device;
A second information terminal device receiving the media control information transmitted from the first information terminal device;
A second information terminal device presenting the shared media data based on the received media control information. In order to share the media data presentation method shared with the device on the communication partner side,
First storage means for storing a plurality of media control information defining media data to be presented and a method for presenting the media data to be presented;
A second storage means for storing a table that defines the correspondence between the number of media data to be shared with the communication counterpart device and / or its characteristics and the media control information for the own terminal;
When switching media data to be shared with a communication partner side device, the table corresponding to the number of media data to be shared after switching and / or its characteristics is referred to by referring to the table in the second storage means. A presentation method determining means for identifying the media control information for the terminal and reading the identified media control information from the first storage means;
A shared media data presentation program for functioning as presentation means for presenting media data to be shared after switching based on the read media control information.

Images