JP4990718B2 - Media stream processing system, media stream processing method, component realization apparatus - Google Patents

Description

  The present invention relates to a media stream processing system, a media stream processing method, and a component realization apparatus for a service provider including a carrier other than a carrier to develop a media service in the next generation network of the carrier.

  When developing a service for processing a media stream in real time, a device such as a media server having a media stream processing function is used. The media stream processing function here refers to a function that outputs one or more media streams in response to one or more inputs of the media stream.

  As a method for developing a service by controlling a special media stream processing function of these media servers, VoiceXML (Voice Extensible Markup Language: Non-Patent Document 1), CCXML (Cal1 Control Extensible Markup Language: Non-Patent Document 2), MSCML Standardized audio media server control languages such as (Media Server Control Markup Language: Non-Patent Document 3) and unique control interfaces are used for each media server. Non-Patent Document 4 describes ParlayX used for connection between communication terminals such as SIP terminals.

  FIG. 6 shows an example of the functional configuration of a conventional media server. In FIG. 6, the media server 501 has a plurality of media stream processing functions 512a to 512c in addition to a plurality of media control functions 511a and 511b for reproducing and recording media. These media stream processing functions 512a to 512c are used from the service scenarios 514a to 514c via the control interface 513.

  The control interface 513 is intended only for controlling the functions of the media server 501 and cannot control the functions of other media servers. Therefore, the media stream processing functions 512a to 512c can be combined but cannot be combined with the media stream processing function of the external media server.

In general, what controls various media (playback, recording, etc.) in addition to the media stream processing function is called a media server. In the following description, only these media server and media stream processing function are used. Both devices are called media servers.
“Voice Extensible Markup Language (VoiceXML) Version 2.0, W3C Recommendation 16 March 2004”, “online”, “search August 13, 2007”, Internet <URL: http://www.w3.org/TR/voicexml20 /> "Voice Browser Call Control: CCXML Version 1.0, W3C Working Draft 19 January 2007", "online", "August 13, 2007 search", Internet <URL: http://www.w3.org/TR/ccxml /> “Media Server Control Markup Language (MSCML) and Protocol”, “online”, “search August 13, 2007”, Internet <URL: http://www.ietf.org/rfc/rfc4722.txt> “ETSI ES 202 391-2 V1.2.1 (2006-12), Open Sercice Access (OSA); Parlay X Web Services; Part 2: Third Party Copy (Parlay X2)”, “online”, “August 2007 Search on the 13th ", Internet <URL: http://webapp.etsi.org/action%5CPU/20061219/es_20239102v010201p.pdf>

  The interface as shown in the above-mentioned non-patent document is for using the media stream processing function on a specific media server device, and means for inputting an external media stream to the media stream processing function, There is no means for outputting the media stream from the processing function to the outside. For this reason, a plurality of media stream processing functions on a plurality of media servers cannot be used in combination.

  For example, media stream processing functions A and B possessed by a certain media server can be used in combination, but A and B and media stream processing function C possessed by another media server can be combined and utilized. There wasn't. Therefore, when developing a media service for processing a media stream, it is necessary to focus on a specific media server and use only the media stream processing function provided on the media server.

  If only the media server selected as the target does not satisfy the media stream processing function to achieve the target service, prepare multiple media server products and modify the media server program directly, or create your own multiple media server It was necessary to develop a program that enables mutual use of media stream processing functions.

  Even in the latter case, it is difficult to add a media server with another media stream processing function when developing a mutual use program considering only the target media server. Connectivity is lacking, and there is no way to link between general-purpose media stream processing functions.

  Accordingly, an object of the present invention is to provide a media stream processing system, a media stream processing method, and a component realization apparatus that can realize a cooperation method between media stream processing functions of a plurality of media servers.

  In order to solve the above-described problems, the present invention provides a network between a plurality of media servers having a function of processing stream data of a plurality of protocols, a terminal connected to a call control network, and the plurality of media servers. A component implementation unit configured to generate a corresponding component processing unit for the media stream processing function of each media server, and to perform a media stream processing process by designating a combination of the component processing units from an application; The component implementation unit includes a media stream processing function table in which a type of media stream processing function corresponding to the component processing unit and an address of a media server that provides the media stream processing function are described in association with each other Remember and In response to a component generation API call request from the application server, a corresponding component processing unit is generated for any one or more of the specified media stream processing functions recorded in the media stream processing function table. Perform session connection between the generated component processing units, and based on the service request from the terminal, using the address of the media server that provides the media stream processing function corresponding to the generated component processing unit, A media stream processing system for performing session connection between a media server and the terminal.

In the present invention, the component realization unit generates the component processing unit and reserves resources of a media server that provides the media stream processing function corresponding to the generated component processing unit.

The present invention relates to a media stream processing method in a media stream processing system including a plurality of media servers having a function of processing stream data of a plurality of protocols, the terminal connected to a call control network, Media stream processing is performed by creating a corresponding component processing unit for the media stream processing function of each media server provided between the media servers and specifying the combination of the component processing units from the application. The media for storing a media stream processing function table in which a type of media stream processing function corresponding to the component processing unit and an address of a media server that provides the media stream processing function are described in association with each other The component realization unit in the stream processing system responds to a specified one or more of the media stream processing functions recorded in the media stream processing function table based on a component generation API call request from the application server. Generate a component processing unit, perform session connection between the generated component processing units, and provide the media stream processing function corresponding to the generated component processing unit based on a service request from the terminal A media stream processing method comprising performing session connection between the media server and the terminal using the address of the media server.

The present invention provides a media stream processing system including a plurality of media servers having a function of processing stream data of a plurality of protocols, and is provided between a terminal connected to a call control network and the plurality of media servers. A component implementation device that generates a corresponding component processing unit for the media stream processing function of each media server and performs a media stream processing process by designating a combination of the component processing units from an application, A media stream processing function table in which a type of media stream processing function corresponding to the component processing unit and an address of a media server that provides the media stream processing function are described in association with each other; Based on the component generation API call request from the server, corresponding component processing units for any one or more of the media stream processing functions recorded in the media stream processing function table are generated and Perform session connection between the generated component processing units, and based on the service request from the terminal, using the address of the media server that provides the media stream processing function corresponding to the generated component processing unit, A component realization apparatus that performs session connection between a media server and the terminal.

  According to the present invention, it is possible to realize a media service that uses a plurality of media stream processing functions in combination. This makes it possible to process a variety of media streams in real time in the media service without adding a function to the media server itself, thereby widening the range of expression and enabling rapid media service development.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 shows a network configuration according to an embodiment of the present invention.
In this embodiment, as a media stream processing method, SIP (Session initiation Protocol) is used as a connection format between components and between terminals, and SOAP (Simple Object Access) is used to execute an API (Application Programming Interface) of a component implementation unit from a service. Protocol).

  SIP is a standard technology used for session control in next-generation networks, and is used for connections between terminals, between terminals and application servers. SOAP is one of the standard formats used when calling a Web service.

  In FIG. 1, 101a, 101b, and 101c are terminals. The terminals 101a, 101b, and 101c can establish a session by the SIP protocol and become SIP terminals. The terminals 101a, 101b, and 101c are connected to the component realization unit 103 via the call control network 102.

  104a and 104b are media servers. The media servers 104a and 104b provide a data streaming service using a plurality of protocols. As a protocol of the media server, VoiceXML, CCXML, MSCML, etc. are assumed. The media servers 104a and 104b include control interfaces 140a and 140b and media stream processing functions 141a and 141b, 141c and 141d. The media servers 104a and 104b are connected to the component implementation unit 103 via media control networks 105a and 105b, respectively.

  Reference numeral 103 denotes a component realization unit. In order to realize cooperation between the media stream processing functions on the plurality of media servers 104a and 104b, the component implementation unit 103 connects the call control network 102 and the media control networks 105a and 105b of the media servers 104a and 104b. Installed at the boundary. The component realization unit 103 generates components 131a, 131b, and 131c corresponding to the media stream processing functions 141a and 141b, 141c, and 141d of the media servers 104a and 104b on the component realization unit 103 by using the API 132. Is published. The component realization unit 103 manages what media stream processing functions exist on the connected media servers 104 a and 104 b by using the media stream processing function table 133 on the component realization unit 103. In this media stream processing function table 133, the type of each media stream processing function and the addresses of the media servers 104a and 104b that provide it are described in association with each other.

  Reference numeral 106 denotes an application server. The application server 106 has an application processing unit 160. The application server 106 is connected to the component realization unit 103 via the information network 107. The application server 106 provides an application that connects the components 131a, 131b, and 131c.

  In response to the component generation API call from the application server 106, a component having the type of media stream processing function specified by the API call is generated, and the specified type of component 131a, 131b, 131c is displayed on the component implementation unit 103. Is generated.

  The generated components 131a, 131b, and 131c have addresses of media stream processing function objects on the corresponding media servers 104a and 104b, and are instructed that an actual media stream flows from this address when a session is connected.

  When an API for connecting a plurality of generated components 131 a to 131 c is called from the application on the application server 106 connected to the component realization unit 103 via the information network 107, the component realization unit 103 A session is established between these components 131a to 131c.

  The API 132 of the component realization unit 103 can use the following three services.

・ Component generation API
This is a command for generating components corresponding to the media stream processing functions 141a and 141b, 141c and 141d of the media servers 104a and 104b connected to the component realization unit 103 and securing the resources of the media server. The media stream processing function designation and its detailed operation designation are given as parameters. It is possible to generate a plurality of components for one media stream processing function.

・ Component destruction API
This is a command for discarding the component generated by the component generation API and releasing the resource on the component realization unit 103 and the resource of the media server secured when generating the component to be discarded.

・ Inter-component connection API
Instruction to establish a session between components. The actual media stream is transmitted and received between the media servers 104a and 104b having the corresponding media stream processing functions 141a and 141b, 141c and 141d of the component.

  The following types are listed as the most basic components in the media stream processing method according to the embodiment of the present invention. These functions are classified into five types of basic media stream processing functions provided in a general media server.

Mixer The mixer component combines multiple media stream inputs and outputs them as a single media stream. For example, in the case of audio, an operation such as outputting a waveform obtained by superimposing all the waveforms of the input stream or selecting a sound with the highest volume is performed by designating an application. There are various methods for video composition (such as a method of duplicating one video with the other translucent video, or a method of projecting the other video in a specific area of one video). The composite method is designated from the media service, and all of them are collectively called a mixer here.

Distributor The distributor component is a function that distributes the input of a single media stream as multiple outputs.

Multiplexer The multiplexer component is a function that performs two types of output, an audio-only stream and an image-only stream, for an input media stream in which audio and video are integrated. At the same time, the audio input from the component that outputs only audio and the video that is input from the component that outputs only video are output to the input side where video and audio are integrated. I do.

・ Conference As used in conventional telephone conferences, the same mixing process as that of the mixer component is performed for all inputs except for the input from the output target component. For video, the method of outputting only the video of the input media stream with the maximum volume for audio to all components, the method of dividing the screen by the number of connected components and combining the video of all input media streams, etc. Among them, the operation specified by the service is performed. In general, the media stream processing function is rarely provided in the form of a mixer or a distributor, and in many cases only the conference function is provided. A mixer can be regarded as a conference with one output target and other outputs invalidated, and a distributor can be regarded as a conference with one input and other inputs invalidated.

Switch The switch component is a function that distributes input and output targets to different components, whereas components connected by normal session control are input and output targets. When the three connected components are set as A, B, and C, respectively, an input to the switch from A is output to B, and an input to the switch from C is output to A.

Filter The filter component is a function that outputs one of the input media streams with the contents of the media stream modified. The filter component generally includes echoes, reverberation, audio rotation, enlargement / reduction, and color correction, which are generally called “media processing”. These filter types are specified by service for each component.

  In order for the application processing unit 160 on the application server 106 to actually perform these controls, the API 132 of the component implementation unit 103 is called through the information network 107.

  FIG. 2 shows a detailed functional configuration example of the component realization unit 103.

  In FIG. 2, the media stream processing function table 133 has a table of all media stream processing functions 141 a to 141 d of all the media servers 104 a and 104 b connected to the component realization unit 103. Here, as a basic media stream processing function, a mixer 311, a distributor 312, a multiplexer 313, a conference 314, a switch 315, filters 316, 317, etc. are included in the table as an example.

  The API 132 includes a component generation API 321, a component discard API 322, an inter-component connection API 323, and the like.

  When the component generation API 321 is called, the media stream processing function can be used by the control interfaces 140a and 140b on the media servers 104a and 104b having the media stream processing functions 141a to 141d corresponding to the specified component type. Processing is performed. After this initialization process, a component is generated on the component implementation unit 103.

  The component destruction API 322 can destroy these generated component objects.

  For connection between the generated component objects, an inter-component connection API 323 is used. In this embodiment, since SIP is assumed as a connection format between components, it is assumed here that an API corresponding to the MakeCall instruction of ParlayX (Non-Patent Document 4) used for connection between SIP terminals is provided. be able to. The inter-component connection API 323 performs connection between components via the call control network 102.

  When all the connections between the components designated by the application processing unit 160 are completed, the service provision preparation is completed, and the terminal 101a to 101c has a SIP URI through which the service is disclosed via the call control network 102. A connection is made to the component.

  The basic component type media stream processing function will be described with reference to FIG. Here, other components connected to the components (corresponding to the components 131a to 131c in FIG. 1) and SIP terminals (corresponding to the terminals 101a to 101c in FIG. 1) are SIP UAs (User Agents) equally connected by SIP. For these reasons, these are collectively referred to simply as SIP UA.

  The mixer C10 combines and connects the SIP UA C11 to SIP UA C13 media streams connected to the mixer C10, and outputs them to the SIPUA C14.

  The distributor C20 distributes the media stream from the SIP UA C21 connected to the distributor C20 to the SIP UA C22 to the SIP UA 24.

  The multiplexer C30 outputs another SIP UAC32 from which the video and audio from the SIP UA C31 are integrated and separates only the video, and further outputs another SIP UA C33 from which only the audio is separated. . Also, the video media stream input from the SIP UA C32 and the audio media stream input from the SIP UA C33 are combined, and the media stream in which the video and audio are integrated is output to the SIP UA C31.

  The conference C40 outputs a composite of the media streams input from the SIP UA C42 and the SIP UA C43 to the SIP UA C41, and similarly, from the SIP UAC 41 and the SIP UA C43 to the SIP UA C42. For the SIP UA C43, a composite of inputs other than the input to be output is output, such as a composite of inputs from the SIP UA C41 and SIP UA C42.

  The switch C50 outputs an input from the SIP UA C51 to the SIP UA C53, and outputs an input from another SIP UA C52 to the SIP UA C51.

  The filter C60 makes various changes to the input media stream from the SIP UA C61, and then outputs it to the SIP UA C62.

  FIG. 4 shows how an actual media stream is transmitted and received for session control between the component and the terminal. Here, as an example, three media servers A40, A41, and A42 (corresponding to the media servers 104a and 104b in FIG. 1) are used, and the video content provided by the media server A42 is combined with a reduced CM video. Service to be output to the SIP terminal A10 (corresponding to the terminals 101a to 101c in FIG. 1).

  In this example, two components are generated on the component realization unit (mixer A20 and filter A21). In addition, SIP UAs (CM video A30 and video content A31) possessed by the SIP terminal A10 and the media server A42 are established as shown in the figure. The video media stream of the CM video A30 is passed to the filter A21 that performs enlargement / reduction, and the reduced video media stream is passed to the mixer A20. A video media stream of the video content A31 is also input to the mixer A20, and a composite of these two video media streams is output to the SIP terminal A10. The actual media stream is a media stream that is passed from the media server A42 to the media server A40 via the media server A41 (CM video), and a media stream that is passed directly from the media server A42 to the media server A40 (video content). There are two flows, and these are combined on the media server A40 and output from the media server A40 to the SIP terminal A10.

  FIG. 5 shows a service example in which service users using remote SIP terminals can perform karaoke by combining four components. Here, it is assumed that subtitle video (video media stream) X20 and karaoke music (audio media stream) X21 of the song designated by the singer are output from different SIP UAs from the media server of the service provider. The video and audio media stream of the karaoke song X21 distributed via the subtitle video X20 and the distributor X31 is handled by the multiplexer X30 as a single media stream in which the video and audio are integrated. The media stream is input to the singer's terminal X10 via the switch X32 and is reproduced on the terminal X10. The reproduced subtitle (lyric) video and the input (video plus sound) sung by the singer along with the karaoke music are output to the conference X33 via the switch X32. The audio media stream of the karaoke music piece X21 distributed by the distributor X31 is also input to the conference X33, and these and the audio input from the listener terminals X11 and X12 are combined and output to the listener terminals X11 and X12. Are reproduced on the terminals X11 and X12.

  Since the input / output format of the components is made common by the present invention, the combination of components can be expressed by a directed graph as shown in FIG. By making it possible to control components with a GUI (graphical user interface) based on such an expression, a service developer can intuitively specify a combination of components.

  As described above, in the embodiment of the present invention, by providing the component realization unit 103, the media stream processing functions of a plurality of media servers can be used in combination. As a method, the media stream processing function of the media server is associated with an abstract functional block called a component for each function, and the service provider combines the media stream functions of multiple media servers by combining these individual components. Making it possible.

  By creating these components and connecting them, the service developer can use the media stream processing functions on multiple media servers in combination without being aware of the difference between the types of actual media servers. It becomes possible. In addition, by using the session control technology used for the connection method between ordinary terminals for the connection method between these components, a special gateway or the like is not required when connecting to the terminal, It is possible to combine other systems that perform media stream processing as one of the components by arranging them during these sessions. A user of a media service uses a service that combines these media stream processing functions by making a session connection to a component address disclosed by the service provider from a terminal (telephone, TV, etc.). Can do.

  The present invention is not limited to the above-described embodiments, and various modifications and applications can be made without departing from the gist of the present invention.

It is a block diagram which shows the function structure of the network of embodiment of this invention. It is a block diagram which shows the function structure of the component implementation part in embodiment of this invention. It is explanatory drawing of the basic component in embodiment of this invention. It is explanatory drawing which shows the connection between a component and a SIP terminal, and the flow of a media stream in embodiment of this invention. It is explanatory drawing which shows the example of a combination of the component in a karaoke service. It is explanatory drawing of the conventional media server.

Explanation of symbols

101a to 101c ... terminal 102 ... call control network 103 ... component realization unit 104a, 104b ... media server 104a ... each media server 105a, 105b ... media control network 106 ... · Application server 107 ··· Information network 131a to 131c · Component 133 · Media stream processing function table 140a and 140b · Control interface 141a to 141d · Media stream processing function 160 ··· Application processing unit

Claims (4)

A plurality of media servers having a function of processing stream data of a plurality of protocols;
A component processing unit that is provided between a terminal connected to a call control network and the plurality of media servers, and that corresponds to the media stream processing function of each media server, is generated from the application. A component implementation unit that performs media stream processing by specifying a combination of processing units ,
The component realization unit
Storing a media stream processing function table in which a type of a media stream processing function corresponding to the component processing unit and an address of a media server providing the media stream processing function are described in association with each other;
Based on the component generation API call request from the application server, generate corresponding component processing units for any one or more of the media stream processing functions recorded in the media stream processing function table, and Make a session connection between the generated component processing units,
Based on a service request from the terminal, a session connection between the media server and the terminal is performed using an address of a media server that provides the media stream processing function corresponding to the generated component processing unit. A featured media stream processing system. The said component realization part produces | generates the said component processing part, The resource of the media server which provides the said media stream processing function corresponding to the said produced | generated component processing part is ensured . Media stream processing system. A media stream processing method in a media stream processing system including a plurality of media servers having a function of processing stream data of a plurality of protocols,
A component processing unit that is provided between a terminal connected to a call control network and the plurality of media servers, and that corresponds to the media stream processing function of each media server, is generated from the application. A media stream processing process is performed by specifying a combination of processing units, and the type of media stream processing function corresponding to the component processing unit and the address of a media server that provides the media stream processing function are described in association with each other. A component realization unit in the media stream processing system for storing the media stream processing function table,
Based on the component generation API call request from the application server, generate corresponding component processing units for any one or more of the media stream processing functions recorded in the media stream processing function table, and Make a session connection between the generated component processing units,
Based on a service request from the terminal, a session connection between the media server and the terminal is performed using an address of a media server that provides the media stream processing function corresponding to the generated component processing unit. A featured media stream processing method. In a media stream processing system including a plurality of media servers having a function of processing stream data of a plurality of protocols, provided between a terminal connected to a call control network and the plurality of media servers, A component realization apparatus that performs a media stream processing process by generating a corresponding component processing unit for a media stream processing function of a media server and specifying a combination of the component processing units from an application,
Storing a media stream processing function table in which a type of a media stream processing function corresponding to the component processing unit and an address of a media server providing the media stream processing function are described in association with each other;
Based on the component generation API call request from the application server, generate corresponding component processing units for any one or more of the media stream processing functions recorded in the media stream processing function table, and Make a session connection between the generated component processing units,
Based on the service request from the terminal, session connection between the media server and the terminal is performed using the address of the media server that provides the media stream processing function corresponding to the generated component processing unit.
A component realization apparatus characterized by that .

Images