WO2005112453A1

WO2005112453A1 - Ip videophone service system

Info

Publication number: WO2005112453A1
Application number: PCT/JP2005/008500
Authority: WO
Inventors: Arnold Siboro; Weeyee Lim
Original assignee: Arnold Siboro; Weeyee Lim
Priority date: 2004-05-14
Filing date: 2005-05-10
Publication date: 2005-11-24
Also published as: JP2005328394A

Abstract

There is provided an IP videophone service system capable of realizing bi-directional service based on non-actual time video which cannot be provided by the conventional telephone. The system includes means for deciding various media such as moving picture clip, still image, characters, and audio distributed by response of a user of the IP video phone service and their distribution method and distributing these media as streaming video to the IP videophone, according to the distribution procedure preset in a distribution server (50) connected to an IP network (10) where the IP videophone service is provided. Thus, the user of the IP videophone can acquirer various information interactively and audio-visually only by the conventional telephone operation and telephone communication.

Description

Specification

IP videophone service system

Technical field

The present invention relates to an IP videophone service system for realizing an interactive service using non-real-time video using an IP network.

Background art

[0002] IP (Internet Protocol) telephone technology has become practical with the spread of always-on broadband in recent years. However, today's IP telephony service has two people simply by using an always-on broadband IP network and new infrastructure, and there are two people! /, Which has been provided by more voice calls and traditional telephones. The service strength that has been developed almost,

The following describes conventional IP telephone technology with reference to FIGS. 3 and 4. FIG.

[0003] A. IP phone technology

The main IP telephone technology used by the IP telephone service is H.323 (International Standard Organization) standard (used in networks where the transmission quality of communication is not guaranteed, such as the Internet and LAN. Based on IP telephony technology based on the ITU-T Recommendation (standard) for multimedia compression, multiplexing, demultiplexing, and terminal control, and based on the Session Initiation Protocol (SIP) specified by the Internet Engineering Task Force (IETF) There is IP phone technology.

[0004] B. IP telephone technology that realizes various services of conventional telephone

1. One-to-one voice calls

Real-time calls can be made between two IP phones or IP phone software connected over an IP network.

2. n-to-n voice communication (conference call)

Using an MCU (Multipoint Control Unit), as shown in Fig. 3 (b), there are two or more IP telephones, realizing simultaneous real-time calls between IP telephone software and using IP telephones. Multi-site conference call is possible. 3. Videophone technology

In addition to these voice calling technologies, you can watch the video of the other party in real time

As a conventional technology that enables IP videophone and IP videoconference, there is a messaging technology. This is a technology that allows a caller to receive a message when the other party cannot answer the phone or is away, so that the other party can play back the message later. Such a videophone device is disclosed in Patent Document 1 for one-to-one communication and in Patent Document 2 for n-to-n communication.

[0005] Further, as a development of these conventional technologies, a message messaging technology for integrating messages transmitted and received via various routes such as telephone, IP telephone, fax, and e-mail and centrally managing the messages has been developed. There is.

FIGS. 3 (a) and 3 (b) show a system configuration diagram of the related art related to the videophone of the present invention described above, and FIG. 4 shows a block diagram of the related art.

In FIG. 3A, IP videophones 1 and 2 are connected to the IP network 10 on a one-to-one basis, and communicate with each other under the control of a gatekeeper 20 described later. In FIG. 3 (b), a plurality of IP videophones l to n are connected to the IP network 10, and communication is possible between the plurality of IP videophones l to n by the gatekeeper 20 and the multipoint connection unit (MCU) 30. And can hold a video conference.

As shown in FIG. 4, an IP videophone 1 (2,..., N) encodes a video signal with a video input unit 11 that inputs a video signal from an imaging unit such as a CCD camera. Encoding means 12, transfer means 13 for transferring a video signal together with audio data to IP network 10, decoding means 15 for decoding a received video signal, and video output means for outputting a video signal to a liquid crystal display or the like. 16 and has. Means for handling audio signals are omitted.

[0007] C. Gatekeeper

The gatekeeper technology is a technology that provides address translation and LAN access control for H.323 terminals and the like. The gatekeeper 20 shown in FIG. 4 manages a registry of devices in the multimedia network. These devices register with the gatekeeper at startup and request the gatekeeper to admit (accept) the call. [0008] The gatekeeper 20 has the following four essential functions.

1. Address translation means 21

Translates H.323ID and E.164 (standard telephone numbers) to endpoint IP addresses

2.Reception control means 22

Admission Control—Controls admission of endpoints to the H.323 network. To achieve this function, the gatekeeper uses:

(1) H.225RAS (Registration, Admission, and Status) message

(2) Admission Request (ARQ, admission request)

(3) Admission Confirm (ACF, admission confirmation)

(4) Admission Reject (ARJ, admission rejection)

3.Band control means 23

Manage endpoint bandwidth requirements. To achieve this function, the gatekeeper uses the following H.225 RAS message:

(1) Bandwidth Request (BRQ)

(2) Bandwidth Confirm (BCF, bandwidth confirmation)

(3) Bandwidth Reject (BRJ)

4.Zone control means 24

The gateway manages the zone for all registered endpoints in the zone. For example, the registration process of the endpoint is controlled.

[0009] There are the following four optional functions.

(1) Call authentication means 25

Using this option, it is possible to restrict access to specific terminals or gateways and to implement time-of-day (ToD) policy access restrictions.

(2) Call management means 26

With this option, the gateway retains information about active calls and uses this information to indicate busy endpoints and redirect calls. (3) Bandwidth management means 27

This option allows the gateway to deny admission when the required bandwidth is not available.

(4) Call control signaling means 28

This option allows the gateway to route call signaling messages between H.323 endpoints using the GKRCS (Gatekeeper-Routed Call Signaling) model.

Alternatively, the endpoints can send H.225 call signaling messages directly to each other.

[0010] D. Multipoint connection device (MCU)

The multipoint connection device 30 uses means (transfer means 31, encoding means 32, decoding means 33, IP telephone command means 34) such as IP video phones l to n to connect to the IP video phones l to n. In addition to the multi-point control means 35 and the multi-point processing means 36 necessary for conducting a telephone conference. The multipoint control means 35 negotiates (handshake) with each IP videophone participating in the conference call, determines a common function, and controls conference resources such as multicast. The multipoint processing means 36 receives actual audio and video data from each IP videophone participating in the telephone conference and performs multiplexing and the like.

[0011] E. Multimedia content

There are conventional technologies such as SMIL and XTL as conventional technologies related to a video description language composed of various media in combination. There is also a conventional technology called MPEG, in which the description of video and various media themselves are integrated. These are described below.

[0012] 1. SMIL (Synchronized Multimedia Integration Language)

SMIL (a language that handles multimedia files in an integrated manner) is a language that controls and synchronizes the playback of various types of data, such as moving images, still images, audio, music, and characters, and is described in XML. It was recommended in April 1998 as a W3C standard. You can control what position, when, and how long to display. A product that implements this is RealNetworks' RealSystem G2.

[0013] 2. XTL XTL is a runtime description language that uses Microsoft (trademark), DirectShow (trademark), and Editing Services (DES), and is described in XML, similar to SMIL. DES uses the core technology of DirectShow, an application programming interface (API), to enable nonlinear video editing and playback of various media formats. Therefore, it can be said that it is basically similar to SMIL.

[0014] 3. MPEG4

MPEG4 is a multimedia standard that can handle media such as 3D objects, sprites, texts, etc. that can only be used with audio and video. MPEG4 includes 3D objects and animation objects in addition to audio video! This constitutes each scene of MPEG4.

To describe an MPEG scene, a framework called XMT (Extensible MPEG-4 Textual format) is used. XMT is based on SMIL, a content description language that has already been standardized!

Patent Document 1: JP-A-2002-140162

Patent Document 2: JP 2004-48551 A

Disclosure of the invention

Problems to be solved by the invention

[0016] Infonetics Research reports that the global IP phone equipment market has increased by 31% in the last three months of 2003 and is forecast to triple by 2007. On the other hand, according to the Yano Research Institute, “Survey Results on IP Telephone Fields” (June 2003), the market size of IP telephone-related equipment was 38.8 billion yen at the end of 2002 and the largest at the end of 2007. It is expected to be 128 billion yen.

[0017] The above-mentioned market directions and forecasts are likely to be bright prospects for the IP phone market in the future. According to the research institute, the problem with the IP phone market is that related equipment is merely a replacement business for conventional legacy systems. Without comprehensive proposals such as integration with various applications and integration including communication systems, price competition will occur. In addition, the report states that the service has become a service that was naturally required as an ISP, and that the service does not make a profit beyond the rise in access charges and capital investment. [0018] Right now, the situation is as described above. From the user's point of view, in most cases, the only motivation to use an IP phone is its cheapness. The major factor in this is that the services offered by IP phones are almost evolved from those offered by conventional phones. The services provided by current IP phones, such as voice calls, videophone calls, and voice messages, have been available for at least 10 years, and many of them had been active before the development of the IP network (Internet) in 1973. After all, the new part of IP telephony is the always-on broadband IP network and the power of only infrastructure! The new infrastructure that users perceive is only effective at low call rates. Therefore, the merit of introducing IP phones for users is very little other than this low price!

[0019] For the industry, this situation means price competition, and as a result, only major companies can continue to provide this IP telephone business. IP telephony services are truly unprofitable services and are used by large, well-funded companies as a means of attracting customers for other profitable services.

To break out of this brute force, a completely new service must be created. The needs of the industry are driving new, more profitable businesses than ever before.

Creating and deploying IP telephone services. This will enable users to find the motivation for introducing IP phones in addition to the price, create a sound business model in the Japanese economy, and make it possible for IP phones to penetrate various parts of their lives.

[0020] The e-Japan plan enacted an IT policy. The IT industry has undergone major changes, and the spread of broadband (high-speed, large-capacity) has progressed at a rapid pace. The government has invested money and effort, and the industry has also competed, bringing widespread adoption of broadband, such as ADSL, now with the unthinkable introduction and price in 2000, when the e-Japan plan was enacted. According to the latest Ministry of Internal Affairs and Communications report, the cumulative number of services provided by ADSL in January 2004 was 10,612,671, a 1.7-fold increase from one year ago. According to a telecommunications monitor survey compiled by the Ministry of Internal Affairs and Communications, the number of users connecting to the Internet using ADSL reached 41.7% in September 2003, and the number of users connecting via the fixed-line telephone line was "35.0%" for dial-up. Optical fiber connection accounts for 4.0%, confirming the spread of high-speed network connection. However, by age, 48.8% of users aged 60 and over use “dial-up”. By region, 50.7% of Kyushu users were dial-up users. Depending on the region and age group, many users do not have an environment where they can use video services that utilize broadband. Although the Internet is widespread, the information gap between young people and metropolitan areas still remains. The reality is that almost everyone who can freely use computers can benefit from the spread of broadband.

[0022] It is said that killer content is necessary for the further spread of broadband. There are more situations than this, and killer applications that carry killer content are considered indispensable. The killer application must be something that anyone can treat as everyday. Researching and developing such applications will be the role of the industry in the future and will be the key to growth.

[0023] In the conventional IP telephone technology, the IP telephone technology focuses on the real-time communication of the conventional telephone, and the technology for supporting the non-real-time interactive service is very insufficient. It is only used as a substitute for the network (PSTN). Therefore, it can provide only the services that were conventionally provided by telephone (such as one-to-one voice or television calls or conference calls, and message services).

[0024] In addition, since the conventional multimedia content technology has complicated processing on the client side, services using the multimedia content technology require advanced clients, and require users to have some advanced knowledge. There is a problem that there is. As a result, conventional multimedia content technology cannot provide content to a wide audience.

[0025] An object of the present invention is to provide an IP videophone service system capable of realizing a bidirectional service based on non-real-time video, which has not been able to be provided by conventional telephones, based on the conventional accumulation technology.

Means for solving the problem

[0026] In a first configuration of the present invention for solving the above-mentioned problem, a distribution server connected to an IP network provided with an IP videophone service performs an IP videophone call according to a preset distribution procedure. Video clips, still images, It is characterized by having means for determining various media such as characters and voices and the distribution method thereof, and distributing these media as streaming video to IP videophones. According to the first configuration, the user of the IP videophone can obtain various information interactively and audiovisually by the conventional telephone operation and telephone call.

[0027] A second configuration of the present invention is characterized in that the distribution server includes means for distributing only recorded media or a combination of recorded media and live broadcast media. I do.

With this second configuration, it is possible to distribute not only the recorded media but also a combination with a live broadcast media as needed.

[0028] A third configuration of the present invention is characterized in that the distribution procedure describes behaviors such as a distribution time, a distribution time, a position on an IP videophone screen, and a display method of the various media. I do.

With this third configuration, when and for how long a certain media is to be distributed, and in the following procedure, the media is displayed in which position on the IP videophone screen and in what manner of display Can be defined.

In this case, the distribution procedure can be described to determine the behavior of various media in response to any response from the user based on the time at which the response was received.

[0029] In a fourth configuration of the present invention, the distribution server according to the distribution procedure according to the user environment such as an IP television telephone number and a language setting of an IP video telephone service user, distributes the video to be distributed. And means for adjusting the selection and distribution method. With this fourth configuration, it is possible to select the display in a language corresponding to the language used by the IP videophone service user, and to adjust the distribution method depending on whether the user is a beginner or an experienced user. .

[0030] The present invention can also have the following configurations.

(1) In the first configuration, when multiple distribution servers are provided to prepare for a large amount of access, the IP videophone terminal automatically accepts connections when connecting to the distribution server so as not to be overloaded. Decide the distribution server.

(2) In the first to third configurations, the user also speaks the IP video phone terminal power normally In this way, the distribution server receives the user's audio and video, and automatically creates a description of the distribution procedure (also with the distribution procedure manual or scenario ヽぅ) and the respective media.

[0031] In a fifth configuration of the present invention, the distribution server automatically creates a scenario in accordance with an instruction of the IP videophone service user, and receives the voice or video of the IP videophone service user. With the means to automatically create each media! /, Is characterized.

According to the fifth configuration, it is possible to automatically create a scenario and various media using the received voice and video according to the instruction of the IP videophone service user.

[0032] A sixth configuration of the present invention provides an IP videophone terminal for using an IP videophone service, a distribution server that distributes video to the IP videophone terminal, a video clip to be distributed, and a still image. In an IP network composed of a file server for storing various media such as pictures, characters, voices and the like and a cache thereof, and a gatekeeper for performing IP video phone number resolution and bandwidth management, the distribution server is an IP network. Transfer means for transferring data to the IP video telephone terminal, decoding means for decoding data encoded as data of the IP video telephone terminal into audio data, video data, and the like, and transmitting audio data and video data to the IP video telephone terminal. Encoding means that encodes data as data, and IP telephones that issue commands necessary to exchange data with IP videophone terminals A command means is also in the range and a medium access means for inputting and outputting various media stored in the Huai Rusaba.

According to the sixth configuration, the user of the IP videophone can obtain various information interactively and audiovisually by the conventional telephone operation and telephone call.

[0033] In a seventh configuration of the present invention, the file server includes a scenario database that holds a scenario, and the distribution server further processes the scenario and executes the scenario according to a preset distribution procedure. Means, and the distribution procedure describes behaviors such as a distribution time, a distribution time, a position on an IP videophone screen, and a display method of the various media.

In the seventh configuration, the IP telephone number and language setting of the IP telephone service user According to the user's environment, it is possible to adjust the selection of the video to be distributed and the distribution method according to the distribution procedure described above.

[0034] The present invention can further have the following configurations.

(1) In the sixth configuration, the distribution server is provided with multipoint control means and multipoint processing means for converting a plurality of IP phone images into one image, so that only the media recorded on the file server or Once recorded, it is possible to distribute the combined media and live broadcast media.

[0035] (2) In the seventh configuration, the file server includes a scenario database that holds a scenario, and the distribution server further stores a video composed of various media cards stored in the storage device. It is equipped with a content caching means that is stored in a storage device and read out when necessary, and by describing the distribution time, distribution time, position on the IP videophone screen, and display method of various media as distribution procedures, It is possible to specify how long a certain media is to be delivered at the timing, and in the following procedure, at which position on the IP videophone screen and in what display manner the media is displayed. In addition, the distribution procedure can be described so as to determine the behavior of various media based on the time at which the response is received for any response from the user.

(3) According to the sixth configuration, the file server includes a scenario database that holds scenarios, a media database that holds various media such as video clips, still images, characters, and audio, and storage. Various types of media stored in the device A cache database that holds images created by the distribution server is provided.The distribution server has a user response processing unit and a scenario processing unit that processes a scenario and executes the scenario as it is. In addition, as a distribution procedure, for any response from the user, the behavior of the various media is determined based on the time at which the response was received. It is possible to provide services such as

[0037] (4) In addition to the seventh configuration, the gatekeeper has an address conversion unit and a server load management unit, and the distribution server monitors the usage status of the CPU and the memory of the distribution server from time to time. By providing load monitoring means, in a service consisting of multiple distribution networks to prepare for a large amount of access, when an IP videophone connects to a distribution server, A distribution server that automatically accepts connections can be determined so as not to be overloaded. The invention's effect

According to the present invention, when the IP videophone service is provided, the distribution server connected to the IP network distributes the IP videophone service in response to the IP videophone service user according to a preset distribution procedure. Media, such as video clips, still images, text, audio, etc., and methods for distributing them, and a means for distributing these media as streaming video to IP TV phones! Users can interact with audio-visually rich information contents through conventional telephone calls and telephone operations.

In addition, by establishing IP videophone technology that is suitable for non-real-time communication, it is possible to solve the problem of being limited to call services by enabling services outside of calls that have been used up to now.

Brief Description of Drawings

FIG. 1 is a system configuration diagram showing an embodiment of the present invention.

FIG. 2 is a block diagram showing an embodiment of the present invention.

FIG. 3 is a system configuration diagram of a conventional technology.

FIG. 4 is a block diagram of a conventional technique.

FIG. 5 is a system configuration diagram showing a configuration of Embodiment 1 of the present invention.

FIG. 6 is a block diagram illustrating a configuration of a distribution server according to the first embodiment.

FIG. 7 is a block diagram showing a configuration of a part of a distribution server according to the first embodiment.

FIG. 8 is a block diagram showing a configuration of an interaction engine in Embodiment 1.

FIG. 9 is a block diagram showing a configuration of a media engine in Embodiment 1.

FIG. 10 is an explanatory diagram showing a creation state of a dialog scenario in the first embodiment.

FIG. 11 is an explanatory diagram showing a configuration example of a content according to the first embodiment.

FIG. 12a is a list in which the content configuration of FIG. 11 is used as a scenario.

[FIG. 12b] A list in which the content configuration of FIG. 11 is used as a scenario.

FIG. 13 is a system diagram showing a media configuration of the scenario shown in FIG. 12.

FIG. 14 is a list showing description contents of a media scenario.

FIG. 15 is an explanatory diagram of a basic operation in Embodiment 1. FIG. 16 is a time-series flow chart of the operation shown in FIG.

FIG. 17 is an explanatory diagram showing a mechanism of a scenario using a combination of recorded media and live broadcast media.

FIG. 18 is an explanatory diagram showing a processing flow for realizing the scenario of FIG. 17.

FIG. 19 is an image of a screen displayed on a user's IP videophone.

[20] Fig. 20 is an explanatory diagram of a scenario in which delivery is instructed according to the environment of the user in the second embodiment.

[21] FIG. 21 is an explanatory diagram showing interactive and real-time exchanges between a clerk and a customer in the third embodiment.

FIG. 22 is a time chart showing a flow of exchange in Example 3.

FIG. 23 is an image of an exchange screen in Embodiment 3.

FIG. 24 is a block diagram showing a flow in an IP videophone load distribution service in Embodiment 5.

FIG. 25 is an image of a screen of an IP videophone user participation type service according to the sixth embodiment.

1 to n IP videophone

10 IP network

11 Video input means

12 Encoding means

13 Transfer means

14 IP phone command means

15 Decoding means

16 Video output means

20 Gatekeeper

21 Address translation means

22 Reception control means

23 Bandwidth control means Zone control means Call authentication means

Call management means

Bandwidth management

Call control signaling means distribution server

Transfer means

Decoding means

Encoding means

User response processing means

IP telephone command means Multipoint processing means Multipoint control means Scenario processing means Medium access means File server

Scenario DB

Media DB

Cache DB

Content caching means Server load monitoring means

HTTPZFTP command means User authentication means Access transfer means Server load management means Load status DB

Network load monitoring means

-Audio codec 102 bidet codec

103 audio channels

104 video channels

105 audio input / output device

106 Video input / output device

107 Media Engine

108 Interaction Engine

109 audio buffer

110 bidet buffer

111 H.323 end point

112

113 Scenario DB

114 Session Information DB

115 Media DB

116 Clip DB

117 cache

120 Interaction Engine Coff

121 Local Media Access

122 Remote Media Access

123 备 Recognition Enon ^: N

124 DTMF

125 Riko

Say

126 videos

131 Media Interpreter

132 Native Media Script

133 XTL

134 SMIL

135 MPEG-4 136 Video Processor

BEST MODE FOR CARRYING OUT THE INVENTION

Hereinafter, embodiments of the present invention will be described with reference to a system configuration diagram shown in FIG. 1 and a block diagram shown in FIG.

The embodiment of the present invention includes an IP videophone 1 to n for using the IP videophone service, a distribution server 50 for distributing video to the IP videophone terminal, a video clip to be distributed, a still image, It comprises an IP network 10 composed of a file server 60 for storing various media such as characters and voices and their caches, and a gatekeeper 20 for IP video phone number resolution and bandwidth management.

[0042] The IP videophones l to n include a video input unit 11 for inputting a video signal of an imaging unit such as a CCD camera, an encoding unit 12 for encoding a video signal, and a video signal together with audio data. A transfer means 13 for transferring the video signal to an IP telephone command means 14, a decoding means 15 for decoding a received video signal, and a video output means 16 for outputting a video signal to a liquid crystal display or the like. .

The distribution server 50 includes a transfer unit 51 for transferring data on the IP network, and a decoding unit 52 for decoding data encoded as data of the IP television phones l to n into audio data, video data, and the like. Encoding means 53 for encoding audio data, video data, and the like as data of an IP videophone, and user response processing means 54 for receiving an IP videophone button operation, a user's voice, and the like, and processing them according to a scenario. IP phone command means 55 for issuing commands necessary for communication with the IP videophone, and negotiating (handshake) with each IP videophone participating in the teleconference, determining common functions, and conducting conferences such as multicast It receives multi-point processing means 56 for controlling resources and receives actual audio and video data of each IP video telephone participating in the conference call. Multi-point control means 57 for processing, for example, scenario processing means 58 for processing and executing the scenario as it is, medium access means 59 for inputting and outputting various media stored in each database of the file server 60, Content caching means 71, which stores the various media powers stored in the file server 60 in a database and reads it out when necessary, and a server that monitors the usage status of the CPU and memory of the distribution server 50 from time to time load Monitoring means 72, HTTPZFTP command means 73 for generating commands for transferring various media output from the medium access means 59 to the IP network 10, and user authentication means for authenticating the user of the connected IP television telephone. 74 and access transfer means 75 for transferring the IP videophone connected to the distribution server 50 to another distribution server.

[0044] The file server 60 includes a scenario DB (database) 61 for storing scenarios, a media DB62 for storing various media such as video clips, still images, characters, and audio, and various types of data stored in a storage device. The media is also equipped with a cache DB63 that holds the images created by the processing of the distribution server!

[0045] The gatekeeper 20 manages the address conversion means 21 for converting between an IP address and a telephone number, the reception control means 22 for accepting the connection of the incoming IP videophone, and the bandwidth request of the IP videophone. Bandwidth control means 23, which performs processing for connecting the IP videophone to the gatekeeper 24, call authentication means 25 which performs authentication to restrict access and policy access, and connection between IP videophones A call management means 26 for managing the situation, a bandwidth management means 27 for rejecting admission when the required bandwidth cannot be secured, and a call for controlling signaling for establishing a connection with an incoming IP videophone. Control signaling means 28, server load management means 81 for managing load status information of each distribution server, and CPU and memory usage rates of each distribution server Having a load situation D B82, a network load monitoring means 83 for constantly monitoring the load status of each distribution server.

Examples of the present invention will be described below.

Example 1

Embodiment 1 of the present invention relates to an IP videophone service, and will be described with reference to FIG.

In the present embodiment, the IP videophones l to n for using the IP videophone service, the distribution server 50 for distributing video to the IP videophones l to n, the video clips to be distributed, still images, and text , A file server 60 for storing various media such as audio data and their caches, and a gatekeeper 20 for IP telephone number resolution and bandwidth management. Transfer data Transmitting means 51, decoding means 52 for decoding data encoded as data of the IP videophones l to n into audio data / video data, and the like, and transmitting audio data and video data to the IP videophones l to n. Encoding means 53 for encoding as data, IP phone command means 55 for issuing commands necessary for exchange with IP videophones l to n, and medium for inputting and outputting various media stored in file server 60 It has access means 59.

[0047] The transfer means 51 includes TCP (Transmission Control Protocol) and UDP (User

Data is transferred over the IP network 10 using a protocol such as Datagram Protocol. The data passed to the transfer means 51 can be passed using RTP (ReaH: ime Transport Protocol) or HTTP (Hyper Text Transfer Protocol) technology.

The IP telephone command means 55 can use H.323 or SIP (Session Initiation Protocol) technology. H.323 has been adopted as a protocol for Voice over IP (VoIP) for realizing Internet telephony, as well as integrating real-time multimedia data such as video conferencing between multiple locations. It can handle and multicast communication is also possible. H.261 (64 kbps to 2 Mbps) or H.263 (64 kbps or less), which is the standard based on MPEG-4, is used as a video compression technology, and G.711 and G.711 are used as audio compression technologies. 722, G.723.1, etc. Power used.

[0048] With the configuration of the first embodiment, the user of the IP videophone can obtain various information interactively and audiovisually by the conventional telephone operation and telephone call.

FIG. 5 shows a configuration example of the first embodiment. This will be described with reference to FIG.

The distribution server 50 includes a distribution server 50-1, a distribution server 50-2, and a distribution server 50-3. These distribution servers share the file server 60. IP videophones l to n consist of IP videophone 1, IP videophone 2, and IP videophone 3.

An embodiment of the distribution server 50 will be described with reference to FIGS. FIG. 6 is a diagram for explaining the entire distribution server 50. FIG. 7 is a diagram for explaining the interaction engine and the media engine of the distribution server 50. FIG. 8 is a diagram for explaining the interaction engine. FIG. 9 is a diagram for explaining a media engine.

First, the entire distribution server 50 will be described with reference to FIG. Decoding means 52 and encoding means 53 are realized as audio codec 101, video codec 102, audio channel 103, video channel 104, audio input / output device 105, and video input / output device 106 for video and audio data, respectively. ing.

The scenario processing means 58 and the user response processing means 54 are realized as a media engine 107 and an interaction engine 108. The audio input / output device 105 acquires data from the audio buffer 109, and the video input / output device 106 acquires data from the video buffer 110, respectively, of audio ′ video to be delivered to the IP telephones l to n.

The IP telephone command means 55 is realized by the H.323 endpoint I 1 and the H.323 connection 112.

When an incoming call is generated from the IP videophones l to n to the distribution server 50, the H.323 endpoint 111 receives the incoming call, establishes an H.323 connection 112, and, based on the scenario, an interaction engine 108 and a media engine 107. Have the video (video and audio) created. The video input / output device 106 acquires the created video from the video buffer 110, and the audio input / output device 105 acquires the created audio from the audio buffer 109, and distributes them to the IP videophones l to n through the H.323 connection 112. .

Next, the interaction engine 108 and the media engine 107 of the distribution server 50 will be described with reference to FIG.

The interaction engine 108 performs a process for dialogue with the user based on the scenario DB 113, and causes the media engine 107 to create a video and an audio corresponding thereto. The interaction engine 108 reads the scenario from the scenario DB 113 and processes the scenario, and stores the connection and conversation status with the IP telephones 1 to n in the session information DB 114.

The media engine 107 creates a video 'audio' according to the instruction of the interaction engine 108, and distributes it to the IP videophones l to n. The video and audio are stored in the media DB 115, the clip DB 116, and the cache 117.

Next, the interaction engine will be described with reference to FIG.

The interaction engine 108 performs a dialog processing centering on the interaction engine core 120. The interaction engine core 120 uses a local media access 121 for a scenario stored on a local disk, and accesses a scenario stored on a remote disk using a remote media access 122. The accessed scenario is processed, and the response recognition engine 123 recognizes various information such as the user environment and the user response transferred from the IP videophones l to n based on the scenario. The response recognition engine 123 recognizes DTMF (numeric input from a videophone button) 124, voice 125, and video 126. The interaction engine core 120 determines a video / audio to be created based on the recognition result based on the scenario, and notifies the media engine 107 of the determined video / audio.

Next, the media engine will be described with reference to FIG.

The media engine 107 receives the notification from the interaction engine 108 and creates media (video). The notification includes a file name including a description of video and audio constituting the video to be created, and the media engine 107 reads the file, passes the file through the media interpreter 131, and decrypts the file. This description file may be in a native format (native media script 132), or in a format such as XTL133, SMIL134, or MPEG-4135. A media interpreter corresponding to each format is prepared.

The media engine 107 causes the video processor 136 to create a video based on the decoding result, and distributes the video to the IP videophones l to n.

The distribution procedure manual will be described with reference to FIG.

The distribution procedure manual is a description of the behavior of various media, such as distribution time, distribution time, position on the IP videophone screen, and display method, and is also called a scenario.

Scenario DB113 consists of two parts: one for dialogue and one for creating video. A scenario for dialogue is called a dialogue scenario, and a scenario for creating video is called a media scenario.

The media scenario describes how to create a media DB (video) 115 to be created. The clip DB 116 is video or audio data itself having a single content. As shown in Figure 10, the interaction scenario references a media scenario. Dialogue scenarios combine multiple media scenarios to build advanced interactive multimedia content. The media scenario refers to the clip DB 116. Media scenarios combine multiple clips to create complex video and audio.

Also, as shown in Figure 10, one dialog scenario can refer to another dialog scenario. Unreferenced conversation scenarios are called layer 1 conversation scenarios. The layer of the dialog scenario referred to by this dialog scenario is Layer 2. In this way, the layer numbers of the dialog scenario are assigned. The layer number is a relative number.

A scenario for describing specific content will be described with reference to the drawings.

Fig. 11 shows the structure of the content. The content also has four main menu capabilities: “Introduction to Kyushu”, “Introduction to Iizuka”, “Watch New Song”, and “End”. In addition, “Introduction to Kyushu” has nine menus, “Introduction to Iizuka” has five menus, and “Watch new songs” also has five menus! The number before and after each menu item in FIG. 11 is a number selected by the user, and when that number is selected, the menu is executed.

FIG. 12 (FIGS. 12a and 12b) shows the above content configuration as a scenario. Figure 12 includes a dialog scenario and a media scenario, but the actual media scenario content is stored in a separate file. The “PlayMedia ()” part in the scenario is a part for creating and distributing media (video), which is a part of the media scenario. For example, the interaction engine reads "PlayMedianndex.mml", true); When reading J, it passes index.mml and other interaction scenarios to the media engine, which creates media based on it and sends it to the IP videophone. Store in a buffer for distribution. The delivery server delivers this to the decoded UP videophone.

"GetDigitri23456789"); "in the scenario means that the user's response accepts numeric input of 1-9. A scenario corresponding to each number is also written. For example, when a response of "1" is received, "PlayMediarscene010.mml", true); is executed, and a media scenario called sceneOlO.mml is sent to the media engine. It will be.

FIG. 13 shows the media configuration of the scenario shown in FIG. The number is the number accepted as a response from the user. Figure 14 shows the contents of index.mml where the media scenario is described. The media combines each clip to create one video. Specify clip with clip. A clip is located on a track that can be specified multiple times, and a tract is located in a group. There are two groups, one for video and one for audio. group is located in the timeline. The timeline is the time axis of the whole image. In Fig. 14, a video is created by combining a video clip called butterfly.avi, menuindex.bmp, V, and a still image, and this is combined with the audio part of butterfly.avi to create a video.

With reference to FIG. 15, a description will be given of how to obtain various information interactively and audio-visually by a conventional telephone operation and telephone call.

When one of the IP videophones l to n dials any of the distribution servers 501 to 50-n using the buttons of the IP videophone, a reception request is sent from the IP videophone to the gatekeeper 20. (1). After the reception process, the gatekeeper 20 sends a reception completion notice to the IP videophone (2). This reception completion notification includes the IP addresses of the distribution servers 50-1 to 50-n. The IP videophone sends a connection establishment request to this IP address (3). The distribution server establishes a connection (4). Then, the distribution server refers to the scenario DB61 and determines the video to be distributed first (5). It obtains various clips that are the material for creating the video and creates the video (6). Encode the created video for IP videophone (7). Then the video is distributed to the IP videophone (8).

When the connection between the distribution server and the IP videophone is established, the IP videophone user can send a response to the distribution server. For example, when the first video is distributed from the distribution server to the IP videophone (8), when the user of the IP videophone operates the telephone button, the information of the pressed button is sent to the distribution server (9). ). The scenario processing means 58 having received the information refers to the scenario DB 61 and determines an operation in response to the user's response. In other words, what kind of video should be delivered in response to the user's response?

When the operation shown in FIG. 15 is applied to the scenario in FIG. 12, the distribution server first

The video created based on index.mml is distributed to IP videophones. Then, when the user of the IP television phone presses 1 with the telephone button, the information of “1” is sent to the distribution server, recognized by the user response processing means 54, and sceneOll.mml is created based on the dialog scenario. The media scenario is read and a video is created based on the media scenario and distributed to the IP TV phone. In this way, the video sent in response to the user's instantaneous response changes.

FIG. 16 shows the operation shown in FIG. 15 in chronological order. The left side is the time axis of the server, and the right side is the time axis of the user. The time lapse is from top to bottom. By repeating this operation in which an appropriate video is sent from Sano in response to the user's response, interactive and audio-visual exchanges are realized.

In the first embodiment, by using the multi-point processing means 56 and the multi-point control means 57 provided in the distribution server 50, not only the media recorded in the file server 60 but also the recorded media can be used. Combined distribution with live broadcast media is possible.

This will be described with reference to the drawings. Figure 17 shows the mechanism of a scenario that uses a combination of recorded media and live broadcast media. Figure 19 shows a screen image displayed on the user's IP videophone.

As shown in FIG. 17, the scenario in this case is similar to the scenario shown in FIG. 10, except that the powerful clip is connected to a distribution server that is not the video or audio that actually exists on the disc. Clips obtained directly from the phone have different characteristics.

As shown in FIG. 18, when the user accesses this scenario (1), the distribution server acquires the scenario as usual (2), processes it, and reads various clips (2). If the live media is included in the scenario, the distribution server is designated and sends a call to the IP videophone (this IP videophone is called the service provider's IP videophone), and the IP videophone Initiate a connection. Power of the user's IP videophone The video that is sent is distributed to the service provider's IP videophone [4], and the distribution server uses this video as a clip required by the scenario [5]. An image is created with this clip and distributed to IP videophone users [6].

As shown in Fig. 18, the video of the service provider's IP videophone is placed on the user's IP videophone screen along with other clips on the disk according to the scenario. Example 2

The second embodiment of the present invention relates to an IP videophone user-oriented service. This service has, in addition to the basic configuration of the first embodiment, a scenario DB 61 for storing a scenario in the file server 60, and a scenario processing in which the distribution server 50 further processes the scenario and executes the scenario according to a preset distribution procedure. Means 58 are provided.

The distribution procedure describes the distribution time, distribution time, position on the IP videophone screen, and display method of various media.

As a result, the distribution procedure is followed according to the user environment such as the IP phone number and language setting of the IP phone service user!映像 You can select the video to distribute and adjust the distribution method.

This will be described with reference to the drawings.

Figure 20 shows a scenario that instructs distribution according to the user's environment. Consider the case where a certain shopping content sends different images to users who transmit from two regional powers. Users from region 1 distribute product information limited to region 1, and users from region 2 distribute product information limited to region 2. However, some territories are common throughout the region. Also, since each region uses two languages, Language 1 and Language 2, it is necessary to distribute audio that corresponds to the language of the user who sends audio even if the video is the same depending on the user's language. Has become. This mechanism is shown in the scenario configuration shown in Figure 20.

A specific operation will be described with reference to FIG. For example, when a user who speaks language 1 from region 1 sends a call, the region 1 media scenario is used according to the description of the dialogue scenario (1). Furthermore, according to the media scenario of region 1, video clips are used in region 1 (common) (2), but audio clips (audio parts) are in language 1 (3).

Example 3 of the present invention relates to an IP videophone conversation service. In this service, in addition to the basic configuration of the first embodiment, a file server 60 includes a scenario DB 61 for storing scenarios and media for storing various media such as video clips, still images, characters, and audio. The distribution server 50 includes a user DB 62 and a cache DB 63 that retains various types of media stored in the storage device and images that are obtained by the processing of the distribution server. This has a scenario processing means 58 to be executed. By this, it is possible to determine at what timing and for how long a certain media is to be distributed, and in the following procedure, at what position on the IP videophone screen and what display The ability to display the media in different ways. As a delivery procedure, a description is made for any response of the user to determine the behavior of the various media based on the time at which the response was received.

As a result, it is possible to provide the user with a service in which the user is interacting in real time.

[0065] This will be described with reference to the drawings.

Figure 21 shows the interaction between the clerk and the customer in real time.

Realizing such a real-time dialogue with a non-real-time clip collection is the exchange of video between the distribution server and the user shown in FIG. The distribution server distributes different images one after another according to the user's intention.

Figure 23 shows an image of the screen in Figure 22. As shown in this screen image, there is a background video 'audio', a menu video showing options and so on. Also, the person (clerk) in the picture behaves like a person on the other side, but is actually a video clip recorded on a disc. Each of these is a collection of clips, and the attributes of each clip are controlled by the scenario!

These videos are created based on scenarios that determine the selection of various clips, display start time, display period, display method (effects, etc.), display position on the screen, etc., so that users can interact in real time. Let them experience.

Example 4

Embodiment 4 of the present invention relates to an IP videophone user-oriented service. In addition to the basic configuration of the second embodiment, this service is a content cache that stores videos composed of various media stored in the storage device in the storage device and reads them when necessary. By providing the distribution time, distribution time, position on the IP telephone screen, and display method of various media as a distribution procedure, a certain media can be distributed at which timing and for how long. In addition, the following procedure can be used to specify at which position on the IP videophone screen the media should be displayed in the following manner. In addition, the distribution procedure can be described so as to determine the behavior of various media based on the time at which any response from the user is received.

Example 5

Embodiment 5 of the present invention relates to an IP videophone load distribution service. In this service, in addition to the configuration of the second embodiment, the gatekeeper 20 has an address translation unit 21 and a server load management unit 81, and the distribution server 50 manages the usage status of the CPU and the memory of the distribution server. The provision of the server load monitoring means 72 for monitoring from time to time enables the IP videophones l to n to be overloaded when connected to the distribution server 50 in a service composed of a plurality of distribution servers 50 to prepare for a large amount of access. It is possible to determine the distribution server 50 that automatically accepts the connection so as not to be out of order.

[0068] This will be described with reference to the drawings.

FIG. 24 shows load distribution of the distribution server using the gatekeeper 20. With IP videophone 1 connected to distribution server 50-1 and receiving services, IP videophone 2 makes a request to gatekeeper 20 to start connection (1). The gatekeeper 20 receiving the reception request checks the load status 82 of each server by the server load management means 81 (2). The IP address of the server that will not be overloaded by providing service to this IP videophone 2 is notified (3). Here, since it is determined that the distribution server 50-1 cannot provide a service to a new IP videophone without being overloaded, the gatekeeper 20 determines the IP address of the distribution server 50-2 that has room. Notify (5). With this, the IP videophone accesses the distribution server 50-2 as usual and the video is sent.

Example 6

Embodiment 6 of the present invention relates to an IP videophone user participation type service. In this service, a user response processing means 54 is provided in the force distribution server 50, which can be realized by any of the configurations of the first to fifth embodiments, so that the user can speak normally from the IP videophones l to n. Thus, the distribution server 50 has a function of receiving the user's audio and video and automatically creating an IP television telephone scenario and respective media. As shown in Fig. 15, when a user of an IP videophone makes a call to the distribution server and accesses a scenario for automatic scenario generation by the user, a screen as shown in Fig. 25 is displayed on the user's IP videophone. Is done. In this screen, the live video of the user of the IP videophone is displayed together with the video created by the scenario as usual. The user follows the guidance of the guide shown in the video and speaks self-introduction naturally as if talking to a human on the telephone. The video of the IP videophone user is sent to the distribution server and stored as audio and video clips. Then, the scenario is generated according to the scenario for automatic generation. When any IP videophone user accesses this scenario, the content is played back.

Industrial applicability

INDUSTRIAL APPLICABILITY The present invention can be used as an IP videophone service system that realizes an interactive service using non-real-time video using an IP network.

Claims

The scope of the claims

[1] When an IP videophone service is provided and connected to an IP network, the distribution server follows a preset distribution procedure and responds to the IP videophone service user's response to deliver video clips and still images. An IP videophone service system comprising means for determining various media such as text, voice, and the like and a distribution method thereof, and distributing these media as streaming video to the IP videophone.

[2] The IP according to claim 1, wherein the distribution server includes means for distributing only recorded media, or a combination of recorded media and live broadcast media. Videophone service system.

[3] The distribution procedure is characterized in that distribution time, distribution time, position on an IP videophone screen, display method, and behavior of the various media are described. Or the IP videophone service system according to 2.

[4] The distribution server includes means for selecting a video to be distributed and adjusting a distribution method according to the distribution procedure according to the user environment such as an IP video telephone number and a language setting of the IP video telephone service user. The IP videophone service system according to any one of claims 1 to 3, wherein the IP videophone service system is provided.

[5] The distribution server automatically creates a distribution procedure according to the instructions of the IP videophone service user, receives the audio and video of the IP videophone service user, and automatically executes the respective media. The IP videophone service system according to any one of claims 1 to 4, further comprising means for creating the IP videophone service.

[6] An IP video phone terminal for using the IP video phone service,

A distribution server that distributes video to the IP videophone terminal,

A file server that stores various media such as video clips, still images, characters, and audio to be distributed, and their caches;

In an IP network consisting of a gatekeeper that performs IP video phone number resolution and bandwidth management,

The distribution server,

Transfer means for transferring data over the IP network; Decoding means for decoding data encoded as IP videophone terminal data into audio data, video data, and the like;

Encoding means for encoding audio data, video data, etc. as IP videophone terminal data;

An IP phone command means for issuing commands necessary for communication with the IP videophone terminal;

Medium access means for inputting / outputting various media stored in the file server;

An IP videophone service system comprising:

The file server includes a scenario database that holds a description of a distribution procedure, and the distribution server further includes a scenario processing unit that processes the description of the distribution procedure and executes the description according to a preset distribution procedure.

7. The IP according to claim 6, wherein the distribution procedure describes a distribution time, a distribution time, a position on an IP videophone screen, a display method, and a behavior of the various media. Videophone service system.