JP2003296239A - Device and method for transmission, system and method for contents distribution, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To efficiently send contents, provided by a viewer side, to a sender side and to perform live distribution by using the contents. <P>SOLUTION: The contents of a program as an object of live distribution from a user terminal 10 as a sender side and sender specification information for specifying this user terminal 10 on the Internet 11 are distributed to a client terminal 12 through a streaming distribution server 13. Contents provided from the side of the client terminal 12 are sent directly to the user terminal 10 according to the sender specification information and on the side of this user terminal 10, the live distribution is carried out by using the contents sent from the client terminal 12. <P>COPYRIGHT: (C)2004,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a transmitting apparatus and a transmitting method for delivering required content via a network connected to a plurality of viewing apparatuses and a server apparatus, and further content generated by a transmitting apparatus serving as a transmitting side. The present invention relates to a content distribution system and a content distribution method for distributing content to a viewing device via a server device. It also relates to a program that realizes such a content distribution system.

[0002]

2. Description of the Related Art The so-called Internet (The Internet) has been widely used as a computer network to which a large number of computer devices are connected. Internet is TCP / IP (Transmission Control)
It is possible to freely transmit and receive various information between computers by using various protocols (interconnection procedure) represented by Protocol / Internet Protocol).

In order to distribute a moving image picked up by an image pickup device such as a digital video camera to a large number of viewers using the Internet, there are the following methods.

That is, as a first method, a file including a moving image is transferred (uploaded) from a computer device on the sender side to a server device, and another computer device issues an acquisition request to the server device. When it is performed, the transferred file may be freely acquired (downloaded). As a result, the viewer can download a moving image file from the server device using a computer device connected to the Internet and browse the moving image on the display screen of this computer device.

As a second method, the sender of the moving image prepares a server device himself and holds the file including the moving image in the server device in a freely downloadable state. As a result, similarly to the first method described above, the moving image can be distributed from this server device to the viewer-side computer device.

When the sender side prepares the server device himself / herself in this way, not only can the completed moving image file edited in advance be delivered,
By connecting the imaging device to the server device or the computer device on the caller side connected to the server device, the moving image captured by the imaging device can be live-distributed in real time.

Such live distribution can be realized by using various streaming distribution techniques, that is, a technique of sequentially reproducing moving image data held in a server device while downloading it via a computer network. it can. As streaming distribution technology, for example, "Real Networks"
System "(for example, see Non-Patent Document 1)," QuickTime Streaming Serve "of Apple Computer Co., Ltd.
r "(for example, see Non-Patent Document 2)," Windows Media Technology "of Microsoft Corporation (for example,
See Non-Patent Document 3. ) Etc. can be mentioned.

[0008]

[Non-Patent Document 1] Internet <URL: http: // ww
w.jp.real.com/ ＞

[Non-Patent Document 2] Internet <URL: http: // ww
w.apple.co.jp/>

[Non-Patent Document 3] Internet <URL: http: // ww
w.microsoft.com/japan/ ＞

[0009]

By the way, in the case where a moving image or the like is live-distributed to the viewer side through the network as described above, the number of senders of the moving image is limited to one per one live distribution. Therefore, it was not possible to perform a distribution in which a moving image from another provider located far from the sender was captured in a live distribution program and connected to each other by relay.

Further, for example, in order to reflect the opinions and claims from the viewer side in the program for live distribution, it is necessary to use a system different from the system for live distribution. Specifically, for example, in a system that acquires the opinions and assertions of the viewer side in the form of characters and still images using application software that realizes e-mail and chat, and distributes the acquired characters and still images live. It is necessary to read it. Alternatively, it is necessary to use the application software that realizes e-mail, chat, etc. to exchange opinions and assertions in the form of simultaneous progress with the system for live distribution.

However, in the case of exchanging opinions and assertions with the viewer side and performing live distribution by the above-described method, it is necessary to operate a plurality of application software that are sparsely related, There was a problem that was complicated. In addition, it is unavoidable that the relationship between the live video and the content of the chat is weakened, and opinions and assertions can be exchanged smoothly between the video sender and the viewer. Becomes difficult.
Furthermore, even if the moving image distributed live is recorded, there is a problem that the moving image alone does not constitute one program.

Therefore, the present invention has been made in view of the above-mentioned conventional circumstances, and the content such as the moving image, the still image, or the text provided by the viewer side can be freely used by the sender side. It is an object of the present invention to provide a transmitting device, a transmitting method, a content distribution system, a content distribution method, and a program capable of performing the same.

[0013]

A transmitting device according to the present invention is a transmitting device connected via a network to a server device for delivering contents and a plurality of viewing devices for receiving contents delivered from the server device. And
The communication device includes communication means for communicating with another device via a network, and control means for generating content to be distributed to the viewing device via the server device. The control means connects the process of acquiring the first content distributed to the viewing device via the server device and the second content distributed to the viewing device via the server device to the network. A process of directly acquiring from another device, a process of generating a transmission content to be transmitted to the server device using the first content and the second content, and a process of transmitting the transmission content via the communication means. It is configured to execute a process of transmitting to the server device.

Further, the transmission method according to the present invention is a transmission method for delivering contents to a plurality of viewing devices via a server device for delivering contents, and is delivered to the viewing devices via the server device. Acquiring first content, acquiring second content distributed to the viewing device via the server device from another device connected to the network, the first content and the second content Generating a transmission content to be transmitted to the server device using the content;
Transmitting the transmission content to the server device.

Further, the program according to the present invention performs a content transmission process to a device connected via a network to a server device that distributes content and a plurality of viewing devices that receive the content distributed from the server device. A program to be executed, a process of controlling to obtain the first content distributed to the viewing device via the server device, and a second content distributed to the viewing device via the server device, A process of performing control for obtaining from another device connected to the network; a process of performing control for generating transmission content to be transmitted to the server device by using the first content and the second content; And a process of controlling transmission of transmission contents to the server device.

A content distribution system according to the present invention is a content distribution system connected to a plurality of viewing devices via a network, and a distribution means for distributing content to the plurality of viewing devices, and the distribution means. And generating means for generating content to be distributed to the viewing device via the. The generation means is connected to the network for obtaining the first content distributed to the viewing device via the distribution means and the second content distributed to the viewing device via the distribution means. Further, a process of directly obtaining from another device and a process of using the first content and the second content to generate a distribution content to be distributed to the viewing device via the distribution means are executed. Further, the distribution unit executes a process of distributing the distribution content generated by the generation unit to the plurality of viewing devices.

Further, the content distribution method according to the present invention has a content distribution having a distribution means for distributing the content to a plurality of viewing devices and a generation means for generating the content to be distributed to the viewing devices via the distribution means. A content distribution method for distributing content in a system, the method comprising: acquiring first content distributed to the viewing device via the distribution means, and distributing to the viewing device via the distribution means. 2 contents,
Obtaining directly from another device connected to the network, and using the first content and the second content to generate distribution content to be distributed to the viewing device via the distribution means, Delivering the delivery content generated by the generating means to the plurality of viewing devices.

Further, the program according to the present invention can be applied to a system having distribution means for distributing contents to a plurality of viewing devices, and generation means for generating contents to be distributed to the viewing devices via the distribution means. And a second content distributed to the viewing device via the distribution unit, the program acquiring the first content distributed to the viewing device via the distribution unit. , A process of directly obtaining from another device connected to the network, and a process of using the first content and the second content to generate a delivery content to be delivered to the viewing device via the delivery means. And a process of distributing the distribution content generated by the generation means to the plurality of viewing devices.

According to the transmitting device and the transmitting method of the present invention,
It is possible to insert the content transmitted from a plurality of viewing devices into the content to be transmitted and distribute this content. Therefore, the content such as the moving image, the still image, or the text provided from the viewing device can be freely used by the transmitting device to distribute the content. For this reason, opinions and claims can be exchanged extremely smoothly between the viewer and sender of the distributed content. Further, in the content distribution system and the content distribution method of the present invention, the content is directly transmitted from the other device to the transmission device. Therefore, the content inserted into the distributed content can be exchanged extremely smoothly. With the program of the present invention, it is possible to smoothly transfer the content as described above in the device in which the program is processed.

[0020]

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. The present invention utilizes a plurality of computer devices and a server device connected to a computer network to generate content generated by a computer device, which is an example of a transmitting device,
It is suitable to be applied to the case of delivering to a computer device which is an example of a viewing device via a server device. Therefore, in the following, first, the configuration of a basic system for delivering contents in this manner will be schematically described.

(1) Basic Configuration of Content Delivery System The content delivery system 1 described in this example is, for example, as shown in FIG. 1, a digital video camera (Digital Vide Camera).
(Camera: DV camera) Video data and audio data obtained by shooting by the user terminal 10 including a personal computer device equipped with or connected to a DV camera, or input from an input device such as a keyboard provided in the user terminal 10. Content such as text data is transmitted to a plurality of client terminals 12a, through a computer network such as the Internet 11.
Live distribution is performed for 12b, 12c, ....

Here, the user terminal 10 and the client terminals 12a, 12b, 12c, ... Are personal computer devices and PDAs (Personal Digital Assistants).
s) Devices, such as mobile phones equipped with communication functions.
In addition, the content to be distributed includes video data, audio data, which is stored in the user terminal 10 and which has been edited in advance.
It may be text data or the like.

In this content distribution system 1,
The user terminal 10 is equipped with a multi-window function that is a function of simultaneously displaying a plurality of windows on the display screen. In particular, the user terminal 10 has a window as a main screen that is displayed on the display screen for each function of an application program that is executed when content is distributed, and a setting screen for performing various settings by the user. By seamlessly migrating windows, the relationship between the main screen and the setting screen can be clarified, and an easy-to-understand user interface can be provided to the user. More specifically, the user terminal 10 introduces the concept of a panel window in which a main screen for each function is divided vertically on the display screen in order to represent the sharing of various functions on the display screen. , It contributes to the realization of a personal casting service under easy operation.

In the following, prior to a detailed description of such a user terminal 10, the basic overall configuration of the content distribution system 1 will be described.

As shown in FIG. 1, the content delivery system 1 includes a user terminal 10 for delivering content and a plurality of client terminals 12a, 12b, 1 for receiving and viewing the content delivered by the user terminal 10.
2c, ..., and a streaming distribution server 1 described later.
A server use reservation management center 14 that manages reservation information for the user terminal 10 to use the content distribution function according to No. 3, and the content distributed by the user terminal 10 to a plurality of client terminals 12a, 12b, 12c,
Streaming distribution server 13 that distributes to ...
With.

The user terminal 10 is equipped with or connected to a digital video camera, and transmits video data and audio data captured by the digital video camera while compressing them in real time based on a predetermined encoding format. The user terminal 10 can also transmit edited video data and audio data recorded in a recording device such as a hard disk.

Specifically, the user terminal 10 includes an ISP (Internet Service Provider) (not shown), a telephone line network 15a, a cable television network 15b or ADSL.
The environment is such that it can be connected to the Internet 11 via a public line network 15 composed of an (Asymmetric Digital Subscriber Line) network 15c. At this time, the user terminal 10
May be wired or wireless as long as it can connect to the Internet 11. For example, the user terminal 10
Is a so-called Personal Handyphone Sy
stem; hereinafter referred to as PHS. ), Etc., and may be configured to connect to the Internet 11.

The user terminal 10 needs to reserve the distribution of the content with respect to the server use reservation management center 14 before transmitting the content. User terminal 10
Accesses the server use reservation management center 14 via the public line network 15 and the Internet 11 to request the use of the streaming distribution function of the content by the streaming distribution server 13, the number of viewers, and the viewer password. A reservation is made by registering predetermined reservation information.

Then, the user terminal 10 receives the reservation and then the public line network 15 and the Internet 11
The content is transmitted to the streaming distribution server 13 via. In addition, the user terminal 10 is, for example, via the public line network 15 a server connection dedicated network 16
PPP (Point to Point Protoco
l) By making a connection, a transmission path with the streaming distribution server 13 can be established, and contents can be transmitted to the streaming distribution server 13 via this transmission path. Furthermore, the user terminal 10 can also record the transmitted content in a recording device such as a hard disk, if necessary.

Client terminals 12a, 12b, 12
Reference numerals c, ... Represent information processing devices such as personal computers and portable information terminals, which are connected to the Internet 11. Client terminal 12
Each of a, 12b, 12c, ... Can enter the content distribution system 1 as a viewing terminal by inputting predetermined information such as a viewer password. Client terminals 12a, 12b, 12
When the distribution start date / time of the content reserved and registered by the user terminal 10 comes, the contents c, ... Are distributed from the streaming distribution server 13 via the Internet 11 and receive the distributed contents. Then, the client terminals 12a, 12b, 12
c, ... Decode the received contents in real time and output them via a display screen, a speaker, or the like.

The server use reservation management center 14 manages reservation information for the user terminal 10 to use the streaming distribution function of contents by the streaming distribution server 13. When the predetermined reservation information is registered by the user terminal 10, the server use reservation management center 14 receives the streaming distribution server 13 by the user terminal 10.
Book for use. In addition, the server use reservation management center 1
4 is connected to the streaming distribution server 13 via a dedicated line 17, and streaming distribution is performed via the dedicated line 17 during the authentication processing for the user terminal 10 and the client terminals 12a, 12b, 12c, .... The authentication data is exchanged with the server 13.

The streaming distribution server 13 receives the content transmitted from the user terminal 10 via the Internet 11 or the server connection dedicated network 16, and receives a plurality of client terminals 12a, 12b, 12c, ... Stream to. At this time, the streaming distribution server 1
3 is connected to the server use reservation management center 14 via a dedicated line 17, and via the dedicated line 17 during the authentication processing for the user terminal 10 and the client terminals 12a, 12b, 12c, .... Authentication data is exchanged with the server use reservation management center 14. The streaming delivery server 13, when the delivery start date and time of the content reserved and registered by the user terminal 10 comes,
While receiving the content transmitted from the user terminal 10 via the Internet 11 or the server connection-dedicated network 16 and storing it in a buffer (not shown), the content is read from the buffer and the client terminals 12a, 12b, 12c that have made a distribution request. Deliver to ...

In the content distribution system 1 as described above, a user who is a content distributor accesses the server use reservation management center 14 via the user terminal 10 and registers the above-mentioned reservation information so that the content live. Delivery is scheduled.

Specifically, in the content distribution system 1, the WWW (World Wide Web) browsing window of the website providing the service of the content distribution system 1 is displayed on the display screen of the user terminal 10, and the user is allowed to browse the window. Performs user registration in the content distribution system 1 via the user terminal 10, logs in by inputting a given user ID (IDentification) and password. As a result, a predetermined window for making a reservation for content distribution is displayed on the display screen of the user terminal 10.

Then, when the user inputs predetermined reservation information through the window displayed on the display screen of the user terminal 10 and the personal identification is successful by inputting the credit card number or the like, the contents at the designated date and time are displayed. Is scheduled for live distribution. In the content distribution system 1, when a reservation for live distribution is accepted, the reservation setting file is stored in the server use reservation management center 14
Is downloaded to the user terminal 10. In this reservation setting file, in addition to the set reservation information, the IP (Inte
rnet Protocol) address and port, user ID, password, etc. are described.

In the content distribution system 1, the user terminal 10 executes a live distribution application program, which will be described later, to automatically read these pieces of information, so that live distribution can be carried out without requiring a complicated setting work by the user. It becomes possible to do.

Next, in the content distribution system 1, the server use reservation management center 1 is operated from the user terminal 10.
For 4, the reservation confirmation as to whether or not to actually perform the reserved live distribution is performed before a predetermined time, for example, about 3 hours before the distribution start date and time. Specifically, in the content distribution system 1, the user terminal 10 displays a WWW browsing window for accessing the server use reservation management center 14 on the display screen, and the user selects a predetermined window through this window. Reservation confirmation is performed by inputting the information. In the content distribution system 1, the reservation is confirmed accordingly. In the content distribution system 1, the user needs to contact the client, who wants to view the content, with the information necessary for viewing the content, such as the viewer password, in advance by the distribution start date and time.

Then, in the content distribution system 1, the user terminal 10 sends the streaming distribution server 13 a predetermined time before the distribution start date and time, for example, about 5 minutes before.
When the delivery start date and time is reached, the user terminal 1
At the same time as shooting with a digital video camera mounted or connected to 0, while video data and audio data are compressed in real time based on a predetermined encoding format,
It is transmitted to the streaming distribution server 13. In the content distribution system 1, the client serving as the viewer inputs the predetermined information such as the viewer password, and thereby the client terminals 12a, 12b, 12
Content can be received and viewed via c, ....

In this way, the content distribution system 1
Is video data or audio data obtained by shooting with the user terminal 10, pre-edited video data or audio data stored in the user terminal 10, or input using the input device of the user terminal 10. Content such as text data is sent to the client terminals 12a, 12b, 12c, ...
To be delivered live. That is, so to speak, it is possible to realize a personal casting service in which an individual user delivers content live to a plurality of viewers.

In the content distribution system 1, after the user terminal 10 makes the above-mentioned reservation and setting of the live distribution and before the actual distribution is started, the effects required for the program and the user terminal 10 are set. It is possible to preset and register archive files recorded on the hard disk of the. At this time, the user terminal 10 can perform various settings using a predetermined guide function. In addition, in the content distribution system 1, the user terminal 10 can take a trial shot of content to be live-distributed to a recording device such as a hard disk at times other than the reserved time. At this time,
As in the case of actual distribution, the user terminal 10 applies various effects to the content and records video data and audio data obtained by shooting with a digital video camera that can be selected as an input source on the hard disk. It is possible to switch (switch) with the existing video data and audio data.

In the content distribution system 1, the user terminal 10 and the streaming distribution server 13
When an effect operation or a file switching operation is performed by the user terminal 10 in a state in which the connection with is established and the content is delivered live, these operations are also reflected in the delivered stream according to this operation. Finally, the client terminals 12a, 12b, 12c, ...
It is also reflected in the display screen in.

In the content distribution system 1, when the distribution end time comes, the distribution of the content automatically ends, and the client terminals 12a, 12b, 12c, ...
The program ends on the display screen in. In the content distribution system 1, information about the content that has been distributed is recorded in an album format on a recording device such as a hard disk in the user terminal 10, and details of the distribution history and the distributed content are reproduced and viewed by the user terminal 10. You can

(2) Configuration of User Terminal Next, the configuration of the user terminal 10 that constitutes such a content distribution system 1 will be described below.
Note that, for convenience of description, the user terminal 10 equipped with a digital video camera will be described below. However, in the user terminal 10, a digital video camera configured as an external device may be connected by wire or wirelessly, or for example to a computer network such as a LAN (Local Area Network) to which the user terminal 10 is connected. A digital video camera that can be directly connected may be used, and the image data and the audio data captured by the digital video camera may be captured by the user terminal 10 via a computer network.

As shown in FIG. 2, the user terminal 10 includes a CPU (Central Processing Unit) 20 for executing various arithmetic processes and controlling each section in an integrated manner, and a CP (Central Processing Unit) 20.
RAM (Random A) that functions as a work area for U20
ccess Memory) 21 and a read-only R that stores information including various programs executed by the CPU 20
An OM (Read Only Memory) 22 and an operating system (Operating System) executed by the CPU 20.
(hard disk drive) 2 in which various programs such as a stem) and application programs are recorded, and various data such as distributed contents are recorded and reproduced.
3, a display section 24 for displaying various information, and this display section 2
4, a display interface 25 for exchanging data between the CPU 4 and the CPU 20, an operation unit 26 for inputting various information and instruction operations by the user, and data between the operation unit 26 and the CPU 20. An operation interface 27 for exchanging data and the public line network 1 described above.
A network interface 28 for exchanging data with an external device connected to the Internet 11 or the server connection dedicated network 16 via 5.
A digital video camera 29 for photographing a subject to obtain video data and audio data, and a so-called MPEG2 (Moving P
and an MPEG2 codec 30 that performs compression encoding and decoding based on the icture Experts Group phase 2) method.

The user terminal 10 has the following units.
CPU 20, RAM 21, ROM 22, HDD 23, display interface 25, operation interface 2
7, the network interface 28, the digital video camera 29, and the MPEG2 codec 30 on the bus 3
It is configured to be connected via 1.

The digital video camera 29 in the user terminal 10 may not be mounted in the user terminal 10 but may be connected to the bus 31 via, for example, a predetermined interface for connecting an external device or the network interface 28. . In addition, the user terminal 10 includes, for example, a removable recording / reproducing device that performs recording / reproducing on / from a removable recording medium, if necessary.
It may be mounted or connected in addition to No. 3, and can record / reproduce various data to / from this recording / reproducing apparatus.

The CPU 20 is a RAM via the bus 31.
21, the ROM 22, the HDD 23, the display interface 25, the operation interface 27, the network interface 28, the digital video camera 29, and the MPEG2 codec 30. CPU2
Reference numeral 0 centrally controls each unit and executes an operating system and various application programs recorded in the HDD 23 and the like. Especially CPU
20 performs processing and control regarding reservation of content distribution to the server use reservation management center 14, transmission of content to the streaming distribution server 13, and recording of the transmitted content in the HDD 23.

The RAM 21 functions as a work area when the CPU 20 executes various programs.
Under the control of 0, various data are temporarily stored.

The ROM 22 stores various programs and setting information necessary for starting the user terminal 10. Various programs and setting information stored in the ROM 22 are read when the user terminal 10 is started, and
Used by PU20.

Under the control of the CPU 20, the HDD 23 operates under the control of operating system and applications.
Recording and reproducing various programs such as programs and data. Further, the transmitted content is recorded in the HDD 23. The HDD 23 can also record the edited content in advance, and can read the content at the time of distribution under the control of the CPU 20.

The display unit 24 is, for example, an LCD (Liquid Cry).
stal display), and under the control of the CPU 20,
Various information such as data recorded in the HDD 23 is displayed on the display screen. In particular, the display unit 24 is a WW for accessing the server use reservation management center 14 and the streaming distribution server 13 under the control of the CPU 20.
A predetermined graphical user interface such as a window for W browsing and a predetermined window for executing various application programs for performing various processing including processing for content to be transmitted is displayed on the display screen.

The display interface 25 is the CPU 2
Data is exchanged between 0 and the display unit 24. That is, the display interface 25 supplies various information supplied from the CPU 20 via the bus 31 to the display unit 24.

The operation unit 26 accepts an operation by a user using a user interface such as a pointing device such as a keyboard and a mouse, or a so-called rotary operation switch such as a so-called jog dial, and outputs a control signal indicating the operation content to the operation interface. It is supplied to the CPU 20 via 27.

The operation interface 27 is the CPU 2
Data is exchanged between 0 and the operation unit 26. That is, the operation interface 27 supplies the control signal supplied from the operation unit 26 to the CPU 20 via the bus 31.

The network interface 28 is C
It functions as an interface for communicating with the outside under the control of the PU 20. That is, the network interface 28 is provided for exchanging data between the user terminal 10 and an external device connected to the Internet 11 or the server connection dedicated network 16 via the public line network 15. In particular, the network interface 28 transmits the content to the outside under the control of the CPU 20 and also transmits and receives various information such as the reservation information and the reservation ID described above.

The digital video camera 29 includes a predetermined optical system for photographing an object and a CCD (Charge Coupled).
Devices) and other devices for photoelectric conversion. The video data and audio data captured by the digital video camera 29 are supplied to the MPEG2 codec 30 via the bus 31 under the control of the CPU 20.

The MPEG2 codec 30 uses the video data and audio data supplied from the digital video camera 29 via the bus 31 under the control of the CPU 20 as an MP.
The compression encoding based on the EG2 method is performed. At this time, MP
The EG2 codec 30 is under the control of the CPU 20.
Video and audio data are compressed and encoded in real time. The MPEG2 codec 30 can also decode data that is compression-encoded based on the MPEG2 system.

(3) Live Distribution Application Program Next, a live distribution application program executed by the user terminal 10 as described above will be described. The live distribution application program is composed of a series of program groups in which a plurality of modules each realizing an individual function can be freely combined, and for example, the HDD 23 of the user terminal 10.
It is recorded in. The user terminal 10 performs live distribution of the content by executing the live distribution application program by the CPU 20. The live distribution application program can be provided by a predetermined recording medium such as a so-called compact disc or a transmission medium such as the Internet.

By executing this live distribution application program, the display unit 2 of the user terminal 10
In 4, a window as shown in FIG. 3, for example, is displayed.

As shown in FIG. 3, this window is displayed on the main window 50 created by the main module which is the backbone of the live distribution application program.
Guidance bar 5 created by the main module
1 and a status bar 52, and a plurality of panel windows 53a, 53b created by each functional module,
53c, 53d, 53e are arranged.

In the main window 50, a horizontally long strip-shaped guidance bar 51 is arranged at the upper part, and a horizontally long strip-shaped status bar 52 is arranged at the lower part, and the guidance bar 51 and the status bar 52 are vertically arranged. A plurality of panel windows 53a, 53b, 53 vertically divided in a sandwiched manner corresponding to each function.
c, 53d, and 53e are arranged in parallel and displayed on the display unit 24. Here, each panel window 53a, 53b, 53
c, 53d, and 53e are each implemented as one independent window.

In FIG. 3, a panel window is a sound panel window 53a for performing an operation relating to sound, and an operation relating to image data obtained by photographing with the digital video camera 29 selectable as an input source is performed. Camera input panel window 53b, a main monitor panel window 53c for performing an operation related to the content being live-distributed, and a file input panel window 53d for performing an operation related to video data recorded in the HDD 23 selectable as an input source. , And an effect panel window 53e for performing an operation related to an effect to be applied to content to be distributed is shown.

The guidance bar 51 has a plurality of buttons for inputting various operations. The user can perform various operations on the live distribution application program by clicking a button displayed on the guidance bar 51 with a mouse.

The status bar 52 is, for example, the above-mentioned P
An area for displaying the radio wave state when connecting to the Internet 11 using the HS, the connection state when performing the above-mentioned PPP connection, the remaining capacity state of the battery mounted in the user terminal 10, the remaining capacity state of the HDD 23, and the like. Implement.

The sound panel window 53a has a plurality of icons 53a for showing a list of voice / music data such as wav files registered to be played as background music during live distribution.
1, a button 53a2 for selecting these icons, and a button 53a3 for playing or stopping voice / music data corresponding to the selected icon are mounted. In addition,
In the sound panel window 53a, other voice / music data can be freely added to the list or the voice / music data registered in the list can be deleted.

The camera input panel window 53b displays a preview area 53b for displaying a preview image for previewing video data obtained by shooting with the digital video camera 29 selectable as an input source.
1. Button 53 for setting the input source to the video data obtained by shooting with digital video camera 29
b2, an HDD capable of selecting audio data obtained by shooting with the digital video camera 29 as an input source
A button 53b3 for performing so-called post-recording, which is combined with the video data recorded in 23, a button 53b4 for activating a setting application program of the digital video camera 29, and the like are mounted.

The main monitor panel window 53c is
A preview area 53c1 for previewing content to be delivered live, a button 53c2 for starting trial shooting or actual delivery, a button 53c3 for temporarily stopping playback or delivery of content, and mute processing for audio data A button 53c4 for, and a time display area 53c5 for displaying the elapsed time of distribution, the remaining time, and the like are mounted.

The file input panel window 53d is
A thumbnail area 53d1 for displaying a thumbnail image showing a list of video data recorded in the HDD 23 selectable as an input source, a preview area 53d2 for displaying a preview image for previewing the video data, and a thumbnail image as a list. Button 53d3 for selecting, and button 53 for reproducing or stopping the video data corresponding to the selected thumbnail image.
Implement d4 etc.

The effect panel window 53e has a thumbnail area 53e1 for displaying a thumbnail image for showing a list of various effect materials to be applied to the contents being live-distributed, a button 53e2 for selecting these thumbnail images, and the selected button. Button 5 for executing or canceling the effect corresponding to the thumbnail image
3e3, a box 53e4 for inputting a character string to be displayed as a telop in real time as an effect on video data
Etc. are implemented. In the effect panel window 53e, it is possible to add another effect material to the list or delete the effect material registered in the list.

In this way, the user terminal 10 divides the panel windows 53a, 53b, 53c, 53d, 53e corresponding to the respective functions in the vertical direction and displays them on the display unit 24 in order to represent the sharing of the various functions. . This allows the user to intuitively and easily carry out the operation for each function and the necessary buttons. For example, the user may pay attention only to the effect panel window 53e when performing the process related to the effect.

The user terminal 10 can create contents to be distributed by freely combining video data, audio data, text data, etc. using the above-mentioned live distribution application program. Then, the created content is sent to the streaming distribution server 13 via the Internet 11 or the server connection-dedicated network 16, and the streaming distribution server 13 sends the client terminals 12a, 12b, 1 to the contents.
2c is delivered.

(4) Content Distribution System for Acquiring Content from Viewing Side Next, based on the basic configuration of the content distribution system 1 described above, the user terminal 10 directly acquires content from the client terminal on the viewing side. A case where live distribution is performed using the acquired content will be described with reference to FIG.

FIG. 4 shows a case where the digital video camera (DV camera) 29 is configured as an external device with respect to the user terminal 10 and the DV camera 29 is connected to the user terminal 10.

In the content distribution system 1, as described above, when performing live distribution by the user terminal 10, the content created by the live distribution application program executed on the user terminal 10 is
As shown by an arrow A1 in FIG. 4, this user terminal 10 sends it to the streaming distribution server 13 via the Internet 11. Then, the content is distributed from the streaming distribution server 13 to the plurality of client terminals 12 as indicated by an arrow A2 in the figure.

Here, in addition to the above-mentioned live distribution application program, the user terminal 10 executes a communication application program for acquiring and using the content transmitted from the client terminal.

This communication application program is recorded in the HDD 23 of the user terminal 10, for example, similarly to the live distribution application program. The user terminal 10 can acquire and use the content transmitted from the client terminal by executing this communication application program by the CPU 20. An application program for communication is, for example, a so-called compact disc (Compact Disk).
It can also be provided by a predetermined recording medium such as sc) or a transmission medium such as the Internet.

The communication application program operates in close cooperation with the above-mentioned live distribution application program, as will be described later. It should be noted that the communication application program may be executed independently or independently of the live distribution application program.

On the other hand, in the content distribution system 1, the client terminal 12 can send out the content to be used exclusively for viewing the content distributed from the streaming distribution server 13 and to the user terminal 10 performing live distribution. It can be broadly divided into what was made possible.

Client terminal 1 used exclusively for viewing
2, it is sufficient that at least an application program for viewing the distributed content can be executed, such as a Web browser.

Further, as shown in FIG. 4, the client terminal 12 capable of transmitting contents has the DV camera 29 mounted or connectable similarly to the user terminal 10. The client terminal 12 can execute at least an application program for viewing the distributed content (for example, a Web browser) and a communication application program for transmitting the content to the user terminal 10. It is said that.

In the client terminal 12 capable of transmitting contents, contents such as video data and audio data photographed by the DV camera 29 or text data inputted by the input device of the client terminal 12 are stored. As shown by an arrow A3 in FIG. 4, the data is directly transmitted to the user terminal 10 via the Internet 11 without passing through the streaming distribution server 13.

In the content distribution system 1, when the content is directly sent from the client terminal 12 to the user terminal 10, it is realized by the method described below.

That is, from the user terminal 10, the sender identification information for identifying the user terminal 10 on the Internet 11 is sent to the streaming delivery server 13 together with the content to be live delivered. And
From the streaming distribution server 13, the content and the sender identification information are sent to each client terminal 12.

As the sender identification information, for example, information unique to the user terminal 10 on the network, such as an IP address, or the user terminal 1 is used.
0, the communication application program includes a port number and the like used for receiving contents from the client terminal 12.

Then, the client terminal 12 which sends the content to the user terminal 10 specifies the user terminal 10 on the Internet 11 based on the caller specifying information distributed from the streaming distribution server 13 together with the content. Connection (session) for transmitting / receiving contents to / from the user terminal 10 on the network
Establish. The client terminal 12 is the user terminal 10
The content is directly transmitted to the client via so-called peer-to-peer without going through the streaming distribution server 13. That is, the client terminal 12
Does not send the content to a specific server on the network, but sends the content to the user terminal 10.

In the content distribution system 1, as described above, the client terminal 12 directly communicates with the user terminal 10 based on the sender identification information without passing through a specific server device such as the streaming distribution server 13. Since the content is transmitted, the client terminal 1 can be operated without considering the load on the server device.
It is possible to exchange contents very smoothly between the user terminal 2 and the user terminal 10.

(5) Actual Transmission / Reception of Content Between Client Terminal and User Terminal Here, the user terminal 10 acquires the content transmitted from the client terminal 12 as described above, and uses this content for live distribution. The case of performing will be specifically described.

When the contents sent from the client terminal 12 are used while the above-described live distribution application program is being executed on the user terminal 10, the guidance bar 51 of the main window 50 shown in FIG. 3 is used. The button 51a for starting the displayed communication application program is selected.

As a result, in the user terminal 10, a communication application program that operates in cooperation with the live distribution application program is activated, and as shown in FIG.
A communication window 70 created by the communication application program is displayed below the main window 50 of the program.

The button 51a for starting the communication application program displayed on the guidance bar 51 of the main window 50 can be selected only when the content is being sent from the client terminal 12 side. Client terminal 12
When the content is not sent from the side, it may be displayed in a non-selectable state.

This communication window 70
Is the DV of each client terminal 12 as shown in FIG.
A preview area 71 is arranged to display images picked up by the camera 29 and sent to the user terminal 10. The image displayed in the preview area 71 is a video image captured by the DV camera 29 of the client terminal 12 and transmitted to the user terminal 10, and may be a moving image or a still image. Good.

As the image displayed in the preview area 71, for example, an image of the face of the operator of each client terminal 12 taken by the DV camera 29 is used. Further, the client terminal 12 is not limited to sending the video imaged by the DV camera 29 to the user terminal 10. For example, video data previously captured by the DV camera 29 and recorded. It may be image data created by various image creating application programs, or image data prepared in advance in a communication application program executed on the client terminal 12 side described later.

Also, the communication window 7
At the bottom of the preview area 71, 0 is an insertion start button 72 for selecting to insert each image displayed in the preview area 71 (that is, the image transmitted from the client terminal 12) into the video to be delivered live. An insertion stop button 73 for stopping the insertion of the image sent from the client terminal 12 into the video to be delivered live,
Help reference button 7 for referring to the help information about the communication application program
4 and an end button 75 for ending the communication application program.

Here, in the state shown in FIG. 5, the image captured by the DV camera 29 of the user terminal 10 is displayed in the preview area 53b1 of the camera input panel window 53b in the main window 50 of the live distribution application program. The image is selected for live distribution, and the preview area 53c1 of the main monitor panel window 53c is selected.
Is also displayed. Therefore, in the state shown in FIG. 5, the video displayed in the preview area 53c1 is transmitted from the user terminal 10 to the streaming distribution server 13.
It has been sent to and is being delivered live.

The image displayed in the preview area 53c1 at this time is, for example, D of the user terminal 10.
This is an image of the face of the operator of the user terminal 10, that is, the face of the sender of the program that is live-distributed, taken by the V-camera 29.

At this time, in the content distribution system 1, the Web browser is activated in each client terminal 12, and the window 80 created by this Web browser is displayed on the display screen as shown in FIG. There is. Then, the Web browser in the client terminal 12 receives the content distributed from the streaming distribution server 13, and the HTML (Hyper Text) acquired from the streaming distribution server 13.
Predetermined contents are displayed in the window 80 based on the Mark-up Language) file.

Here, a moving image or a still image distributed from the streaming distribution server 13 is displayed on the Web browser.
And a predetermined plug-in program capable of displaying / reproducing content such as audio data is installed, and the content display area 81 created by this plug-in program is a window 80 created by a Web browser. It is displayed at a predetermined position inside.

As described above, the delivered content is We
b Since it is configured to be displayed in the window 80 of the browser, it is not necessary to prepare a dedicated application program for viewing a live distributed program. Further, by displaying the content in the window 80 of the Web browser, it is possible to easily display information related to the content to be distributed, and various information from the streaming distribution server 13 to the client terminal 12 side. It can be provided, and various and flexible contents can be distributed.

Further, since the content is displayed in the window 80 of the Web browser, reliable and good content can be provided without being influenced by the environment dependent on the operation system operating on the client terminal 12. Viewing can be enabled.

Then, in the content display area 81,
The content transmitted from the user terminal 10 is displayed / reproduced. As a result, in the content display area 81 of the client terminal 12, the content transmitted by the user terminal 10, that is, the preview area 5 of the main monitor panel window 53c of the user terminal 10 is displayed.
3c1 is displayed, and the content delivered live is displayed.

In the window 80 of the Web browser, in addition to the content display area 81 created by the above-mentioned plug-in program, the contents of the program displayed in the content display area 81, information about the sender, and the like. May be displayed by text or an image. The client terminal 12 can acquire the content of such a program, information about the sender, and the like from the streaming distribution server 13 as an HTML file.

Further, in the window 80 of the Web browser, a participation button 82 for participating in the program distributed from the streaming distribution server 13 is displayed below the content display area 81. By selecting the participation button 82, on the client terminal 12 side, a communication application program is activated separately from the Web browser.

The communication application program activated on the client terminal 12 side is
Various data can be directly exchanged with the communication application program activated on the user terminal 10 side. The communication application program activated on the client terminal 12 side may be configured as a separate program corresponding to the communication application program activated on the user terminal 10 side, or activated on the user terminal 10 side. It may be a program equivalent to the communication application program. When the communication application program executed by the client terminal 12 and the user terminal 10 is shared, the communication application program is designated by the user in advance, or the situation is automatically judged. The startup mode can be changed appropriately.

When the communication application program is activated on the client terminal 12 side, the communication window 90 created by this communication application program is displayed on the client terminal 12 as shown in FIG.
Is displayed on the display screen of. The communication window 90 may be displayed on the display screen at the same time as the window 80 of the Web browser, or may be switched and displayed.

The communication window 90 is
As shown in FIG. 7, a self-portrait display area 91 for displaying an image captured by the DV camera 29 mounted or connected to the client terminal 12 is provided.
Around 1 is arranged a plurality of buttons 92a to 92g for selecting various instructing operations. Further, a plurality of image display areas 93 for displaying images sent from other client terminals 12 are arranged on the outer periphery of the plurality of buttons 92a to 92g.

The self-portrait display area 91 is a DV camera 29 mounted or connected to the client terminal 12 that executes this communication application program.
This is an area for displaying an image captured by, and normally displays an image in which the face of the operator of the client terminal 12 is captured.

The plurality of buttons 92a to 92g each have the following functions.

The Web browser start button 92a is the We
b This is a button for starting the browser. This Web
By selecting the browser start button 92a,
Operations such as calling and starting an external Web browser program or starting a Web browser built in this communication application program are performed.

The emotion button 92b is used for the self-portrait display area 91.
This is a button for adding effects expressing various emotions to the image displayed on. By selecting this emotion button 92b, various emotions such as “crying”, “laughing”, and “angry” can be added to the image displayed in the self-portrait display area 91 by effects such as animation and sound effects. Can be expressed.

The action button 92c is a button for giving an effect representing various actions to an image sent from another client terminal 12 displayed in the image display area 93. When this action button 92c is selected, for example, "striking" or "stroking"
Various actions such as “twitch” can be expressed by adding an effect such as animation or sound effect to any of the image display areas 93.

The help button 92d is a button for referring to help information regarding the communication application program. This help button 92
By selecting d, a window displaying help information is opened, and various operation explanations and setting methods can be confirmed.

The return button 92e is a button for ending this communication application program or returning to the previous menu display stage. When the return button 92e is selected, the communication application program can be terminated, or the menu display stage immediately before, that is, the display stage immediately before in the hierarchical menu display can be returned.

The image / audio button 92f is a button for selecting an image to be displayed in the self-image display area 91. When the image / audio button 92f is selected, the image to be displayed in the self-portrait display area 91 is selected again from the face images registered in the communication application program in advance, or is connected to the client terminal 12. The moving image captured by the DV camera 29 can be selected as the self-portrait and set to be displayed in the self-portrait display area 91. Further, when the moving image captured by the DV camera 29 is selected as the self-portrait, the output level of the sound input from the DV camera 29 can be set.

The chat log button 92g is used for this application for communication as will be described later.
Text data input by the program is transmitted and received between the user terminal 10 and another client terminal 12,
It is a button for displaying log data when a so-called chat function is used.

Here, on the client terminal 12, FIG.
When the participation button 82 displayed on the window 80 created by the Web browser shown in FIG. 6 is selected and the communication application program is activated and the communication window 90 is displayed, the communication application program is displayed. By directly exchanging various data with the user terminal 10 based on the sender identification information transmitted from the streaming distribution server 13, the moving image or the still image displayed in the self-portrait display area 91 is displayed on the user terminal 1
Send to 0.

Thus, the display unit 24 of the user terminal 10
In the preview area 71 in the communication window 70 displayed in Fig. 5, the moving image or still image sent from each client terminal 12 as described above is displayed as shown in Fig. 5.

Here, in the preview area 71, as shown in FIG. 5, contents provided from each client terminal 12 are displayed in a list, so that the user terminal 1
The user who operates 0 can easily visually recognize which participant can participate in the program to be live-distributed, and can intuitively select the participant who participates in the program.

At this time, the insertion start button 72 of the communication window 70 is displayed on the user terminal 10.
Is selected and one of the plurality of face images displayed side by side in the preview area 71 is selected, the selected face image is displayed in the preview area 53c1 of the main monitor panel window 53c as shown in FIG. It is inserted at a predetermined position in the displayed image.

In FIG. 8, the display content of the self-portrait display area 91 of the communication window 90 on the client terminal 12 side shown in FIG. 7 is reflected as the face image 71a in the preview area 71 of the communication window 70. The state is shown, and this face image 71a is selected, and the main monitor panel window 5 is displayed.
3C shows a state in which the face image 100 is inserted in the preview area 53c1 of 3c.

In this way, the face image 100 sent from the client terminal 12 by the user terminal 10 is displayed.
Is inserted in the original distribution image, the moving image in which the face image 100 is inserted is sent to the streaming distribution server 13, and this moving image is live-distributed to each client terminal 12. That is, on the client terminal 12 side, in the content display area 81 in the window 80 of the Web browser for browsing the distributed program,
As shown in, the moving image corresponding to the content displayed in the preview area 53c1 of the main monitor panel window 53c shown in FIG. 8 is displayed.

Further, in the communication application program executed on the client terminal 12, when the mouse cursor is moved to the self-portrait display area 91, as shown in FIG. 10, the self-portrait display area 91 is moved to a position corresponding to the self-portrait display area 91. Text input box 91a
Is displayed.

Here, in the client terminal 12 in which the face image to be inserted in the moving image live-distributed on the user terminal 10 side is selected, the text input box 91a is displayed.
Within, for example, text data such as "Hello!" Is, description will be given of a case that has been input by the input device such as a keyboard, which is provided to the client terminal 12.

In this case, the communication application program executed on the client terminal 12 specifies the user terminal 10 on the Internet 11 based on the sender specifying information distributed from the streaming distribution server 13, and inputs the text. The text data input in the box 91a is directly sent to the user terminal 10.

As a result, in the user terminal 10,
As shown in FIG. 11, the main monitor panel window 5
The image displayed on the 3c of the preview area 53c1, so that the to be "Hi!" Sent from the client terminal 12 becomes text data as described above is displayed is inserted.

From the user terminal 10, the preview area 53c1 of the main monitor panel window 53c is displayed.
The image displayed on the, that is, the face image 100 sent from the client terminal 12, "Hello!" As text data and the moving image that has been inserted is sent to the streaming server 13, streaming server this video image 13 is delivered to each client terminal 12 live.

[0126] As a result, the content display area 81 of the Web browser window 80 in each client terminal 12, as shown in FIG. 12, the face image 100 "Hello!" Is a moving image and text data has been inserted Will be displayed.

In the content distribution system 1 configured as described above, the client terminal 12 on the viewer side of the program to be live-distributed to the user terminal 10 on the sender side is based on the sender identification information. Content such as images and text can be sent directly. Further, the content transmitted from the client terminal 12 can be freely used on the user terminal 10 side and reflected in the program to be live-distributed.

Therefore, according to the content distribution system 1, a so-called viewer participation type program can be realized using the Internet with extremely simple and good operability. For this reason, for example, various questions and quiz questions can be asked to the viewer, and the response from the viewer can be displayed as a telop in text, so that a variety of flexible programs can be easily delivered live. You can Also, for example, the user terminal 10 on the calling side
An image captured by a viewer who is away from the user, input text data, and the like are displayed on the user terminal 10.
Since it can be freely combined with the moving image that is live-distributed on the side, it is possible to realize a program that connects various places by relay using the Internet.

(6) Processing in Communication Application Program Next, a series of processing executed on the user terminal 10 side by the communication application program described above will be described step by step with reference to a flowchart.

The communication application program is activated in a different state depending on whether it is activated by the live distribution application program on the user terminal 10 side or by the Web browser on the client terminal 12 side, for example. , In each case, different functions may be executable. However, in the following, as shown in FIG. 5, FIG. 8, and FIG. 11, a series of processing by the communication application program in a state of being activated by the live distribution application program will be described. In addition, a series of processes described below is performed by the CP of the user terminal 10 according to the flow of the processing procedure described in the communication application program.
It is realized by the U20 performing various arithmetic processes and controlling the operation of each unit.

Application for communication
When the program is started by the live distribution application program, session establishment preparation is performed as shown in step S10 in the flowchart shown in FIG. In this step S10, various kinds of data are exchanged with the live distribution application program to prepare for acquiring contents such as a face image sent from the client terminal 12.
The details of the processing in step S10 will be described later.

Next, in step S11, it is determined whether or not a connection request from the communication application program activated by the client terminal 12 has been received. That is, in step S11, it is determined whether the content such as the face image 100 is transmitted from the client terminal 12. If the result of this determination is that a connection request has been received, processing is performed in step S.
If not, the process proceeds to step S13.

In step S12, the content such as the face image 100 sent from the client terminal 12 is added to the preview area 71 of the communication window 70. This allows
The new participant is ready to participate in the program transmitted from the user terminal 10. Note that this step S12
Details of the processing in step 1 will be described later.

In step S13, the insertion start button 7 arranged in the communication window 70 is inserted.
2 is selected to determine if a new participant in the program has been selected to appear. If the result of this determination is that the insert start button 72 has been selected, the processing will proceed to step S.
If not selected, the process proceeds to step S15.

In step S14, a process for selecting which of the contents transmitted from each client terminal 12 and arranged in the preview area 71 of the communication window 70 to be inserted as live performers in the contents to be live-distributed I do. In addition,
Details of the processing in step S14 will be described later.

In step S15, the insertion stop button 7 arranged on the communication window 70 is inserted.
3 is selected, and it is determined in step S14 whether insertion stop of the inserted content is selected. That is, when the performer from the client terminal 12 is inserted in the program to be live-distributed, it is determined whether or not to stop the appearance. As a result of this determination, the insertion stop button 73
If is selected, the process proceeds to step S16, and if not selected, the process proceeds to step S17.

In step S16, a message requesting the appearance stop of the performer inserted in the program to be live-distributed is transmitted to the live-distribution application program, and the insertion processing of the content from the client terminal 12 is stopped. To the live distribution application program. This allows
The live distribution application program stops the insertion of the content from the client terminal 12 into the program transmitted from the user terminal 10.

In step S17, it is determined whether or not new content such as image data or text data has been transmitted from the client terminal 12 which is the transmission source of the content inserted in the program to be live-distributed. judge. As a result of this determination, if new content is received from the appearing client terminal 12, the process proceeds to step S18; otherwise, the process proceeds to step S19.

In step S18, the contents such as image data and text data newly received from the client terminal 12 that is currently appearing are transmitted to the live distribution application program. As a result, the new content is inserted and displayed in the program to be delivered live by the live delivery application program.

In step S19, the contents such as the text data and the image data acquired from the appearing client terminal 12 are transferred to the live distribution application.
The program is transmitted and handed over to update the content to be inserted into the program to be delivered live.

Next, in step S20, it is determined whether or not the user who is the sender operating the user terminal 10 has requested to terminate the communication application program. For such a request, for example, an end button arranged in the communication window 70 is selected, or an input for requesting the end of the communication application program is made by the operation unit 26 including, for example, a keyboard. Done by Then, as a result of this determination, when it is requested to end, the end processing of the communication application program is performed, the communication window 70 is closed, and the screen state shown in FIG. 3 is obtained. If it is not requested to end, the process is returned to step S11, and the series of processes described above is continued.

Next, the session establishment preparation in step S10 described above will be described with reference to FIG.

When the processing in step S10 is started, as shown in step S30 in FIG.
The IP address of the user terminal 10 itself is acquired from the live distribution application program. This IP address is information for identifying the user terminal 10 on the Internet 11, and is generally represented by four sets of numbers such as “134.55.22.33”.

Next, in step S31, the connection data necessary for the client terminal 12 to connect to the communication application program executed on the user terminal 10, that is, the sender identification information is generated, This sender identification information is passed to the live distribution application program.

This sender identification information is, for example, as shown in FIG. 15, an access key for authenticating when a communication application program executed on the user terminal 10 is accessed. Port number of the network interface 28 of the user terminal 10 used by the application program, IP used when connecting to this communication application program
It is composed of the number of addresses and a series of IP addresses used when connecting to this communication application program.

Next, in step S32, the communication application program starts the server process for realizing the securement of the connection state with the client terminal 12. Thereby, the client terminal 1
The user terminal 10 side can acquire the content transmitted from the terminal 2, and various data can be exchanged between the user terminal 10 and the client terminal 12.

Then, after the series of session establishment preparations in step S10 have been processed as described above, the process is returned to step S11.

Next, the participant addition processing in step S12 described above will be described with reference to FIG.

When the participant addition processing in step S12 is started, the access key included in the data sent from the client terminal 12 is used as the sender identification information shown in FIG. 15 in step S40 shown in FIG. Is compared with the access key included in the communication key, and it is determined whether or not this matches the access key created by the communication application program. As a result, the content transmitted from the client terminal 12 is transmitted to the communication application / application.
It is possible to determine whether or not it was sent for the purpose of being used by the program. Then, as a result of this determination, if the access keys match, the processing is performed in step S
41. If they do not match, the participant addition process in step S12 is stopped and the process proceeds to step S13.

In step S41, the content sent from the client terminal 12 is additionally registered in the participant list. This participant list corresponds to contents such as face images displayed in a list in the preview area 71 of the communication window 70.

Next, in step S42, contents such as the face image 100 and the registered name are acquired from the client terminal 12 side. Next, in step S43, the face image 100 and the registered name acquired in step S42 are displayed in the preview area 71. That is, the preview area 71 of the communication window 70 displays the content of the content registered in the participant list based on the participant list.

Then, after the series of participant addition processing in step S12 is processed as described above,
The process is returned to 13.

Next, the performer selection process in step S14 will be described with reference to FIG.

When the performer selecting process in step S14 is started, as shown in step S50 in FIG. 17, contents such as a plurality of face images arranged in the preview area 71 of the communication window 70 are selected. Switch to mode.

Next, in step S51, it is determined whether any one of the contents such as a plurality of face images arranged in the preview area 71 is selected. At this time, the selection of the content is performed, for example, by clicking any one of the plurality of face images with a mouse or the like that constitutes the operation unit 26 of the user terminal 10. Then, as a result of this determination, if any content is selected, the processing is performed in step S
If no content is selected, the process proceeds to step S55.

In step S52, step S5
It is stored that the content selected in 1 is inserted into a program to be live-distributed and the program is made to appear. Next, step S5
In 3, the content contents are acquired from the client terminal 12 which is the transmission source of this content. next,
In step S54, the information about the content acquired in step S53 is sent to the live distribution application.
The content is delivered to the program, whereby the content delivered from the client terminal 12 on the live delivery application program side is inserted into the program delivered live. When this step S54 is completed, the process proceeds to step S56.

In step S55, the preview area 7 is displayed by, for example, the mouse forming the operation unit 26.
It is determined whether or not a place other than the content such as the face image arranged in 1 is clicked. As a result of this judgment,
When the place other than the content is clicked, the process proceeds to step S56, and when the click operation is not performed, the process returns to step S51 and the processes after step S51 are continued. That is, in step S55, it is determined whether or not to cancel the mode for selecting the content such as the face image.

In step S56, the mode for selecting content such as a face image is canceled, and the process returns to step S15.

Next, the content transfer processing for the live distribution application program in step S19 described above will be described with reference to FIG.

When the content transfer processing for the live distribution application program in step S19 is started, step S6 in FIG.
As shown in 0, it is determined whether or not an inquiry about the size of content such as text data or image data to be inserted is made by the live distribution application program. The size inquired at this time is, for example, the number of pixels in the vertical and horizontal directions of the content to be inserted. Then, as a result of this determination, if the inquiry is made, the process proceeds to step S61, and if not, the process proceeds to step S62.

In step S61, an inquiry is made by the live distribution application program,
The size of the content to be inserted is calculated and passed to the live distribution application program.
As a result, the live distribution application program can know the size of the content to be inserted into the program to be live-distributed before actually receiving the content, and can perform various kinds of preprocessing.

In step S62, it is determined whether or not the information regarding the memory (shared memory) shared by the live distribution application program and the communication application program has been exchanged by the live distribution application program. To do. The information about the shared memory includes, for example, a start address in the RAM 21 provided in the user terminal 10 and the number of blocks secured in the RAM 21 as the shared memory. Then, as a result of this determination, if the information regarding the shared memory is exchanged, the process proceeds to step S63, and if the exchange is not performed, the contents of the content for the live distribution application program in step S19 and thereafter are transferred. The delivery process is stopped and the process proceeds to step S20.

In step S63, the content such as the face image sent from the client terminal 12 is acquired. Next, in step S64, step S63
The contents of the content such as the image data such as the face image and the text data obtained in step 3 are stored in the shared memory. next,
In step S65, RAM2 is used as a shared memory.
The memory area secured in 1 is released (closed), and the process returns to step S20.

In the user terminal 10, a series of processing is executed by the communication application program as described above. Accordingly, the user terminal 10 can receive the content sent from the client terminal 12 and pass it to the live distribution application program by executing the series of processes described above by the communication application program. .

(7) Processing in Live Distribution Application Program Next, of the series of processing executed on the user terminal 10 side by the above-described live distribution application program, the content transmitted from the client terminal 12 is inserted. The processing in the case of performing will be described step by step with reference to the flowchart shown in FIG.
The series of processes described below is realized by the CPU 20 of the user terminal 10 performing various arithmetic processes and controlling the operation of each unit in accordance with the flow of the processing procedure described in the live distribution application program. It

In the case where the content delivered from the client terminal 12 is inserted with the live distribution application program activated on the user terminal 10, first, in step S70 shown in FIG. 19, the communication application program is executed. It is determined whether content such as image data or text data has been transmitted from. This step S70 corresponds to the processing in step S18 in the communication application program. Then, as a result of this determination, if the content to be inserted from the communication application program is transmitted, the process proceeds to step S71, and if not, the process proceeds to step S76.

In step S71, an inquiry is made to the communication application program regarding the size of contents such as text data and image data to be inserted. The process in step S71 is performed by the communication application.
This corresponds to the processing in step S60 in the program. Then, after acquiring the size of the content to be inserted from the communication application program, the process proceeds to step S72.

In step S72, step S7
Based on the size of the content acquired in step 2, the shared memory having a sufficient capacity to store this content is R
Secure in AM21.

Next, in step S73, the information regarding the secured shared memory is transferred to the communication application program. The process in step S73 corresponds to the process in step S62 in the communication application program.

Next, after the content of the content transmitted from the client terminal 12 is stored in the shared memory by the communication application program, in step S74, this content is read from the shared memory and the program to be live-distributed is read. Insert it by combining it with the content. Next, step S75.
At, the memory area secured in the RAM 21 as the shared memory is released, and the process proceeds to step S76.

[0171] In step S76, it is determined whether or not a message requesting the appearance stop of the performer inserted in the program to be live-distributed is transmitted from the communication application program. The process in step S76 corresponds to the process in step S16 in the communication application program. And as a result of this determination,
If the message requesting the appearance stop is transmitted, the process proceeds to step S77. If not transmitted, the process for inserting the transmitted content is temporarily terminated and other processes are continued. .

Note that the series of processing shown in FIG. 19 shows only the processing for inserting the transmitted content among the various processing in the live distribution application program, and the processing after step S70 is as follows. It is repeatedly performed at a predetermined time or at intervals while various other processes are executed.

In step S77, the process of inserting the content delivered from the communication application program is stopped, and the user terminal 10
Content stored in advance in the user terminal 10
Live distribution is performed by the content imaged by the DV camera 29 mounted or connected to.

(8) Screen example when inserting a plurality of contents In the above description, one of the plurality of face images listed in the preview area 71 of the communication window 70 on the user terminal 10 side is selected. The moving image in which the face image is inserted is displayed on each client terminal 12
The case where it is delivered live is illustrated.

However, the content distribution system 1
In the above, the plurality of contents may be inserted when the user terminal 10 side acquires contents such as face images transmitted from the plurality of client terminals 12 and generates the content to be live-distributed on the user terminal 10 side. .

Therefore, a case where a plurality of contents are inserted in the original distribution image and live distribution is performed will be described below. In this example, for example, in the state shown in FIG.
A case where three of the plurality of face images displayed in the preview area 71 of the communication window 70 are selected will be described with reference to FIG.

When selecting a plurality of face images in this way, the plurality of face images displayed in the preview area 71 are individually clicked by the user of the user terminal 10, for example. To be done. Further, for example, a “participate in all” button similar to the insert start button 72 described above is displayed in the preview area 71, for example, and when this “all in attendance” button is selected by the user, the current session is All face images corresponding to the established users may be inserted into the image to be delivered live.

When a plurality of face images are selected in this way, as shown in FIG. 20, the selected face images are displayed in the preview area 53c1 of the main monitor panel window 53c.
It is inserted at a predetermined position of the image displayed on. Note that in FIG. 20, three face images 100a, 100
The case where b and 100c are selected and inserted in the preview area 53c1 will be illustrated.

Then, when the three face images 100a, 100b, 100c sent from each client terminal 12 are inserted into the original delivery image, the inserted delivery image is sent to the streaming delivery server 13, and each client terminal is sent. It will be delivered live to 12. As a result, on each client terminal 12 side, the three face images 100a, 10a
The distribution image in which 0b and 100c are inserted is browsed.

As described above, the content distribution system 1
Can insert content (face images 100a, 100b, 100c) respectively acquired from a plurality of client terminals 12 into a distribution image and perform live distribution, so that, for example, a plurality of people can participate in a program for a discussion or a meeting. For example, live distribution with more versatile and flexible content can be performed. In the following, a case where the discussion and the meeting are held in this way will be described with a specific example.

First, the communication application program operating on the client terminal 12 side is provided with a function of synthesizing various marks (images such as symbols and figures) on the face image to be transmitted to the user terminal 10 and transmitting it. The case will be described. In this case, the communication window 90 displayed by the communication application program on the client terminal 12 side is in a state as shown in FIG. 21, for example. The communication window 90 shown in FIG. 21 is the same as that shown in FIG. 7, but shows a state in which a functional module that realizes the mark synthesizing function is operating in a communication application program having a module structure. Shows.

At this time, as shown in FIG. 21, a plurality of buttons 92 shown in FIG. 7 are provided around the self-portrait display area 91.
Buttons 94a to 94f are arranged instead of a to 92g. These buttons 94a to 94f correspond to the following functions, respectively.

The circle button 94a, the cross button 94b, the cross button 94c, and the cross button 94d are respectively a "maru" mark, a "cross" mark, and a "cross".
It corresponds to the function of synthesizing a mark and a "shikaku" mark with the face image displayed in the self-portrait display area 91. When these buttons are selected by the user, the mark corresponding to the selected button is displayed. It is combined with the face image.
Further, the face image in which the marks are combined in this way is sent to the user terminal 10. Also, the help button 94e
The return button 94f and the return button 94f have functions corresponding to the help button 92d and the return button 92e shown in FIG. 7, respectively.

At this time, for example, the client terminal 12
The communication window 90 shown in FIG.
When the "maru" button 94a is selected, the "maru" mark is combined with the face image sent from the client terminal 12 and sent to the user terminal 10. The face image sent in this way is acquired by the communication application program on the user terminal 10 side, and as shown in FIG. 22, a “maru” mark is combined in the preview area 71 of the communication window 70. The status is displayed.

Since the face image inserted by the live distribution application program is obtained from the communication application program,
Also in the preview area 53c1 of the main monitor panel window 53c, the face image 100a combined with the "maru" mark is inserted and displayed in the original image. The image in the preview area 53c1 including the face image in which the mark is combined in this way is distributed to each client terminal 12 in the same manner as described above.

In the content distribution system 1, since it is possible to combine various marks with each face image and perform live distribution as described above, for example, in response to the question given by the sender, More than one participant can specify a mark such as "Maru" or "Batsu" to answer, and hold a quiz competition, a conference, or a debate. Programs can be easily created and distributed.

Next, a case will be described in which the text data sent from the client terminal 12 is received by the user terminal 10 with a plurality of face images inserted in the original image. The following description corresponds to the case described above with reference to FIGS. 10 to 12.

[0188] In this case, the text data with reference to the manner described in on the communication application program to be executed at a predetermined client terminal 12 (for example, the text "Hello!") Is input to Figure 10 Then, the text data is sent from the communication application program on the client terminal 12 side to the communication application program on the user terminal 10 side.

Then, as shown in FIG. 23, the communication application program on the user terminal 10 side displays a position in the communication window 70 corresponding to the face image corresponding to the client terminal 12 (or adjacent to the face image). to the position), to display the contents of the acquired text data ( "Hello!").

Also, the live distribution application program acquires this text data from the communication application program, and
As shown in, is displayed in the preview area 53c1 "Hello!" Formed by inserting the text data.

At this time, the live distribution application program imparts a predetermined effect such as enlarged display to the face image (face image 100b in this example) of the user corresponding to the transmission source of the text data to be inserted. It is desirable to do. As a result, the face image 100 in which a plurality of the text data inserted in the image delivered live is inserted
It is possible to easily identify which of the face images of a, 100b, and 100c is generated by the user. The effect given to the face image at this time is not particularly limited to the enlarged display, and it is sufficient if it is possible to easily identify from which user the text data is issued. Is.
As such an effect, for example, an effect of deforming the face image, imparting a predetermined animation effect, or changing a color tone can be cited.

(9) Processing when inserting a plurality of contents Next, when inserting a plurality of contents and performing live distribution as described above, the communication application program executes the user terminal 10 side. A series of processes will be described step by step with reference to the flowchart.

Here, the contents of the memory area secured in the RAM 21 for live distribution in the user terminal 10 on the transmission side in the content distribution system 1 will be described with reference to FIG. As shown in FIG. 24, the contents of this memory area include a memory area 200 used as a work area in the communication application program, a memory area 201 used as a work area in the live distribution application program, and The memory area 202 is shared by both the communication application program and the live distribution application program.

The memory area 200 includes a participant list for storing information on the user (participant) on the client side with which a session has been established in the communication application program, and the client terminal 12.
It is composed of a character preparation memory area and an image preparation memory area for respectively preparing the text data and the image data which are acquired from the program and passed to the live distribution application program. Of these, the participant list is
"Participant 0", "Participant 1", "Participant 2", ...
・ As indicated by, the required information is stored collectively for each participant.

Information about each participant in the participant list includes, for example, a participant ID for identifying each participant, the name of the participant, a face image corresponding to the participant, and the face image changed. A face image change flag indicating whether or not, text data (character) sent from the participant, a character change flag indicating whether or not this character has changed, a mark sent from the participant, this mark changes It is configured by a mark change flag or the like indicating whether or not the mark is changed.

When the communication application program is started by the live distribution application program in the state where the above memory area is secured in the RAM 21 of the user terminal 10, FIG.
As shown in step S100 in the flowchart shown in FIG. This step S
At 100, various data is exchanged with the live distribution application program to prepare for acquiring content such as a face image sent from the client terminal 12. Note that this step S10
The content of the processing in 0 is the same as step S3 shown in FIG.
Since the processing is the same as the processing in 0 to S32, the description in this example is omitted.

Next, in step S101, it is determined whether or not a connection request from the communication application program activated by the client terminal 12 has been received. That is, in step S101, it is determined whether the content such as the face image 100 is transmitted from the client terminal 12. As a result of this determination, if the connection request is received, the process proceeds to step S102, and if not, the process proceeds to step S103.

In step S102, the content such as the face image 100 sent from the client terminal 12 side is added to the preview area 71 of the communication window 70. As a result, new participants are ready to participate in the program sent from the user terminal 10. This step S
Since the content of the process in 102 is the same as the process in steps S40 to S43 shown in FIG. 16, the description in this example is omitted.

In step S103, for example, when the "participate in all" button arranged in the communication window 70 is selected, all the users whose sessions are established by the communication application program are sent to the live distribution application.・
It is determined whether the program has requested that the program be distributed. If the result of this determination is that all are required to participate, the process proceeds to step S104,
If not requested, the process proceeds to step S105.

In step S104, preparations are made for the user (participant) of each client terminal 12 to participate (appear) in the program distributed by the live distribution application program. Note that this step S10
Details of the processing in 4 will be described later.

At step S105, the insertion stop button 73 arranged on the communication window 70 is selected by the user, and so on.
It is determined whether it has been requested to stop the participant's appearance in the program started at 04. As a result of this determination, if the cancellation of the appearance is requested, the processing is performed in step S1.
If not requested, the process proceeds to step S107.

In step S106, step S
Processing for stopping the appearance of the participant in the program started at 104 is performed. The details of the process in step S106 will be described later.

In step S107, it is determined whether or not the content (that is, face image, text, mark, etc.) sent from the client terminal 12 is allowed to change. As a result of this determination, if a change is recognized, the process proceeds to step S108, and if not, the process proceeds to step S109.

In step S108, a process of changing each change flag such as the face image change flag, the character change flag, or the mark change flag shown in FIG. 24 is performed. By the processing in step S108, the content of the content inserted into the live distribution application program will be updated, which will be described in detail later.

In step S109, processing relating to the image update timer is performed. By the processing in this step S109, the content acquired from each client terminal 12 (that is, face image, text, mark, etc.)
Can be passed to the application program for live distribution at a predetermined time interval, and the user terminal 1
It is possible to prevent the processing load at 0 from unnecessarily increasing. The details of the processing in step S109 will be described later.

In step S110, a process of actually delivering the content acquired from each client terminal 12 to the live distribution application program is performed. The details of the process in step S110 will be described later.

[0207] In step S111, it is determined whether or not the user who is the sender operating the user terminal 10 has requested to terminate the communication application program. As a result of this judgment,
When it is requested to end, the communication window 70 is closed and the communication application program is ended. If it is not requested to end, the process is returned to step S101, and the series of processes described above is repeated.

Next, the appearance start processing in step S104 described above will be described with reference to FIG.

When the process in step S104 is started, each change flag such as a face image change flag, a character change flag, or a mark change flag is set (cleared) to an initial state in step S120 shown in FIG. Next, in step S121, an image update message indicating that the image to be inserted should be updated is sent to the application program for live distribution. next,
In step S122, the image update timer is started. The image update timer is a timer for passing the content acquired from the client terminal 12 to the application program for live distribution at a predetermined time interval, and is realized by, for example, a variable whose count value increases at a predetermined time interval. can do. When the process in step S122 is completed, the process proceeds to step S105.

Next, the appearance canceling process in step S106 described above will be described with reference to FIG.

When the processing in step S106 is started, the image update timer is stopped in step S130 shown in FIG. Next, in step S131, each change flag such as a face image change flag, a character change flag, or a mark change flag is cleared. Next, in step S132, a message requesting the insertion of the content to be stopped is output to the application program for live distribution. This step S
After the processing in 132 is completed, step S107
Proceed to.

Next, the change flag process in step S108 will be described with reference to FIG.

When the process in step S108 is started, it is determined in step S140 shown in FIG. 28 whether or not the face image, character, or mark of each participant has changed by referring to the participant list. , Get the participant ID of the changing participant. In the subsequent processing, the participant ID acquired here will be represented as “participant IDx”.

Next, in step S141, it is determined whether or not the face image has changed. If the face image has changed, the process proceeds to step S142. If not, the process proceeds to step S143. Proceed.

[0215] In step S142, the face image change flag corresponding to the participant IDx in the participant list is set. In the following description, "to set a flag" means to change the value of the flag from "0" to "1". After the processing in step S142, the processing proceeds to step S143.

In step S143, it is determined whether or not the character has changed. If the character has changed, the process proceeds to step S144, and if not, the process proceeds to step S147.

In step S144, the character change flag corresponding to the participant IDx in the participant list is set. Next, in step S145, the live update application program is notified of a text update message requesting that text data (text) has been acquired from the participant and requesting that the text be combined with the distribution image. To send. Next, in step S146, the application for live distribution ...
Send an image update message to the program. After the process in step S146 is completed, the process proceeds to step S147.

In step S147, it is determined whether or not the mark has changed. If the mark has changed, the process proceeds to step S148. If not, the change flag process is completed and the process is executed. Proceed to step S109.

In step S148, the mark change flag corresponding to the participant IDx in the participant list is set. Next, in step S149, the face image change flag corresponding to the participant IDx in the participant list is set. After the process in step S149 is completed, the change flag process is completed and the process proceeds to step S109.

Next, the image update timer processing in step S109 described above will be described with reference to FIG.

When the process in step S109 is started, it is determined in step S150 shown in FIG. 29 whether or not the image update timer is started. As a result, if the image has not been started, the image to be inserted into the live distribution application program is not updated, the image update timer process is completed, and the process proceeds to step S110. If the image update timer has started, the process proceeds to step S151.
Proceed to.

In step S151, by referring to the value of the image update timer, it is determined whether or not a predetermined time (1 second in this example) has elapsed since the image update timer process performed last time. As a result, if it has not elapsed, it is determined that the image to be inserted into the live distribution application program is not updated, the image update timer process is completed, and the process proceeds to step S110. If the predetermined time has elapsed, the process proceeds to step S152.

[0223] In step S152, it is determined by referring to the participant list whether or not there is a participant for whom the face image change flag is set. As a result, if it exists, the process proceeds to step S153, and if it does not exist, the process proceeds to step S155.

In step S153, an image update message is sent to the live distribution application program. Next, in step S154, the face image change flag in the participant list is cleared.
After the processing in step S154 is completed, the process proceeds to step S155.

In step S155, it is determined whether or not a predetermined time (10 seconds in this example) has elapsed since the character change flag was raised. As a result, if it has elapsed, the process proceeds to step S156, and if not, the process proceeds to step S110. This step S
By performing the processing in 155, when a character is inserted into the original image to be live-distributed, this character can be displayed for a predetermined time (10 seconds).

In step S156, an image update message is sent to the live distribution application program. Next, in step S157,
Clear the character change flag in the participant list. After the process in step S157 is completed, the process proceeds to step S110.

Next, the processing in step S110 described above will be described with reference to FIG.

When the processing in step S110 is started, in step S160 shown in FIG. 30, the size (the number of characters) of the text (characters) to be inserted into the distribution image from the application program for live distribution or the insertion into the distribution image is performed. It is determined whether or not an inquiry has been made regarding the size of the image to be processed (for example, the number of vertical and horizontal pixels). As a result, if an inquiry has been made, the process proceeds to step S161, and if no inquiry has been made, the process proceeds to step S163.

In step S161, a process of preparing a character or an image to be passed to the live distribution application program is performed. After this process is complete,
The process proceeds to step S162. The details of the processing in step S161 will be described later.

[0230] In step S162, the live distribution application program is notified of the number of characters to be passed in the subsequent processing or the size of the image. As a result, the live distribution application program can know the number of characters and the size of the image in advance. After the process in step S162 is completed, the process proceeds to step S163.

In step S163, the address information (information indicating the address of the shared memory, etc.) is obtained from the live distribution application program, so that the communication application program and the live distribution application program can communicate with each other. It is determined whether or not the address of the shared memory used when delivering contents such as characters and images is set. As a result, if the address is set, the process proceeds to step S164, and if the address is not set, the content transfer process is stopped and the process proceeds to step S111.

[0232] In step S164, the shared memory is made accessible (open) based on the set address. Next, in step S165, the character or image to be transferred from the communication application program to the live distribution application program is transferred (copied) from the character preparation memory or the image preparation memory to the shared memory. Next, in step S166, access to the shared memory opened in step S164 is stopped (closed).

In this way, by copying the contents such as characters and images to be passed to the live distribution application program to the shared memory, the live distribution application program refers to this shared memory, It is possible to acquire the content that should be acquired from the communication application program. The communication application program advances the process to step S111 after the process in step S164 is completed.

Next, the processing in step S161 described above will be described with reference to FIG.

When the processing in step S161 is started, it is determined in step S170 shown in FIG. 31 whether or not the preparation for the delivery of characters to the live distribution application program is ready. As a result, if the preparation is completed, the processing is performed in step S1.
The process proceeds to 71, and when the preparation is not completed or when it is not necessary to pass the character, the process proceeds to step S172.

In step S171, the text (characters) to be passed to the live distribution application program is copied to the character preparation memory. After the processing in step S171, the processing is performed in step S16.
Proceed to 2.

In step S172, the value of the variable i used for performing the subsequent series of processing for each of the participants appearing in the distribution image is set to "0". Next, in step S173, a face image corresponding to the i-th participant (hereinafter, simply referred to as participant i) is acquired by referring to the participant list, and this face image is stored in the image preparation memory. Store.

Next, in step S174, it is determined by referring to the participant list whether or not the character change flag corresponding to the participant i is set. As a result,
If the character change flag is set, the process proceeds to step S.
If the character change flag is not set, the process proceeds to step S176.

In step S175, the face image of the participant i stored in the image preparation memory is enlarged. As a result of the processing in step S175 shown in FIG.
As shown in FIG. 3, the image of the participant who is the transmission source of the character to be inserted is enlarged and displayed. After the process in step S175, the process is performed in step S17.
Proceed to 6.

In step S176, it is determined whether or not the mark change flag corresponding to the participant i is set by referring to the participant list. As a result, if the mark change flag is set, the process proceeds to step S.
If the mark change flag is not set, the process proceeds to step S178.

In step S177, a process of drawing a predetermined mark on the face image of the participant i stored in the image preparation memory is performed. After the processing in step S177, the process proceeds to step S178.

In step S178, a process of adding "1" to the current value of variable i is performed. Next, step S
In 179, it is determined whether or not the value of the variable i is less than the number of participants who appear in the distributed program, that is, whether or not the processing from step S173 is completed for all the participants. If not, the process proceeds to step S17.
Returning to step 3, the process from step S173 is also performed for the next participant. If the processing has been completed for all participants, the process proceeds to step S162.

In the content distribution system 1, the communication application program performs the above-described processing to receive the content acquired from the plurality of client terminals 12 by the communication application program with respect to the live distribution application program. Can be passed. As a result, a plurality of contents can be inserted in the distribution image on the live distribution application program side.

In the above description, the images of all the participants are combined on the communication application program side using the image preparation memory and then passed to the live distribution application program side. Therefore, on the side of the application program for live distribution, the process of synthesizing the images of all the participants that have been synthesized in advance with the original image, and the text (characters) such as “Good morning!” Acquired from the application program for communication. Two image synthesizing processes, that is, a process of synthesizing with the original image are performed.

However, it goes without saying that in the content distribution system 1, it is optional which of the communication application program and the live distribution application program performs the combining process.

(10) Summary As described above, in the content distribution system 1, the content distributed from the client terminal 12 can be transmitted by the close cooperation between the live distribution application program and the communication application program. , Will be inserted into the content delivered live from the user terminal 10. In addition,
The above-mentioned series of processing is performed by the live distribution application
Programs and communication applications ·
The present invention is not limited to being realized by being executed in cooperation with a program, and may be realized by a single application program, for example.

In the content distribution system 1 described above, the system on the content distribution side includes the user terminal 10 that receives the content transmitted from the client terminal 12 and generates the content to be distributed live.
Although the live distribution content generated by the user terminal 10 is configured with the streaming distribution server 13 that distributes the content to the client terminal 12, the user terminal 10
A plurality of client terminals 12a, 12b, 12c ,.
.. may be configured to have a server function for delivering content that is delivered live to. Even in this case, the moving image provided from the client terminal 12,
Content such as a still image or text can be freely used by the user terminal 10 on the transmission side to distribute the content. Therefore, opinions and claims can be exchanged very smoothly with the viewer of the delivered content.

Further, a software program that causes a desired electronic device to execute a procedure corresponding to the processing executed by the communication application program or the live distribution application program described above is provided by being stored in various recording media. May be

[0249]

Further, according to the present invention, the content transmitted from the viewing device can be inserted into the content to be transmitted by the transmitting device, and this content can be distributed. Therefore, the content such as the moving image, the still image, or the text provided from the viewing device can be freely used by the transmitting device to distribute the content. For this reason, opinions and claims can be exchanged extremely smoothly between the viewer and sender of the distributed content. Further, in the present invention, from another device to the transmitting device,
The content is sent directly. Therefore, the content inserted into the distributed content can be exchanged extremely smoothly.

Therefore, according to the present invention, it is possible to easily distribute the content in such a manner that a moving image from another provider located far from the sender is captured in the live distribution program and relayed to each place. be able to.

[Brief description of drawings]

FIG. 1 is a schematic diagram showing an overall basic configuration of a content distribution system shown as an embodiment of the present invention.

FIG. 2 is a functional block diagram showing a configuration of a user terminal used in the content distribution system.

FIG. 3 is a schematic diagram showing a main window displayed by a live distribution application program executed on the user terminal side in the content distribution system.

FIG. 4 is a schematic diagram showing a connection relationship between a user terminal and a client terminal in the content distribution system.

FIG. 5 is a schematic view showing a display screen in a state where a communication distribution application program is activated from a live distribution application program executed on the user terminal side in the content distribution system.

FIG. 6 is a schematic diagram showing an example of a display screen on the client terminal side for viewing content distributed by the content distribution system.

FIG. 7 is a schematic diagram showing an example of a communication window created by a communication application program executed on the client terminal side in the content distribution system.

FIG. 8 is a schematic diagram showing an example of a display screen when content transmitted from the client terminal side is inserted at the user terminal side in the content distribution system.

FIG. 9 is a schematic diagram showing an example of a display screen on the client terminal side for viewing the content in which the content has been inserted on the user terminal side in the content distribution system.

FIG. 10 is a communication application on the client terminal side in the content distribution system.
It is a schematic diagram which shows the example of a display screen in the state where the text data was input by the program.

FIG. 11 is a schematic diagram showing an example of a display screen when the text data input on the client terminal side in the content distribution system is acquired and used on the user terminal side.

FIG. 12 is a schematic diagram showing an example of a display screen when the content in which the text data is inserted is viewed on the client terminal side in the content distribution system.

FIG. 13 is a flowchart showing an example of an execution procedure by a communication application program executed on the user terminal side in the content distribution system.

FIG. 14 is a flowchart showing an example of an execution procedure when the communication application program prepares to establish a session in the content distribution system.

FIG. 15 is a schematic diagram showing an example of sender identification information (connection data) used in a communication application program in the content distribution system.

FIG. 16 is a flowchart showing an example of an execution procedure when a participant addition process is performed by a communication application program in the content distribution system.

FIG. 17 is a flowchart showing an example of an execution procedure when performing a performer selection process by a communication application program in the content distribution system.

FIG. 18 is a flowchart showing an example of an execution procedure in the case where a communication application program performs a delivery process of characters, images, and the like to the live distribution application program in the content distribution system.

FIG. 19 is a flowchart showing an example of an execution procedure when content is inserted by a live distribution application program in the content distribution system.

FIG. 20 is a schematic diagram showing an example of a display screen when a plurality of contents are inserted in the same content distribution system.

FIG. 21 is a schematic diagram showing an example of a communication window displayed on the display screen of the client terminal when a plurality of contents are inserted in the content distribution system.

FIG. 22 is a schematic diagram showing an example of a display screen when a client terminal side is requested to combine a mark with a face image when inserting a plurality of contents in the same content distribution system.

FIG. 23 is a schematic diagram showing an example of a display screen in the case where a plurality of contents are inserted in the same content distribution system and characters input by a specific participant are combined with an original image and distributed.

FIG. 24 is a schematic diagram showing the contents of a memory area secured in the RAM of the user terminal when inserting a plurality of contents in the same content distribution system.

FIG. 25 is a flowchart showing an example of processing performed on the live distribution application program side when a plurality of contents are inserted in the content distribution system.

FIG. 26 is a flowchart showing appearance start processing of the processing performed on the live distribution application program side when a plurality of contents are inserted in the content distribution system.

FIG. 27 is a flowchart showing appearance canceling processing among the processing performed on the live distribution application program side when a plurality of contents are inserted in the same content distribution system.

FIG. 28 is a flowchart showing change flag processing of the processing performed on the live distribution application program side when a plurality of contents are inserted in the content distribution system.

FIG. 29 is a flowchart showing an image update timer process of the processes performed on the live distribution application program side when a plurality of contents are inserted in the content distribution system.

FIG. 30 is a flowchart showing a process of handing over contents to the live distribution application program among the processes performed on the side of the live distribution application program when inserting a plurality of contents in the content distribution system. Is.

FIG. 31 is a diagram showing a process of preparing content to be delivered to the live distribution application program, among the processes performed by the live distribution application program when inserting a plurality of contents in the content distribution system. It is a flowchart shown.

[Explanation of symbols]

1 content distribution system, 10 user terminals, 11
Internet, 12 client terminals, 13 streaming distribution server, 14 server use reservation management center, 15 public line network, 16 server connection dedicated network, 29 DV camera

Continued front page    (72) Inventor Yasushi Miyajima             6-735 Kita-Shinagawa, Shinagawa-ku, Tokyo Soni             -Inside the corporation (72) Inventor Keigo Ihara             6-735 Kita-Shinagawa, Shinagawa-ku, Tokyo Soni             -Inside the corporation (72) Inventor Takanori Nishimura             6-735 Kita-Shinagawa, Shinagawa-ku, Tokyo Soni             -Inside the corporation (72) Inventor Junko Fukuda             6-735 Kita-Shinagawa, Shinagawa-ku, Tokyo Soni             -Inside the corporation F-term (reference) 5C064 AA02 AC04 AC13 AD08 AD14                       BA07 BB10 BC07 BC18 BC23                       BD02 BD03 BD08

Claims (18)

[Claims] 1. A transmission device connected via a network to a server device for distributing contents and a plurality of viewing devices for receiving contents distributed from the server device, the device being another device via the network. Communication means for communicating with the viewing device, and control means for generating content delivered to the viewing device via the server device, wherein the control means is delivered to the viewing device via the server device. First
A process of acquiring content and a second process of delivering the content to the viewing device via the server device
A process of directly acquiring the content from another device connected to the network, and using the first content and the second content,
A transmitting device, which executes a process of generating a transmission content to be transmitted to the server device and a process of transmitting the transmission content to the server device via the communication means. 2. The control means executes a process of acquiring the second content from the viewing device that receives content distributed from the server device connected to the network, via the communication means. The transmitter according to claim 1, wherein 3. The control means transmits caller identification information for specifying the transmission device on a network to the server device, and from another device that has acquired the caller identification information from the server device. The transmission device according to claim 1, wherein a connection is established with the other device based on a connection request, and a process of receiving the second content from the other device with which the connection is established is executed. 4. The transmission device according to claim 1, wherein the first content and the second content are moving images, still images, audio information, or character information. 5. The transmission device is connected to an image capturing device that captures a moving image or a still image, and the control unit performs a process of acquiring the first content from the image capturing device. The transmitting device according to claim 1. 6. The transmission device comprises image capturing means for capturing a moving image or a still image, and the control means performs a process of acquiring the first content from the image capturing means. The transmission device according to claim 1. 7. The control means performs a process of receiving a moving image or a still image captured by an image capturing device connected to the other device as the second content. Transmitter. 8. The control unit performs a process of receiving, as the second content, a moving image or a still image captured by an image capturing device connected to the viewing device. Transmitter. 9. The control means obtains a moving image or a still image as the first content, and the first content sends the moving image or the still image within a predetermined area of the moving image or the still image from the other device. The transmitting device according to claim 1, wherein a process of inserting the second content is performed. 10. The control unit obtains contents from each of a plurality of other devices, and selects the content selected from among the contents received from each of the plurality of other devices as the second content. The transmitting device according to claim 1, wherein the transmitting device performs a generating process. 11. A transmission method for delivering content to a plurality of viewing devices via a server device for delivering content, wherein the first delivery method is delivered to the viewing device via the server device.
A step of acquiring content, and a second step of being delivered to the viewing device via the server device
Acquiring content from another device connected to the network, using the first content and the second content,
A transmission method comprising: a step of generating transmission contents to be transmitted to the server device; and a step of transmitting the transmission contents to the server device. 12. A program for causing a device connected via a network to a server device for distributing content and a plurality of viewing devices for receiving the content distributed from the server device to execute a content transmission process. First distributed to the viewing device via the server device
A process of controlling to acquire the content, and a second process of delivering the content to the viewing device via the server device.
Using a process of controlling acquisition of content from another device connected to the network, and using the first content and the second content,
A program comprising: a process of controlling generation of a transmission content to be transmitted to the server device, and a process of performing control of transmitting the transmission content to the server device. 13. A content distribution system connected to a plurality of viewing devices via a network, wherein the distribution means distributes content to the plurality of viewing devices, and distributes to the viewing devices via the distribution means. A generation unit configured to generate content, wherein the generation unit acquires the first content distributed to the viewing device via the distribution unit, and is distributed to the viewing device via the distribution unit. A second content that is directly obtained from another device connected to the network, using the first content and the second content,
And a process of generating distribution content to be distributed to the viewing device via the distribution unit, the distribution unit executing a process of distributing the distribution content generated by the generation unit to the plurality of viewing devices. A content distribution system characterized by: 14. The generation unit and the distribution unit are connected via the network, and the generation unit executes a process of transmitting the distribution content to the distribution unit via the network. 14. The content distribution system according to claim 13, wherein: 15. The content distribution system according to claim 13, wherein the generation unit executes a process of acquiring the second content from the viewing device that receives the content distributed by the distribution unit. 16. The generation means distributes, through the distribution means, sender identification information for specifying a sender on the network of the distribution content, and acquires the sender identification information from the distribution means. The process of establishing a connection with the other device based on a connection request from the other device that has established the connection, and executing the process of receiving the second content from the other device with which the connection has been established. Content distribution system described. 17. A content distribution system having a distribution means for distributing contents to a plurality of viewing devices, and a generation means for generating contents to be distributed to the viewing devices via the distribution means. A method of acquiring first content distributed to the viewing device via the distribution means, and connecting a second content distributed to the viewing device via the distribution means to the network. And a step of directly acquiring from another device, using the first content and the second content,
Content comprising: a step of generating distribution content to be distributed to the viewing device via the distribution means; and a step of distributing the distribution content generated by the generation means to the plurality of viewing devices. Delivery method. 18. A program for causing a system having distribution means for distributing contents to a plurality of viewing devices and generating means for generating contents to be distributed to the viewing devices via the distribution means to execute a distribution process of contents. And a process of acquiring the first content distributed to the viewing device via the distribution means, and the second content distributed to the viewing device via the distribution means to the network. Using the process of directly obtaining from another device and the first content and the second content,
A program comprising: a process of generating distribution content to be distributed to the viewing device via the distribution unit; and a process of distributing the distribution content generated by the generation unit to the plurality of viewing devices. .