JP7152778B2 - Information processing system - Google Patents

Description

The present invention relates to an information processing system.

2. Description of the Related Art Conventionally, some teleconferencing systems and moving image distribution services distribute image data including moving images via the Internet.
Here, information that can receive from the management terminal a selection of information to be distributed to other screen sharing terminals (hereinafter referred to as "client terminals") from information input to one or more screen sharing terminals. A processing system has been proposed (see Patent Document 1, for example).

In conventional technologies including the technology described in Patent Document 1, when sharing a plurality of images (images sent from a plurality of users or images of a plurality of documents sent from a single user), the client terminal , an image selected by the management terminal can be displayed.

JP 2018-063662 A

However, as a method of providing image data to users, there has been a demand in recent years for a method that improves convenience for users and reduces costs. are not adequately addressed. In addition, when sharing (distributing) images to a large number of users while improving convenience for users, the cost may increase.
That is, in the conventional technology including the technology described in Patent Document 1, the user cannot display any image in any arrangement on the screen to be browsed, and it can be said that the convenience for the user is high. I didn't.

SUMMARY OF THE INVENTION The present invention has been made in view of such circumstances, and an object of the present invention is to improve convenience for users when providing images to users and to reduce costs associated with image transmission. do.

In order to achieve the above object, an information processing system according to one aspect of the present invention includes:
In an information processing system including a first information processing device that generates data of an image to be presented to a user and a second information processing device that is used when presenting the image to the user,
The first information processing device is
a first acquisition means for acquiring data of a plurality of images as a group of first data;
a first generating means for generating image data including each of the plurality of images in a classifiable manner as second data;
transmission control means for executing control to transmit the second data to the second information processing device;
with
The second information processing device is
a second obtaining means for obtaining the second data transmitted from the first information processing device;
segmentation means for segmenting each of the plurality of images based on the second data;
a third obtaining means for obtaining, as layout information, information indicating how to arrange each of the one or more images to be displayed, with one or more images among the plurality of images to be displayed;
second generation means for generating, as fourth data, image data in which the one or more images to be displayed are arranged based on the arrangement information;
presentation control means for performing control to present the image corresponding to the fourth data to the user;
Prepare.

A program corresponding to the information processing system of one aspect of the present invention is also provided as a program of one aspect of the present invention.

According to the present invention, it is possible to improve the convenience for the user when providing the user with an image, and reduce the cost associated with the transmission of the image.

1 is a block diagram showing the configuration of an information processing system according to one embodiment of the present invention; FIG. 2 is a block diagram showing an example of a hardware configuration of a server in the information processing system of FIG. 1; FIG. 2 is a functional block diagram showing an example of functional configurations of a server and a viewer terminal in the information processing system of FIG. 1; FIG. 4 is a diagram showing an example of image provision in this service provided by the server of FIG. 3. FIG. 4 is a diagram showing an example of arrangement of a main image and sub-images in data of a classifiable image transmitted by the server of FIG. 3; FIG. 6 is a diagram showing an example different from FIG. 5 in the arrangement of the main image and sub-images in the data of the classifiable image transmitted by the server of FIG. 3; FIG. 4 is a flowchart illustrating an example of the flow of image transmission processing executed by a server having the functional configuration of FIG. 3; 4 is a flowchart illustrating an example of the flow of image display processing executed by the viewer terminal having the functional configuration of FIG. 3;

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

In the following description, simply calling an "image" includes both a "moving image" and a "still image".
Also, "moving image" includes images displayed by each of the following first to third processes.
The first process refers to a process of continuously switching and displaying a series of still images consisting of a plurality of images over time for each movement of an object (for example, an image of a natural person) in a planar image (2D image). . Specifically, for example, a two-dimensional animation, a so-called flipbook-like process, corresponds to the first process.
The second process is a process of setting a motion corresponding to each action of an object (for example, an image of a natural person) in a stereoscopic image (3D model image), and displaying the motion while changing it over time. . Specifically, for example, three-dimensional animation corresponds to the second process.
The third process is a process of preparing images (that is, moving images) corresponding to each action of an object (for example, an image of a natural person) and playing the images with the lapse of time.
Here, a "video (that is, moving image)" is composed of images such as a plurality of frames and fields (hereinafter referred to as "unit images"). In the following example, the unit image is assumed to be a frame.

In a service using an information processing system to which the present invention is applied (hereinafter referred to as "this service"), a distributor can perform live distribution.
Note that live distribution refers to distribution of various contents including chat in real time. Also, a distributor is a subject who performs live distribution, and in this embodiment, a person who chats with one or more viewers. A viewer is a person who views various contents (including chat) including live-delivered images.
That is, in the present embodiment, the content of the chat is not limited to the viewer who chats with the distributor, but the content of the chat includes an image, and all viewers who are not limited to the viewer who chats. will also be live-streamed. That is, the chat in this embodiment is a so-called public chat.

Note that in the present embodiment, the distributor conducts a public chat with one viewer out of n viewers (where n is an integer value of 1 or more). Hereinafter, one viewer who conducts a public chat with the broadcaster is referred to as a "chat target person".
Also, as will be described later, the method of selecting a chat target person is not particularly limited, but in the present embodiment, a method of selecting based on an instruction from a broadcaster is adopted. That is, it is assumed that the distributor selects a chat target person from one or more viewers out of n viewers. A viewer who is a candidate to be selected as a chat target person is hereinafter referred to as a “chat candidate”.

Here, in this specification, chat refers to communication in which messages or the like are exchanged in real time between a plurality of terminals via a network including the Internet as necessary. Characters, voices, images (still images and moving images), and the like can be used as means of communication. Chats using each of these means are called "text chat", "voice chat" and "video chat" respectively. Although video chat is assumed in the following description, it is not particularly limited to this, and text chat and voice chat can also be adopted. Here, voice chat can also include so-called "telephone". In other words, if the network is a carrier network managed by a mobile phone carrier, so-called "telephone" can also be understood as voice chat. Also, a video chat and a text chat may be combined, and a message text may be superimposed on an image and displayed by scrolling.

FIG. 1 is a block diagram showing the configuration of an information processing system according to one embodiment of the present invention.

In the information processing system shown in FIG. 1, a server 1, a distributor terminal 2, and n audience terminals 3-1 to 3-n are interconnected via a predetermined network N such as the Internet. It is

The server 1 provides an environment for live distribution to the distributor terminal 2 and the viewer terminals 3-1 to 3-n.

As shown in FIG. 1, the distributor terminal 2 is operated by the distributor U1.
Also, the viewer terminals 3-1 to 3-n are operated by the viewers U2-1 to U2-n, respectively. Note that when there is no need to distinguish between the viewer terminals 3-1 to 3-n individually, they are collectively referred to as the "viewer terminal 3". Also, when collectively referred to as "viewer terminal 3", viewers U2-1 to U2-n are collectively referred to as "viewer U2".

FIG. 2 is a block diagram showing an example of a hardware configuration of a server in the information processing system of FIG. 1. As shown in FIG.

The server 1 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a bus 14, an input/output interface 15, an output section 16, and an input section 17. , a storage unit 18 , a communication unit 19 , and a drive 20 .

The CPU 11 executes various processes according to programs recorded in the ROM 12 or programs loaded from the storage unit 18 to the RAM 13 .
The RAM 13 also stores data necessary for the CPU 11 to execute various processes.

The CPU 11 , ROM 12 and RAM 13 are interconnected via a bus 14 . An input/output interface 15 is also connected to this bus 14 . An output unit 16 , an input unit 17 , a storage unit 18 , a communication unit 19 and a drive 20 are connected to the input/output interface 15 .

The output unit 16 includes a display, a speaker, and the like, and outputs various information as images and sounds.
The input unit 17 is composed of a keyboard, a mouse, etc., and inputs various kinds of information.

The storage unit 18 is composed of a hard disk, a DRAM (Dynamic Random Access Memory), or the like, and stores various data.
The communication unit 19 communicates with other devices (distributor terminal 2 and viewer terminal 3 in the example of FIG. 1) via a network N including the Internet.

A removable medium 31 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory is mounted in the drive 20 as appropriate. A program read from the removable medium 31 by the drive 20 is installed in the storage unit 18 as necessary.
The removable medium 31 can also store various data stored in the storage unit 18 in the same manner as the storage unit 18 .

Although not shown, the distributor terminal 2 and the viewer terminal 3 of the information processing system of FIG. 1 have basically the same hardware configuration as that shown in FIG. However, the distributor terminal 2 or the viewer terminal 3 may have an imaging unit such as a camera as one of the input units, or may have a touch panel.

The following series of processes can be executed by cooperation of various hardware and software of the server 1, the distributor terminal 2, and the viewer terminal 3 of FIG.

In this embodiment, the server 1 extracts one or more chat candidates from the viewers viewing the public chat on the viewer terminal 3 by any method. The server 1 determines a chat target person based on an instruction from the distributor terminal 2, for example, from among the extracted one or more chat candidates.
The server 1 executes various controls necessary for conducting a public chat between the viewer terminal 3 of the chat target determined in this manner and the distributor terminal 2, and performs predetermined content including the contents of the chat. are distributed live to all viewer terminals 3-1 to 3-n.

Here, images (mainly face images of viewers) captured by the viewer terminal 3 of one or more chat candidates who have not been selected as chat candidates are displayed in a list on at least the distributor terminal 2 and distributed. The person U1 is in a state where he can be selected as a chat target person at any time.
By doing so, for example, in live distribution in which the distributor U1 is an idol talent, the distributor U1 can freely switch dialogues with arbitrary fans (viewers U2) and include them in the live distribution. can be done. As a result, this service will create a sense of anticipation among the watching fans that they may be selected someday, and this will serve as an opportunity for more fans to become viewers of the live distribution.

In this way, the information processing system includes, as a series of processes for live distribution, a series of processes (hereinafter referred to as "image transmission A series of processes (hereinafter referred to as "image display process") relating to the stage where the user terminal 3 actually displays an image to a viewer can be executed.
In order to realize such image transmission processing and image display processing, the information processing system in FIG. 1 has a functional configuration as shown in FIG.
That is, FIG. 3 is a functional block diagram showing an example of the functional configuration of the server and the viewer terminal in the information processing system of FIG.

In the description of FIG. 3 and subsequent figures, only the chat target person, one chat candidate, and one general viewer among the viewers U2 will be referred to for easy understanding. A general viewer means a viewer U2 who is neither a chat target nor a chat candidate.
Among the viewer terminals 3-1 to 3-n, the viewer terminal 3 operated by the chat target person will be described as the viewer terminal 3-a (a is an arbitrary integer value from 1 to n). . In this case, the viewer terminal 3-a is operated by the viewer U2-a. Therefore, the viewer U2-a is hereinafter referred to as "chat target person U2-a".
On the other hand, among the viewer terminals 3-1 to 3-n, the viewer terminal 3 operated by the chat candidate is the viewer terminal 3-b (b is an integer value different from a and Any integer value of ). In this case, the viewer terminal 3-b is operated by the viewer U2-b. Therefore, the viewer U2-b is hereinafter referred to as "chat candidate U2-b".
Further, among the viewer terminals 3-1 to 3-n, the viewer terminal 3 operated by the general viewer is designated as the viewer terminal 3-c (c is an integer value different from a and b, any integer value of n). Therefore, the viewer U2-c is hereinafter referred to as "general viewer U2-c".

In FIG. 3, to simplify the explanation, it is assumed that the chat target person U2-a and the chat candidate U2-b have already been selected among the viewers. It should be noted that the distributor U1, chat target person U2-a, and chat candidate U2-b mainly use images of themselves for chat.
The viewer terminal 3-c has the same configuration as in FIG. 2, but only the CPU 311, communication section 312, input section 313, and output section 314 are shown in FIG.

When the image transmission process is executed, in the CPU 11 of the server 1, the image data acquisition unit 111, the distributor presentation image control unit 112, the chat target person presentation image control unit 113, and the chat candidate presentation image control unit 114 and the viewer presentation image control unit 115 function.

Further, when the image display process is executed, the CPU 311 of the viewer terminal 3-c includes a classifiable image data acquisition unit 321, a classifiable image data region dividing unit 322, and a sub-image arrangement information acquiring unit 323. , the display image data generation unit 324 and the display control unit 325 function.

The distributor terminal 2 takes an image of the distributor U1, and transmits data of an image obtained as a result (an image including the distributor U1 as a subject, hereinafter referred to as a “distributor image”) to the server 1 .

The viewer terminal 3-a captures an image of the chat target person U2-a, and an image obtained as a result (an image including the chat target person U2-a as a subject, hereinafter referred to as a "chat target person image"). Send data to server 1.

The viewer terminal 3-b captures an image of the chat candidate U2-b, and an image obtained as a result (an image including the chat candidate U2-b as a subject, hereinafter referred to as a “chat candidate image”). Send data to server 1.

The image data acquisition unit 111 acquires data of a plurality of images as a group of image data (hereinafter referred to as an “image data group”).
The image data acquisition section 111 has a main image data acquisition section 121 and a sub image data acquisition section 122 .

The main image data acquisition unit 121 acquires the data of the distributor image transmitted from the distributor terminal 2 as the data of the main image.

The sub-image data acquisition unit 122 acquires the chat target person image data transmitted from the viewer terminal 3-a as sub-image data.
Further, the sub-image data acquiring unit 122 acquires the chat candidate image data transmitted from the viewer terminal 3-b as sub-image data.

Specifically, for example, in a live distribution in which an idol celebrity or the like is the distributor U1, an image (distributor image) containing the idol talent as a subject is displayed as the main image on the screen viewed by the general viewer U2-c. displayed large. On the screen viewed by the general viewer U2-c, an image including a fan who chats with the idol talent as a subject (chat target person image) is displayed as a small sub image superimposed on the main image. In addition, on the screen viewed by the general viewer U2-c, an image (chat candidate image) including a fan who is a candidate to be selected as a chat target person as a subject is superimposed on the main image as a sub image and displayed in a small size. be.
That is, the main image is an image that is mainly displayed on the screen of the viewer terminal 3-c. A sub-image is an image that is superimposed on the main image and displayed on the screen of the viewer terminal 3-c. That is, on the screen of the viewer terminal 3-c, the chat target person image and the chat candidate image, which are sub-images, are superimposed on the distributor image, which is the main image, and displayed.
An example of superimposing a sub-image on such a main image will be described later with reference to FIG.

As described above, the image data acquisition unit 111 acquires main image data and one or more sub-image data as a group of image data.

Based on the group of image data acquired by the image data acquisition unit 111, the distributor presentation image control unit 112 controls transmission of image data to be presented to the distributor U1.

That is, for example, the distributor presentation image control unit 112 transmits the data of the chat target person image to the distributor terminal 2 . The distributor terminal 2 presents the chat target person image to the distributor U1 based on the chat target person image data transmitted from the server 1 . As a result, the distributor U1 can have a video chat with the chat target person U2-a.
Also, for example, the distributor presentation image control unit 112 transmits the data of the chat candidate image to the distributor terminal 2 . The distributor terminal 2 presents the chat candidate image to the distributor U1 based on the chat candidate image data transmitted from the server 1 . As a result, the broadcaster U1 can switch the chat target person (new chat target person U2-a) to the chat candidate U2-b at a desired timing.

When transmitting a group of image data to the distributor terminal 2, the distributor presentation image control unit 112 transmits each image data constituting the group of image data to the distributor terminal 2 via each connection related to streaming. can be used to send.

Here, streaming is a method (technique) for transmitting and receiving content data such as audio and moving images mainly via a network such as the Internet. For example, an information processing apparatus that receives content data in a streaming manner downloads and reproduces content data in parallel.

Also, when content data is transmitted and received by streaming, it is generally necessary to create a "connection" between an information processing device that transmits content data and an information processing device that receives content data. be.
The term "connection" used in this specification is a broad concept that includes the flow of data transmitted and received in a streaming manner. Specifically, for example, a connection includes a streaming data flow in connectionless communication using User Datagram Protocol (hereinafter referred to as “UDP”).

Also, a connection is generated for each set of one transmitting device and one receiving device and for each content (for example, one moving image). Although the details will be described later, when creating connections to a large number of terminals, the connections may be created using a mechanism of a Content Delivery Network (hereinafter referred to as "CDN").

For example, the distributor presentation image control unit 112 can transmit the data of the chat target person image and the chat candidate image to the distributor terminal 2 using each connection.
That is, although the details will be described later, the distributor terminal 2 acquires the data of the chat target person image and the chat candidate image using each of the plurality of connections. In this case, the connection for chat target person image data and the connection for chat candidate image data are separate connections. In other words, the distributor terminal 2 does not use one connection to receive data of one image in which multiple images are superimposed.
Thereby, the distributor U1 can display the chat target person image and the chat candidate image in an arbitrary arrangement. Further, for the distributor U1, the chat target person image and the chat candidate image are presented without impairing real-time performance.

Based on the group of image data acquired by the image data acquisition unit 111, the chat target person presentation image control unit 113 executes control for transmitting image data to be presented to the chat target person U2-a.

That is, for example, the chat target person presentation image control unit 113 transmits the data of the distributor image to the viewer terminal 3-a. The viewer terminal 3-a presents the distributor image to the chat target person U2-a based on the data of the distributor image transmitted from the server 1. FIG. This allows chat target person U2-a to have a video chat with distributor U1.
Further, for example, the chat target person presentation image control unit 113 transmits the data of the chat candidate image to the viewer terminal 3-a. The viewer terminal 3-a presents the chat candidate image to the chat target person U2-a based on the chat candidate image data transmitted from the server 1. FIG. This allows the chat target person U2-a to grasp the state of the chat candidate U3-b.
At this time, similarly to the screen viewed by the general viewer U2-c described above, an image (chat candidate image) including a fan who is a candidate to be selected as a chat target person as a subject is used as a sub image as a main image (distribution person image) and displayed in a small size. That is, for example, on the screen of the viewer terminal 3-a viewed by the chat target person U2-a, a chat candidate image, which is a sub-image, is superimposed on a distributor image, which is a main image, and displayed.

Based on the group of image data acquired by the image data acquiring unit 111, the chat candidate presentation image control unit 114 controls transmission of image data to be presented to the chat candidate U3-b.

That is, for example, the chat candidate presentation image control unit 114 transmits the data of the distributor image and the data of the chat target person image to the viewer terminal 3-b. The viewer terminal 3-b presents the distributor image and the chat candidate image to the chat candidate U3-b based on the data of the distributor image and the data of the chat candidate image transmitted from the server 1. FIG. This allows the chat target person U2-b to grasp how the broadcaster U1 and the chat target person U2-a are having a video chat. As a result, the chat candidate U3-b watches with anticipation that "it may be selected someday", and prepares the contents of the video chat in case of being selected as a new chat target person U2-a. can be grasped.

The viewer presentation image control unit 115 generates an image to be presented to the general viewer U2-c based on the group of image data acquired by the image data acquisition unit 111, and executes control for transmission.
The viewer presentation image control unit 115 has a classifiable image data generation unit 131 and a classifiable image data transmission control unit 132 .

The classifiable image data generation unit 131 generates image data containing each of the plurality of images contained in the group of image data acquired by the image data acquisition unit 111 in a classifiable manner as classifiable image data.

Here, although the details will be described later, an example in which a plurality of images or the like are transmitted using one streaming connection in a conventional service will be described. In such a case, an information processing device that transmits an image generates an image (hereinafter referred to as a "composite image") in which a sub image is superimposed on a main image in advance, and the composite image is generated. It is common to send images. In principle, composite images sent by such services cannot be separated, and the original main image itself cannot be restored (reproduced) after composition. Therefore, for example, it is difficult for the user to freely determine the layout of sub images.
On the other hand, when distributing a plurality of images using a plurality of connections, the user can freely determine the layout of sub-images. .

On the other hand, in this service, an image that includes each of a plurality of images in a classifiable manner (hereinafter referred to as a “classifiable image”) is adopted and distributed. This classifiable image is, for example, an image synthesized as one image so that each of a plurality of images is not superimposed. Therefore, the original main image can be reproduced from the classifiable image.
That is, in this service, the data of the classifiable image includes each of the plurality of images in a classifiable manner. That is, the data of a segmentable image includes data of a plurality of images while being data of one image. Therefore, in this service, even when data of a plurality of images are transmitted to an extremely large number of viewer terminals 3-c, the number of connections (hereinafter referred to as "the number of connections") is Depends only on the number of 3-c.
For this reason, in this service, for example, while keeping costs down by using only one connection, the user can perform operations with a high degree of freedom, such as freely determining the placement of sub-images. It is possible to achieve both functions.
Note that the information as to in which region of the groupable images each of the plurality of images is arranged may be shared in advance with the groupable image data region dividing unit 322 of the viewer terminal 3-c, which will be described later. .

As described above, for example, the classifiable image data generation unit 131 generates image data that includes a distributor image and a chat target person image in a classifiable manner as classifiable image data. Specifically, for example, image data in which the image of the distributor (main image), the image of the person to be chatted or the image of the chat candidate (sub-image) are arranged so as not to overlap each other, is used as the data of the classifiable image. Generate.

The classifiable image data transmission control unit 132 executes control for transmitting the classifiable image data generated by the classifiable image data generating unit 131 to the viewer terminal 3-c.
That is, for example, the classifiable image data transmission control unit 132 controls the communication unit 19 to transmit the classifiable image data from the server 1 to the viewer terminal 3-c.

Further, the classifiable image data transmission control unit 132 can execute control for transmitting the classifiable image data to the viewer terminal 3-c using the connection.

In other words, an information processing apparatus that transmits image data normally uses one connection to transmit one image data. That is, as described in the distributor-presented image control unit 112, when transmitting a group of image data, each of the plurality of image data constituting the group of image data is transmitted using each of the plurality of connections. sent.
On the other hand, the classifiable image data is data of one image in which the distributor image (main image) and the chat target person image or chat candidate image (sub-image) are arranged so as not to overlap each other. As a result, the classifiable image data transmission control unit 132 treats an image that includes a plurality of images in a classifiable manner as one classifiable image, and uses one connection to transmit the data of the one classifiable image. to execute the control sent by

The functional configuration in the image transmission process has been described above.
Next, a functional configuration in screen display processing will be described.

The classifiable image data acquisition unit 321 of the viewer terminal 3 - c acquires the classifiable image data transmitted from the server 1 .

The segmentable image data region dividing unit 322 segments the segmentable image into a plurality of images based on the data of the segmentable image.
That is, for example, the segmentable image data area partitioning unit 322 partitions the area of the segmentable image so as to correspond to each area of the plurality of images included in the group of image data in the server 1 . As described above, the information as to in which region of the classifiable images each of the plurality of images is arranged may be shared in advance with the classifiable image data generation unit 131 of the server 1 . In this case, the classifiable image data region partitioning unit 322 divides the classifiable images into regions of the plurality of images based on the information as to in which region of the classifiable images each of the plurality of images is arranged. Separate.
That is, each of the plurality of divided image regions corresponds to each of the regions of the distributor image (main image), chat target person image, and chat candidate image (sub-image).

The sub-image layout information acquisition unit 323 acquires, as layout information, information indicating how each of the display targets is to be laid out, with each of one or more sub-images among the plurality of images as the display target.

Here, the “layout information” is information indicating how the sub-images to be displayed are laid out. Specifically, for example, information such as "on the screen of the viewer terminal 3-c, the image of the person to be chatted as a sub-image is arranged (superimposed) in the lower left area of the distributor image as the main image". be. For example, the arrangement information may include information about "which sub-image is arranged at which position and with what size".
The general viewer U2-c operates the touch panel to input the position of the sub-image, the size of the sub-image, etc. according to his/her preference. The sub-image layout information acquisition unit 323 acquires layout information based on the operation of the touch panel by the general viewer U2-c.

Based on the arrangement information acquired by the sub-image arrangement information acquisition section 323, the display image data generation section 324 produces image data in which each of the sub-images to be displayed is arranged as display image data.
That is, for example, the display image data generation unit 324 converts image data in which a chat candidate image, which is a sub image, is arranged (superimposed) on a distributor image, which is a main image, based on the arrangement information. Generate as

The display control unit 325 performs control to present the viewer with a display image corresponding to the display image data generated by the display image data generation unit 324 .
That is, for example, the display control unit 325 controls the display, which is one of the output units 314, to display the display image corresponding to the data of the display image on the display of the viewer terminal 3-c.

As described above, the classifiable image data area dividing section 322 divides the classifiable image data transmitted from the server 1 so as to correspond to the respective areas of the plurality of images included in the group of image data in the server 1. Separate. Further, the sub-image layout information acquisition unit 323 acquires, as layout information, information indicating how each of the display targets is to be laid out, with each of one or more sub-images among the plurality of images as the display target. The display image data generation unit 324 also generates image data in which each of the sub-images to be displayed is arranged as display image data based on the arrangement information.

That is, each of the plurality of image regions segmented from the segmentable image corresponds to each region of the distributor image (main image) and the chat target person image and chat candidate image (sub image). In addition, the general viewer U2-c inputs the position of the sub-image, the size of the sub-image, etc. as the arrangement information according to his or her own preferences. Furthermore, based on the arrangement information, image data in which each of the sub-images to be displayed is arranged is generated as display image data.

In other words, the viewer terminal 3-c can generate the data of the display image based on the layout according to the preferences of the general viewer U2-c in the main image. This allows the user to perform operations with a high degree of freedom, such as the general viewer 3-c freely determining the arrangement of the sub-images.
In other words, in this service, the general viewer U3-c displays a display image in which a chat candidate image, which is a sub image, is superimposed on a distributor image, which is a main image, based on arrangement information according to his or her own preferences. can be viewed.

The functional configuration in the image display processing has been described above.

Hereinafter, the effects achieved by the information processing system of this embodiment executing a series of processes including image transmission processing and image display processing will be described.

First, a technique conventionally employed in a service for transmitting image data to a very large number of information processing apparatuses in real time will be described.

Generally, in a service that transmits image data to a large number of information processing apparatuses in real time, the service provider of the service builds a streaming environment using a CDN.

Here, the number of connections is the product of the number of image data and the number of information processing apparatuses. Also, the cost associated with building and using a streaming environment using a CDN depends on the number of connections. That is, the cost associated with building or using a streaming environment utilizing a CDN increases depending on both the number of image data and the number of information processing devices.

For this reason, conventionally, in building a streaming environment using a CDN or in a service using the CDN, it was not realistic to transmit data of a plurality of images to an extremely large number of information processing apparatuses. Therefore, when image data is transmitted in real time to an extremely large number of information processing apparatuses, the service provider of the service may send the image data, in which the sub-image is superimposed in advance on the main image, from the server as one image data. was sending.
Note that the cost associated with building or using a streaming environment, not limited to a CDN, can be considered to increase depending on both the number of image data and the number of information processing devices.

Next, the costs associated with building and using the streaming environment in this embodiment will be described.

In this embodiment, the segmentable image data segmentably includes each of the plurality of images. That is, the data of a segmentable image includes data of a plurality of images while being data of one image. Therefore, in this embodiment, even if data of a plurality of images are transmitted to a very large number of viewer terminals 3-c, the number of connections depends only on the number of viewer terminals 3-c. In other words, according to this embodiment, the service provider of this service can reduce the costs associated with building and using the streaming environment.

Here, in this service, the general viewer U2-c does not immediately become a new chat target person U2-a. In other words, even if there is a time lag associated with the generation and transmission of the data of the classifiable image, the disadvantage of the viewer terminal 3-c is limited.

To summarize the above, when the conventional technology is adopted, it is practically impossible to transmit data of a plurality of images to an extremely large number of viewer terminals 3-c in a service in which a streaming environment is constructed and used. was not the target. However, according to this embodiment, the service provider of this service can reduce the costs associated with building and using the streaming environment.

Furthermore, in this embodiment, the data format of the segmentable image has the structure of the data format of the image. In other words, by streaming the classifiable image data whose data format is the image format, the viewer terminal 3-c of the present embodiment processes the classifiable image data in the same manner as in normal image processing. can be done. Specifically, for example, it is sufficient for the viewer terminal 3-c of the present embodiment to divide the data area of the separable image into a plurality of image areas.

On the other hand, as a design different from the present embodiment, the service provider transmits one piece of data in which the data obtained by subdividing the data of a plurality of images are continuously arranged (hereinafter referred to as "subdivided continuous data") from the server. It is also conceivable to That is, by transmitting subdivided continuous data through a connection, each of the data of a plurality of images can be divided into small pieces and sequentially transmitted through one connection. Even in this case, since the number of connections depends only on the number of information processing devices, the cost is reduced.
However, the subdivided continuous data is in a data format different from the so-called image format. That is, the subdivided continuous data cannot be reproduced by means (programs, libraries, etc.) for reproducing (processing) ordinary images. Therefore, in the information processing apparatus, it is necessary to reconstruct the segmented image data as data of a plurality of images. In this case, costs are incurred in designing and implementing a program for reconstructing data of a plurality of images. Also, the amount of processing by the CPU of the information processing apparatus may increase when reproducing images.

To summarize the above, according to this embodiment, when the service provider of this service transmits data of a plurality of images to an extremely large number of viewer terminals 3-c, and reduces the number of connections, It is possible to reduce the cost of program design related to image processing in the viewer terminal 3-c.

In the example of FIG. 3, the distributor presentation image control unit 112, the chat target person presentation image control unit 113, and the chat candidate presentation image control unit 114 are configured not to transmit the classifiable image data. In other words, the server 1 is configured not to transmit the data of the classifiable image to the distributor terminal 2 and the viewer terminals 3-a and 3-b. Therefore, the distributor terminal 2 and the viewer terminals 3-a and 3-b receive the distributor image, the chat target person image, etc., using each connection.

Thus, there is no time lag associated with data generation and transmission of the segmentable image. That is, when the chat candidate U2-b is switched to the new chat target person U2-a, the distributor terminal 2 and the viewer terminals 3-a and 3-b do not feel uncomfortable due to the time lag.

However, by adopting a configuration in which data of a classifiable image is transmitted, it is possible to reduce costs associated with building and using a streaming environment. Therefore, when the processing time related to the generation and transmission of the data of the classifiable image is short, or when a sense of incompatibility due to the time lag is allowed, the service provider is not limited to the distributor presentation image control unit 112, but the classifiable image data may be transmitted.

The effect achieved by the information processing system of the present embodiment executing a series of processes including the image transmission process and the image display process has been described above.

FIG. 4 is a diagram showing an example of image provision in this service provided by the server of FIG.

An example of providing a distributor image captured by the distributor U1 and display images presented on the viewer terminals 3-c1 to 3-c3 in this service will be described below with reference to FIG. c1 to c3 are integer values different from each other among the aforementioned c. That is, c1 to c3 are integer values different from each other and different from a and b, and are arbitrary integer values from 1 to n.

In the example of FIG. 4, it is assumed that one broadcaster U1, one chat target U2-a, and one chat candidate U2-b conduct a video chat. Also, this video chat is delivered live and viewed by general viewers U2-c1 to U2-c3.

The distributor U1 operates the distributor terminal 2, activates an application related to this service, and takes an image of himself. A screen of the distributor terminal 2 in FIG. 4 shows a distributor image Pi1 (an image including the distributor U1 as a subject).
As described above, the distributor terminal 2 transmits the data of the distributor image Pi1 to the server 1 .

The chat target person U2-a operates the viewer terminal 3-a, activates an application related to this service, and takes an image of himself/herself. A chat target person image Pi2-a (an image including a chat target person U2-a as a subject) is shown on the screen of the viewer terminal 3-a in FIG.
As described above, the viewer terminal 3-a transmits the data of the chat target person image Pi2-a to the server 1. FIG.

The chat candidate U2-b operates the viewer terminal 3-b, activates an application related to this service, etc., and takes an image of himself/herself. The screen of the viewer terminal 3-b in FIG. 4 shows a chat candidate image Pi2-b (an image including a chat target person U2-b as a subject).
As described above, the viewer terminal 3-b transmits the data of the chat candidate image Pi2-b to the server 1. FIG.

Note that, in FIG. 4, a distributor image Pi1, a chat target person image Pi2-a, and a chat candidate image Pi2-b are displayed on the screens of the distributor terminal 2 and the viewer terminals 3-a and 3-b, respectively. What is illustrated is just an example showing how an image is captured. That is, for example, the distributor terminal 2 can display an image of the chat target person U2-a and an image of the chat candidate U2-b who are the partners of the video chat.

The server 1 receives the distributor image Pi1, the chat target person image Pi2-a, and the chat candidate image Pi2-b from the distributor terminal 2 and the viewer terminals 3-a and 3-b.
That is, as described with reference to FIG. 3, the main image data acquisition unit 121 of the server 1 acquires the distributor image Pi1 in which the distributor U1 is imaged as the main image. Further, the sub-image data acquiring unit 122 of the server 1 obtains the chat target person image Pi2-a and the chat candidate image Pi2-b, which are the chat target person U2-a and the chat candidate U2-b, respectively. , to be obtained as sub-images.

Next, the classifiable image data generation unit 131 of the viewer-presented image control unit 115 of the server 1 generates image data including each of the main image and the sub-image in a classifiable manner as classifiable image data. .
Further, the classifiable image data transmission control unit 132 of the viewer-presented image control unit 115 of the server 1 sends the classifiable image data generated by the classifiable image data generating unit 131 to the viewer terminals 3-c1 to 3-c1. Execute the control to send to c3.

Hereinafter, in FIG. 4, the images displayed on the viewer terminals 3-c1 to 3-c3 are the display images Po2-c1 to Po2-c3, respectively, and an example of different display images will be described. That is, the general viewers U2-c1 to U2-c3 will be described as inputting different layout information (positions and sizes of sub-images).
In the image Pi1 of the distributor shown in FIG. 4, the region in which the face and shoulders of the distributor U1 appear is hereinafter referred to as a "face region", and the region other than the face region is referred to as a "background region". call.

Referring to FIG. 4, the display image Po2-c1 displayed on the viewer terminal 3-c1 has a chat target person image Pi2-a as a sub-image superimposed on the background area of the distributor image Pi1 as the main image. This is an image that has been

For example, assume that general viewer U2-c1 is a fan of distributor U1 and wants to see the face of distributor U1.
In this case, the general viewer U2-c1 sends the layout information to the viewer terminal 3-c1 so that the chat target person image Pi2-a is disposed in the background area instead of the face area of the distributor image Pi1. input. The viewer terminal 3-c1 displays a display image Po2-c1 in which the chat target person image Pi2-a (sub-image) is superimposed on the background area of the distributor image Pi1.
As a result, the general viewer U2-c1 can view the display image Po2-c1 that matches the preferences of the general viewer U2-c1.

Referring to FIG. 4, the display image Po2-c2 displayed on the viewer terminal 3-c2 has a chat target person image Pi2-a as a sub-image in an area such as the face of the distributor image Pi1 as the main image. It is a superimposed image.

For example, general viewer U2-c2 wishes to see the background (scenery) that broadcaster U1 is discussing in the video chat.
In this case, the general viewer U2-c2 sends layout information to the viewer terminal 3-c2 so that the chat target person image Pi2-a is disposed in an area such as a face instead of a background area in the distributor image Pi1. input. The viewer terminal 3-c2 displays a display image Po2-c2 in which a chat target person image Pi2-a (sub-image) is superimposed on an area such as a face in the distributor image Pi1.
As a result, the general viewer U2-c2 can view the display image Po2-c2 that matches the preferences of the general viewer U2-c2.

Referring to FIG. 4, the display image Po2-c3 displayed on the viewer terminal 3-c3 includes a chat target person image Pi2-a as a sub-image and a chat target person image Pi2-a as a sub-image in a background area of a distributor image Pi1 as a main image. This is an image on which the candidate image Pi2-b is superimposed.

For example, general viewer U2-c3 wishes to see the face of chat candidate image Pi2-b with which broadcaster U1 may chat in the future.
In this case, the general viewer U2-c3 sends the placement information to the viewer terminal 3-c3 so that the chat target person image Pi2-a is placed in the background area instead of the face area of the distributor image Pi1. input. Further, the general viewer U2-c3 inputs arrangement information to the viewer terminal 3-c3 so that the chat target person image Pi2-b is further arranged in the background area of the distributor image Pi1. The viewer terminal 3-c3 displays a display image Po2-c1 in which the chat target person image Pi2-a and the chat candidate image Pi2-b (sub-image) are superimposed on the background area of the distributor image Pi1.
As a result, the general viewer U2-c3 can view the display image Po2-c3 that matches the preference of the general viewer U2-c3.

As described above, the display images Po2-c1 to Po2-c3 shown in FIG. 4 are used to display the chat target person image and the chat candidate image, which are sub-images, superimposed on the distributor image, which is the main image. An example image has been described.

In this embodiment, in the image transmission process, the server 1 transmits a plurality of images as data of one classifiable image, and in the image display process, the viewer terminal 3-c is arranged to be input by the general viewer U2-c. Generating display image data according to information has the following effects.

For example, when the general viewer U2-c1 wants to see the background area of the distributor image Pi1, by operating the viewer terminal 3-c1, the chat target person image shown in the display image Po2-c2 in FIG. It can be the configuration of Pi2-a. That is, each of the general viewers U2-c1 to U2-c3 can change the arrangement of the sub-images according to their preference in real time.

When there are only three viewer terminals 3-c1 to 3-c3 shown in FIG. 4, the number of connections from the server 1 to the viewer terminals 3-c1 to 3-c3 is , 3 (the same number as the number of viewer terminals 3-c).

On the other hand, when the conventional technology is used, it is necessary to transmit at least the distributor image Pi1 and the chat target person image Pi2-a to the general viewer U2-c1. Furthermore, if the general viewer U2-c1 is to be able to arrange the chat candidate images Pi2-b in real time, it is necessary to transmit the non-display chat candidate images Pi2-b in parallel. That is, the number of connections for general viewer U2-c1 is 3 (the same number as the number of images).
Similarly, considering general viewers U2-c2 and U2-c3, the total number of connections in FIG. 4 is 9 (the product of the number of viewer terminals 3-c and the number of images).

To summarize the above, in the example of FIG. 4, the number of connections in this embodiment can be one third of the number of connections in the prior art.

Furthermore, although not shown, in this service, there are 1 distributor U1, 1 chat target U2-a, 10 chat candidates U2-b, and 10,000 general viewers U2-c. The number of people can be assumed. In this case, the number of connections in this embodiment is 10,000. On the other hand, the number of connections in the conventional technology is 120,000. That is, the number of connections in this case can be reduced to 1/12 of the number of connections in the prior art.

Note that even if the provider of this service does not transmit the chat candidate images by streaming, the number of connections in the conventional technology is 20,000.
That is, the number of connections in this embodiment can be reduced to half the number of connections in the prior art.

Here, as described above, the cost associated with building and using the streaming environment depends on the number of connections. That is, by reducing the number of connections according to the present embodiment, the cost associated with building and using the streaming environment can be reduced.

An example of the layout of the main image and the sub-images in the data of the segmentable image will be described below with reference to FIGS. 5 and 6. FIG.

FIG. 5 is a diagram showing an example of arrangement of a main image and sub-images in the segmentable image data transmitted by the server of FIG. Here, the image shown in FIG. 5 is not an image actually displayed on the distributor terminal 2 or the viewer terminal 3, but conceptually shows an image corresponding to the data of the classifiable image.

5A and 5B show a distributor image Pi1 as a main image, a chat target person image Pi2-a as a sub-image, and a chat candidate image Pi2- as a sub-image in the classifiable image data D1. 3 shows an example of the arrangement of b.

Looking at FIG. 5A, the distributor image Pi1, which is the main image, is arranged on the left side of the classifiable image data D1. Also, a chat target person image Pi2-a, which is a sub-image of a chat target person U2-a, is arranged in the upper right part of the classifiable image data D1. In addition, the chat candidate image Pi2-b, which is a sub-image of the chat candidate U2-b, is arranged in the right middle row of the classifiable image data D1.

In the example of FIG. 5, the dot density (pixel density) at various locations in the segmentable image data D1 is constant. 5A, the distributor image Pi1, which is the main image, is arranged to occupy a larger area than the chat target person image Pi2-a and the chat candidate image Pi2-b, which are sub-images. there is That is, the chat target person image Pi2-a and the chat candidate image Pi2-b, which are sub-images, occupy a small proportion of the classifiable image data D1. That is, it can be said that the chat target person image Pi2-a and the chat candidate image Pi2-b, which are sub-images, have a smaller number of pixels (lower image quality) than the main image.
This is because when the sub-image is displayed superimposed on the main image in the display image, the number of pixels of the sub-image is smaller than that of the main image (image quality is low) and there is no problem. Assuming that the image quality of the main image is the same, the data D1 of such a classifiable image is simply compared with the data size of the image in which the main image and the sub-image are arranged in the same size, and the data of the entire image. becomes smaller. That is, it is possible to reduce the cost of transmitting the data D1 of the classifiable image.

Also, in FIG. 5A, the line surrounding the chat target person image Pi2-a, which is a sub-image, is thicker than the line surrounding the chat candidate image Pi2-b. This schematically shows that the chat target person image Pi2-a is the image of the chat target person U2-a. That is, in the example of FIG. 5A, the viewer terminal 3-c displays an image in which the chat target person image Pi2-a, which is a sub-image, is superimposed on the distributor image Pi1, which is a main image.

Next, referring to FIG. 5B, a distributor image Pi1, a chat target person image Pi2-a, and a chat candidate image Pi2-b are arranged in the classifiable image data D1. Since the arrangement of each image is basically the same as in FIG. 5A, the explanation is omitted.
However, FIG. 5B differs from FIG. 5A in the following points.

Looking at FIG. 5B, the line enclosing the chat candidate image Pi2-b, which is a sub-image, is thicker than the line enclosing the chat target person image Pi2-a. This schematically shows that the chat candidate image Pi2-b has become a new chat target person image Pi2-a. That is, FIG. 5B is an example of classifiable image data D1 that is transmitted when chat candidate U2-b becomes a new chat target person U2-a. That is, in the example of FIG. 5B, the viewer terminal 3-c adds a chat candidate image Pi2-b (a new chat target person image Pi2- Display an image on which a) is superimposed.

To summarize the above description, when the classifiable image data D1 shown in the examples of FIGS. 5A and 5B is adopted, the chat candidate U2-b becomes the new chat target person U2-a. Even in this case, there is no need to change the data D1 of the classifiable image. That is, the viewer terminal 3-c can change which sub-image is to be superimposed in which sub-image among the sub-images arranged in the data D1 of the classifiable image. That is, the control of the arrangement of the sub-images in the classifiable image data generation unit 131 can be simplified.

When the classifiable image data D1 of FIG. 5 is adopted, sub-images (chat candidate images Pi2-b) other than the chat target person U2-a are also arranged in the classifiable image data D1. As a result, it is possible to increase variations in sub-image superimposition in the viewer terminal 3-c. That is, convenience for the user is improved.

FIG. 6 is a diagram showing an example different from FIG. 5 in the arrangement of the main image and sub-images in the segmentable image data transmitted by the server in FIG.

FIGS. 6A and 6B show the arrangement of the distributor image Pi1 as the main image, the chat target person image Pi2-a as the sub-image, and the chat candidate image Pi2-b in the classifiable image data D2. shows an example.

The arrangement of the distributor image Pi1, which is the main image, and the chat target person image Pi2-a, which is the sub-image, in FIG. 6A is basically the same as in FIG.
However, FIG. 6A differs from FIG. 5A in the following points.

In the classifiable image data D2 in the example of FIG. 6A, only the chat target person image Pi2-a is arranged. That is, in the classifiable image data D2 in the example of FIG. 6A, the chat candidate image Pi2-b is not arranged.

Also, in FIG. 6A, the line surrounding the chat target person image Pi2-a, which is a sub-image, is a thick line. This schematically shows that the chat target person image Pi2-a is the image of the chat target person U2-a. That is, in the example of FIG. 6A, the viewer terminal 3-c displays an image in which the chat target person image Pi2-a, which is a sub-image, is superimposed on the distributor image Pi1, which is a main image.

Next, referring to FIG. 6(B), the layout of the distributor image Pi1 is basically the same as that in FIG. 6(A). Therefore, the description is omitted.
However, FIG. 6B differs from FIG. 6A in the following points.

Looking at FIG. 6B, the chat candidate image Pi2-b, which is a sub-image, is arranged in the upper right part of the data D2 of the classifiable image. That is, the chat candidate image Pi2-b is arranged in FIG. 6B at the place where the chat target person image Pi2-a was arranged in FIG. 6A.

Also, in FIG. 6B, the line surrounding the chat candidate image Pi2-b, which is a sub-image, is a thick line. This schematically shows that the chat candidate image Pi2-b has become a new chat target person image Pi2-a. That is, FIG. 6B is an example of classifiable image data D2 that is transmitted when chat candidate U2-b becomes a new chat target person U2-a. That is, in the example of FIG. 6B, the viewer terminal 3-c adds a chat candidate image Pi2-b (a new chat target person image Pi2- Display an image on which a) is superimposed.

To summarize the above description, when the classifiable image data D2 shown in the examples of FIGS. -a is displayed on the viewer terminal 3-c. In other words, even if the chat candidate U2-b becomes a new chat target person U2-a, the viewer terminal 3-c selects an area to be superimposed as a sub-image out of the data D2 of the classifiable image. No need to change. That is, the classifiable image data generation unit 131 of the server 1 may change which sub-images among the sub-images arranged in the classifiable image data D2 are to be superimposed and in what arrangement. In other words, the control of the arrangement of the sub-images in the viewer terminal 3-c can be simplified.

When the data D2 of the classifiable image in FIG. 6 is adopted, the lower right region, which has neither the main image nor the sub-image, in the data D2 of the classifiable image is compared with the data D1 of the classifiable image in FIG. and get bigger. In general, in an image, the data volume of the image becomes smaller as the area filled with a single color (the area without any image) is wider. That is, if the data D2 of the classifiable image of FIG. 6 is adopted, the cost associated with the transmission of the data D2 of the classifiable image is reduced.

An example of the layout of the main image and the sub-images in the data of the segmentable image has been described above with reference to FIGS. 5 and 6. FIG.

FIG. 7 is a flowchart illustrating an example of the flow of image transmission processing executed by the server having the functional configuration of FIG.
The image transmission process is executed when the server 1 transmits the data of the classifiable image to the viewer terminal 3-c.

In the process of step S<b>11 , the main image data acquisition unit 121 acquires main image data transmitted from the distributor terminal 2 .

In the process of step S12, the sub-image data acquisition unit 122 acquires sub-image data transmitted from the viewer terminal 3-a or the viewer terminal 3-b.

In the process of step S13, the classifiable image data generation unit 131 generates classifiable image data. That is, in the process of step S13, each of the plurality of images included in the group consisting of the data of the main image acquired by the process of step S11 and the data of the sub-images acquired by the process of step S12 can be classified. is generated as segmentable image data.

In the process of step S14, the classifiable image data transmission control unit 132 executes control to transmit the classifiable image data generated by the process of step S13 to the viewer terminal 3-c.

FIG. 8 is a flow chart explaining an example of the flow of image display processing executed by the viewer terminal having the functional configuration of FIG.
The image display process is executed when the viewer terminal 3-c displays a display image on the viewer terminal 3-c.

In the process of step S<b>21 , the classifiable image data acquisition unit 321 acquires classifiable image data sent from the server 1 .

In the process of step S22, the segmentable image data area division unit 322 segments the areas of the main image and the sub-image from the data of the segmentable image acquired by the process of step S21.

In the process of step S23, the sub-image arrangement information acquisition section 323 places the sub-images divided in the process of step S22 on the main image in some way, based on the operation of the viewer terminal 3-c. Gets the placement information indicating whether to place.

In the process of step S24, the display image data generation unit 324 generates display image data in which the sub-image is superimposed on the main image based on the arrangement information acquired in the process of step S22.

In the process of step S25, the display control unit 325 executes control to display the data of the display image generated by the process of step S24 on the viewer terminal 3-c.

Although one embodiment of the present invention has been described above, the present invention is not limited to the above-described embodiment, and modifications, improvements, etc. within the scope of achieving the object of the present invention are included in the present invention. be.

In the present embodiment, the present invention is applied to this service in which the distributor and the chat target person video chat, but the present invention is not particularly limited to this. That is, for example, the present invention can be employed in a service that provides an image in which two or more images are superimposed and displayed.

Specifically, for example, it can be adopted for a game distribution service. That is, for example, the server 1 may acquire an image of a game play screen as a main image, and acquire an image of a distributor playing the game as a sub image. As a result, the general viewer U2-c of the game distribution can place an image (sub-image) of the distributor U1 at an arbitrary position on the image (main image) of the game play screen. That is, convenience for the general viewer U2-c who watches the game distribution is improved.

Further, for example, in the present embodiment, a sub-image that is smaller than the main image is superimposed on the main image and displayed on the screen viewed by the general viewer U2-c. It is not limited to this. That is, for example, the main image and the sub-image may be displayed side by side without being superimposed on the screen viewed by the general viewer U2-c.
Specifically, for example, if the viewer terminal 3 has a sufficiently large display, the viewer terminal 3 can display the sub-image without superimposing it on the main image. As a result, the viewer terminal 3 can display the main image without hiding it with the sub-image.
Even in this case, since the number of connections is reduced as described above, the cost of transmitting multiple images (main image and sub-image) is reduced. Furthermore, by separating the areas of the main image and the sub-image from the separable image, even if the main image and the sub-image are displayed without being superimposed, the main image and the sub-image can be arbitrarily arranged to allow general viewing. Convenience for person U2-c is improved.

Further, for example, in the present embodiment, in the classifiable image data D1, the distributor image Pi1, which is the main image, is larger than the chat target person image Pi2-a and the chat candidate image Pi2-b, which are sub-images. Although it is supposed to be arranged, it is not particularly limited to this. That is, it is sufficient for the image data acquisition unit 111 to acquire data of a plurality of images as a group of data. Specifically, for example, the main image and the sub-image may be the same size and arranged in the segmentable image. As a result, for example, even when the main image is superimposed on the sub-image and displayed, the viewer can view the sub-image without degrading the quality of the sub-image.

Further, for example, in the present embodiment, the classifiable image data D1 in FIG. 5 is arranged such that the data of a plurality of images are arranged so as not to overlap each other, but the present invention is not particularly limited to this. That is, it is sufficient that the data of the classifiable image include each of the plurality of images so that they can be classifiable. Specifically, for example, the data of the classifiable image may be arranged by superimposing each of a plurality of images. Even in this case, it is usually possible to divide (separate) regions of a plurality of images into overlapping portions and non-overlapping portions.

Also, for example, in the present embodiment, the classifiable image data transmission control unit 132 in FIG. 3 is illustrated as an example in which the classifiable image data is directly transmitted to the viewer terminal 3-c, but it is not particularly limited to this. . That is, it is sufficient for the classifiable image data transmission control unit 132 to execute control to transmit the classifiable image data so that the viewer terminal 3-c can receive it. Specifically, for example, the classifiable image data transmission control unit 132 may construct or use a streaming environment to transmit the classifiable image data to the viewer terminal 3-c.

Further, for example, in the present embodiment, the segmentable image data region segmentation unit 322 in FIG. , but is not particularly limited to this. That is, it is sufficient for the segmentable image data area segmentation unit 322 to be able to segment each of the plurality of images based on the data of the segmentable image. Specifically, for example, by analyzing the data of the classifiable image, the edge of each region of the plurality of images included in the data of the classifiable image (for example, the classifiable image shown in FIG. 5A) ) may be detected to divide the area.

Further, for example, in the present embodiment, the display image generated by the display image data generation unit 324 is displayed by superimposing a sub-image that is smaller than the main image on the main image. Not limited to this. That is, the sub-image layout information acquisition unit 323 acquires, as layout information, information indicating how to arrange each of the one or more images to be displayed, with one or more images among the plurality of images to be displayed. That's enough. Specifically, for example, layout information of one image to be displayed among a plurality of images may be obtained and displayed as a display image. As a result, the general viewer U2-c can view one image to be displayed without the sub-image being superimposed thereon. Also, as described above, the display image data generation unit 324 can generate a display image without superimposing a plurality of images.

Further, for example, in the present embodiment, the chat candidates U2-b are extracted by the server 1 from the general viewers U2-c who watch the public chat, but the present invention is not particularly limited to this. Specifically, for example, the chat candidate U2-b may be extracted by another device (not shown), or may be extracted based on an input by a natural person such as a public chat producer or director.

Also, for example, the series of processes described above can be executed by hardware or by software.
In other words, the functional configuration of FIG. 3 is merely an example and is not particularly limited.
That is, it is sufficient that the information processing system has a function capable of executing the above-described series of processes as a whole, and what kind of functional blocks are used to realize this function is not particularly limited to the example in FIG. Also, the locations of the functional blocks are not particularly limited to those shown in FIG. 3, and may be arbitrary. That is, the server 1 may be composed of a plurality of units and executed by a plurality of servers.
Also, one functional block may be composed of hardware alone, software alone, or a combination thereof.

Further, for example, when a series of processes is executed by software, a program constituting the software is installed in a computer or the like from a network or a recording medium.
The computer may be a computer built into dedicated hardware.
Also, the computer may be a computer capable of executing various functions by installing various programs, such as a server, a general-purpose smart phone, or a personal computer.

Also, for example, the recording medium containing such a program is not only composed of a removable medium (not shown) that is distributed separately from the main body of the device in order to provide the program to general viewers U2-c, but also It is composed of a recording medium or the like provided to the general viewer U2-c in a pre-installed state.

In this specification, the steps of writing a program recorded on a recording medium are not only processes that are performed chronologically in that order, but also processes that are not necessarily chronologically processed, and that are performed in parallel or individually. It also includes the processing to be executed.
Further, in this specification, the term "system" means an overall device composed of a plurality of devices, a plurality of means, or the like.

In other words, the information processing apparatus to which the present invention is applied can take various embodiments having the following configurations.
That is, an information processing system to which the present invention is applied (for example, the information processing system in FIG. 1) is
a first information processing device (for example, server 1 in FIG. 3) that generates image data to be presented to a user (for example, general viewer U2-c who operates viewer terminal 3-c in FIG. 3); In an information processing system including a second information processing device (for example, the viewer terminal 3-c in FIG. 3) used when presenting the image to the user,
The first information processing device is
A first obtaining means (for example, 3 image data acquisition unit 111);
A first generating means (for example, a classifiable image data generating section in FIG. 3) for generating image data (for example, classifiable image data D1 in FIG. 5) containing each of the plurality of images in a classifiable manner as second data. Part 131) and
a transmission control means (for example, the classifiable image data transmission control unit 132 in FIG. 3) that executes control for transmitting the second data to the second information processing device;
with
The second information processing device is
a second acquisition unit (for example, the classifiable image data acquisition unit 321 in FIG. 3) that acquires the second data transmitted from the first information processing device;
a segmentation means (for example, segmentable image data region segmentation unit 322 in FIG. 3) for segmenting each of the plurality of images based on the second data;
A third acquisition unit (for example, FIG. 3 sub-image arrangement information acquisition unit 323);
a second generation means (for example, the display image data generation unit 324 in FIG. 3) that generates, as fourth data, image data in which the one or more images to be displayed are arranged based on the arrangement information; ,
Presentation control means (for example, the display control unit 325 in FIG. 3) that performs control to present the image corresponding to the fourth data to the user;
comprising a
An information processing system is sufficient.
In such an information processing system, the first information processing device can transmit image data including each of a plurality of images in a classifiable manner to the second information processing device as one piece of second data. Here, as a cost for transmission, there is a cost for building and using a streaming environment. The cost of building and using a streaming environment depends on the number of connections. Such an information processing system can reduce the number of connections by transmitting one piece of second data.
Further, in such an information processing system, the second information processing device can generate, as the fourth data, image data in which one or more images to be displayed are arranged, based on the arrangement information. . Thereby, the second information processing apparatus can arrange each of the one or more images to be displayed based on the arrangement information. That is, the second information processing apparatus can generate the fourth data arranged based on all arrangement information, and present an image corresponding to the fourth data to the user.
That is, such an information processing system can improve convenience for the user and reduce costs associated with image transmission.

DESCRIPTION OF SYMBOLS 1... Server, 2... Distributor terminal, 3... Viewer terminal, 11... CPU, 19... Communication part, 111... Image data acquisition part, 112... Distributor Presentation image control unit 113 Chat target person presentation image control unit 114 Chat candidate presentation image control unit 115 Viewer presentation image control unit 121 Main image data acquisition unit 122 Sub-image data acquisition unit 131 Divided image data generation unit 132 Divideable image data transmission control unit 311 CPU 312 Communication unit 313 Input Unit 314 Output unit 321 Classifiable image data acquisition unit 322 Classifiable image data area division unit 323 Sub image layout information acquisition unit 324 Display image data generation unit 325 ... display control unit

Claims (4)

In an information processing system including a first information processing device that generates data of an image to be presented to a user and a second information processing device that is used when presenting the image to the user,
The first information processing device is
a first acquisition means for classifying each of the data of the plurality of images into one of N types (N is an integer value of 2 or more) relating to the subject and acquiring the data as a first data group;
image data obtained by arranging each of the plurality of images in the small regions of the classified type among the N types of regions, with respect to the regions previously divided into the N types of small regions in the spatial direction ; , a first generating means for generating as second data;
transmission control means for executing control to transmit the second data to the second information processing device;
with
The second information processing device is
a second obtaining means for obtaining the second data transmitted from the first information processing device;
classification means for classifying each of the plurality of images into one of the N types based on the second data;
Information indicating how to arrange one or more images to be displayed among the plurality of images, wherein each of the one or more images to be displayed is arranged in advance for each of the N types . a third acquisition means for acquiring information for which is determined as arrangement information;
Second generation for generating image data in which the one or more images to be displayed are arranged, as fourth data, based on the arrangement information and the type of each of the one or more images to be displayed. means and
presentation control means for performing control to present the image corresponding to the fourth data to the user;
comprising
Information processing system. The first acquisition means selects M types of the N types to be displayed on the second information processing apparatus side. One or more images classified into a class (M is an integer value equal to or less than N) are taken as a group of the first data group. get,
The arrangement information is information in which arrangement positions are determined in advance for each of the M types.
The information processing system according to claim 1. The image displayed on the second information processing device is an image of the live distribution of the distributor, Type 1 including the broadcaster as a subject when the broadcaster and the chat target are chatting an image, a second-type image including the chat target as a subject, and a chat candidate as the subject Including at least the first type image and the second type image among the third type images,
The information processing system according to claim 1 or 2. The first acquisition means acquires main image data and sub-image data from the first data group,
The first generating means generates image data, which includes the main image of the first image quality and the sub-image of the second image quality in a distinguishable manner, as the second data.
The information processing system according to any one of claims 1 to 3 .

Images