JP2020120336A - Program, method, and information processing device - Google Patents

Abstract

To provide a program capable of improving amusement of content in live distribution, a method, and an information processing device.SOLUTION: A viewing program causes a processor to execute the steps of: receiving first information (step S200); constructing a virtual space (steps S201 to S202); arranging a first object (step S204); operating the first object according to motion data (step S207); arranging a virtual viewpoint of the virtual space in a position which differs from a center point of the virtual space and includes the center point in a visual field region (step S208); and rendering an image of the visual field region at the virtual viewpoint (step S212).SELECTED DRAWING: Figure 6

Description

The present invention relates to a program, a method, and an information processing device.

Conventionally, a technique for showing an omnidirectional image to a user has been developed. For example, there is a technique of pasting a 360-degree panoramic image inside a spherically modeled virtual space and displaying an image obtained by cutting the panoramic image in the field of view of the virtual camera. In addition, a technology has been developed in which a distributor performs live distribution as a content of a music live or game live situation via an avatar.

"Facebook Social VR Demo-Oculus Connect 2016", [online], YouTube, [January 24, 2019 search], Internet (URL: https://www.youtube.com/watch?v=YuIgyKLPt3s)

One aspect of the present invention aims to improve the interest of content in live distribution.

In order to solve the above problems, a program according to one embodiment of the present invention is executed by a computer as a viewing terminal for a distributor to view content to be distributed in real time using a distribution terminal, which includes a processor and a memory. It is a program that is done. The program receives, into the processor, first information, which is information related to the content, which is information that is distributed from the distribution terminal at any time and that includes at least motion data in which the movement of the body of the distributor is detected, and a background image. Of constructing a virtual space in which is expanded, placing a first object that is an avatar corresponding to a distributor in the virtual space, and receiving first information during real-time distribution. In response, operating the first object according to the motion data, and arranging the virtual viewpoint at a position in the virtual space different from the center point of the virtual space and including the center point in the visual field region. , Performing a step of rendering an image of the visual field area of the virtual viewpoint, and a step of displaying the image obtained by the rendering on the display unit.

According to one aspect of the present invention, it is possible to improve the interest of content in live distribution.

FIG. 1 is a diagram showing an outline of a distribution system according to a first embodiment. It is a figure which shows the hardware constitutions of various apparatuses contained in the said delivery system. It is a block diagram which shows the functional structure of the various apparatuses contained in the said delivery system. It is a sequence diagram which shows an example of the flow of the process which concerns on the live delivery and reproduction of the content in the said delivery system. It is a figure which shows the mode of content production by the distributor in the said distribution system. It is a figure which shows an example of the flow of the reproduction process in the said delivery system. State (A) is a diagram showing an example of a virtual space constructed in the distribution system. State (B) is a diagram showing an example of an image of the field of view of the virtual camera shown in state (A). FIG. 9 is a diagram showing an example of an image of a visual field area of the virtual camera shown in FIG. 7. State (A) is a diagram showing another example of the virtual space constructed in the distribution system. State (B) is a diagram showing an example of an image of the field of view of the virtual camera shown in state (A). It is a figure which shows an example of the display screen of a viewing-and-listening terminal. It is a figure which shows an example of a user's operation with respect to the display screen shown in FIG. It is a figure which shows the display screen of the viewing-and-listening terminal at the time of viewing the content delivered live. It is a figure which shows the outline|summary of the delivery system which concerns on Embodiment 2. State (A) is a diagram showing another example of the virtual space constructed in the distribution system. State (B) is a diagram showing an example of an image of the field of view of the virtual camera shown in state (A). State (A) is a diagram showing the virtual space after the second input operation is performed and the position and angle of the virtual camera are switched. State (B) is a diagram showing an example of an image of the field of view of the virtual camera shown in state (A). State (A) is a diagram showing the virtual space after the third input operation is performed and the position and angle of the virtual camera are switched. State (B) is a diagram showing an example of an image of the field of view of the virtual camera shown in state (A). State (A) is a diagram showing the virtual space after the fourth input operation is performed and the position and angle of the virtual camera are switched. State (B) is a diagram showing an example of an image of the field of view of the virtual camera shown in state (A). State (A) is a diagram showing the virtual space after the fifth input operation is performed and the position and angle of the virtual camera are switched. State (B) is a diagram showing an example of an image of the field of view of the virtual camera shown in state (A). It is a figure which shows another example of the image of the visual field area of a virtual camera in the said delivery system.

The distribution system according to the present disclosure is a system for distributing content to viewers in real time. Hereinafter, the distribution system will be described with reference to the drawings. It should be noted that the present invention is not limited to the examples described below and illustrated in the drawings, and is shown by the scope of the claims, and all modifications within the meaning and scope equivalent to the scope of the claims are included in the present invention. Is intended. In the following description, the same elements will be denoted by the same reference symbols in the description of the drawings, without redundant description.

[Embodiment 1]
<Outline of operation of distribution system 1>
FIG. 1 is a diagram showing an outline of a distribution system 1 according to this embodiment. The distribution system 1 includes at least one distribution server 200, at least one distribution terminal 300, and at least one viewing terminal 100.

(Distribution terminal 300)
The distribution terminal 300 is a device that is a main body for live distribution of contents. The distribution terminal 300 is realized by, for example, a personal computer (PC) or a smartphone. Note that, hereinafter, the user of the distribution terminal 300 will be referred to as a “distributor”. The distribution terminal 300 creates content data according to an input operation by the user of the distribution terminal 300. The content data is data for viewing the content created by the distributor on the viewing terminal 100. The distribution terminal 300 sequentially creates content data and distributes it to the distribution server 200 in real time. Hereinafter, delivering the content data in real time is also referred to as “live delivery”.

(Distribution server 200)
The distribution server 200 is a server that distributes the content data received from the distribution terminal 300 in real time to the viewing terminal 100. The distribution server 200 is also a server that mediates the transmission and reception of various information between the viewing terminal 100 and the distribution terminal 300.

The distribution server 200 may collect various kinds of information related to live distribution from one or more viewing terminals 100 and transmit the various kinds of information to the distribution terminal 300. For example, the distribution server 200 may total the number of viewing terminals 100 that are currently playing the content, that is, the current number of viewers, and transmit the total to the distribution terminal 300. In addition, the distribution server 200 may total the total number of viewers from the start of distribution of the content and transmit the total number of viewers to the distribution terminal 300. Further, the distribution server 200 may specify the viewing terminal 100 that has transmitted the above-described input information, and transmit the information indicating the viewing terminal 100 to the distribution terminal 300.

(Viewing terminal 100)
The viewing terminal 100 is a device that receives content data in real time, generates an image using the content data, and reproduces a video. In other words, the viewing terminal 100 is a device that renders and plays content in real time. In the following description, real-time rendering and reproduction of content is also simply referred to as “reproduction of content”. The viewing terminal 100 is realized by, for example, a PC or a smartphone. The viewing terminal 100 receives and reproduces the content data from the distribution server 200 according to the input operation by the user of the viewing terminal 100. Specifically, the viewing terminal 100 receives data such as a motion as content data from the distribution server 200, and the viewing terminal 100 uses these data to generate a video (moving image) by real-time rendering and play it back. .. Hereinafter, the user of the viewing terminal 100 will be referred to as a “viewer”. The method of reproducing the content will be described later in detail.

The viewing terminal 100 may not only reproduce the content data, but may also transmit the information specified or input by the viewer's input operation to the distribution terminal 300 via the distribution server 200. .. For example, the viewing terminal 100 may transmit the comment input by the viewer to the distribution terminal 300 via the distribution server 200.

In this embodiment, an example in which the distribution server 200 and the distribution terminal 300 are separate devices will be described. However, the distribution server 200 and the distribution terminal 300 may be an integrated device. For example, one PC may have both the function as the distribution server 200 and the function as the distribution terminal 300.

<Hardware configuration of distribution system 1>
FIG. 1 is a diagram showing a hardware configuration of the distribution system 1. As illustrated, the distribution system 1 includes a distribution terminal 300, a distribution server 200, and a plurality of viewing terminals 100. The viewing terminal 100 and the distribution terminal 300 are connected to the distribution server 200 via the network 4.

(Network 4)
The network 4 includes various mobile communication systems constructed by the Internet and a wireless base station (not shown). Examples of this mobile communication system include so-called 3G and 4G mobile communication systems, LTE (Long Term Evolution), and a wireless network (for example, Wi-Fi (registered trademark)) that can be connected to the Internet by a predetermined access point. To be

(Distribution server 200)
The distribution server 200 (computer, information processing device) may be a general-purpose computer such as a workstation or a personal computer. The distribution server 200 includes a processor 20, a memory 21, a storage 22, a communication IF 23, and an input/output IF 24. These components included in the distribution server 200 are electrically connected to each other via a communication bus.

(Viewing terminal 100)
The viewing terminal 100 (computer, information processing device) may be a mobile terminal such as a smartphone, a feature phone, a PDA (Personal Digital Assistant), or a tablet computer. The viewing terminal 100 may be a playback device suitable for playing content. As illustrated, the viewing terminal 100 includes a processor 10, a memory 11, a storage 12, a communication interface (IF) 13, an input/output IF 14, and a touch screen 15 (display unit). These components included in the viewing terminal 100 are electrically connected to each other via a communication bus. The viewing terminal 100 may include an input/output IF 14 to which a display (display unit) configured separately from the main body of the viewing terminal 100 can be connected instead of or in addition to the touch screen 15.

Further, as shown in FIG. 2, the viewing terminal 100 may be configured to be able to communicate with one or more controllers 1020. The controller 1020 establishes communication with the viewing terminal 100 according to a communication standard such as Bluetooth (registered trademark). The controller 1020 may have one or more buttons or the like, and transmits an output value based on a user's input operation to the buttons or the like to the viewing terminal 100. Further, the controller 1020 may include various sensors such as an acceleration sensor and an angular velocity sensor, and transmits output values of the various sensors to the viewing terminal 100.

Moreover, the controller 1020 may be configured to be attachable to and detachable from the viewing terminal 100. In this case, a connecting portion with the controller 1020 may be provided on at least one surface of the housing of the viewing terminal 100. When the viewing terminal 100 and the controller 1020 are connected by wire via the connecting unit, the viewing terminal 100 and the controller 1020 send and receive signals via a wire.

As shown in FIG. 2, the viewing terminal 100 may accept mounting of a storage medium 1030 such as an external memory card via the input/output IF 14. Thereby, the viewing terminal 100 can read the program and data recorded in the storage medium 1030. The program recorded in the storage medium 1030 is, for example, a viewing program.

The viewing terminal 100 may store the viewing program acquired by communicating with an external device such as the distribution server 200 in the memory 11 of the viewing terminal 100, or the viewing program acquired by reading from the storage medium 1030 may be stored in the memory. 11 may be stored.

As described above, the viewing terminal 100 includes the communication IF 13, the input/output IF 14, and the touch screen 15 as an example of a mechanism for inputting information to the viewing terminal 100. Each of the above-described units as a mechanism for inputting can be regarded as an operation unit configured to receive an input operation of a user.

For example, when the operation unit includes the touch screen 15, the viewing terminal 100 identifies and accepts the user operation performed on the input unit 151 of the touch screen 15 as the user input operation. Alternatively, when the operation unit is configured by the communication IF 13, the viewing terminal 100 identifies and receives a signal (for example, output value) transmitted from the controller 1020 as a user input operation. Alternatively, when the operation unit is configured by the input/output IF 14, a signal output from an input device (not shown) different from the controller 1020 connected to the input/output IF 14 is specified and accepted as the user's input operation.

(Distribution terminal 300)
The distribution terminal 300 (computer, information processing device) may be a mobile terminal such as a smartphone, a PDA (Personal Digital Assistant), or a tablet computer. The distribution terminal 300 may be a server, a desktop personal computer, a notebook personal computer, a computer such as a tablet, or a computer group combining these.

The distribution terminal 300 may be a device group in which a display device, an input device, and an information processing device are connected so as to be suitable for live distribution. As illustrated, the distribution terminal 300 includes a processor 30, a memory 31, a storage 32, a communication interface (IF) 33, an input/output IF 34, and a touch screen 35 (display unit). These components included in the distribution terminal 300 are electrically connected to each other by a communication bus. The distribution terminal 300 may include an input/output IF 34 to which a display (display unit) configured separately from the main body of the distribution terminal 300 can be connected instead of or in addition to the touch screen 35.

In addition, as shown in FIG. 1, the distribution terminal 300 wirelessly or wiredly communicates with one or more microphones 3010, one or more motion capture devices 3020, and one or more peripheral devices such as a controller 3030. , May be configured to be communicable. The peripheral device wirelessly connected establishes communication with the distribution terminal 300 according to a communication standard such as Bluetooth (registered trademark). Further, a device group including these peripheral devices may be regarded as a set of distribution terminals 300.

The microphone 3010 acquires a sound generated in the surroundings and converts it into an electric signal. The voice converted into the electric signal is transmitted to the distribution terminal 300 as voice data and is received by the distribution terminal 300 via the communication IF 33.

The motion capture device 3020 tracks the motion (including the position and posture of the head, facial expression, mouth movement, etc.) of the tracking target (for example, a person) and transmits the output value as the tracking result to the distribution terminal 300. .. The motion data that is the output value is accepted by the distribution terminal 300 via the communication IF 33. The motion capture method of the motion capture device 3020 is not particularly limited. The motion capture device 3020 is a camera, various sensors, a marker, a suit worn by a model (person), a signal transmitter, an HMD (Head Mount Device), or any other mechanism for capturing motion according to the adopted system. Are selectively equipped.

The controller 3030 may have a physical input mechanism such as one or more buttons, levers, sticks, wheels and the like. The controller 3030 transmits to the distribution terminal 300 an output value based on an input operation input to the input mechanism by the operator of the distribution terminal 300. Further, the controller 3030 may include various sensors such as an acceleration sensor and an angular velocity sensor, and the output values of the various sensors may be transmitted to the distribution terminal 300. The output value described above is accepted by the distribution terminal 300 via the communication IF 33. In the following, a person who performs some input operation on the distribution terminal 300 using the operation unit provided in the distribution terminal 300 or various input mechanisms communicatively connected to the distribution terminal 300 is referred to as an operator. .. The operator includes a person who operates the distribution terminal 300 by using the input unit 351, the controller 3030, and the like, a voice actor who inputs a voice through the microphone 3010, and a movement through the motion capture device 3020. The model to be input is also included.

The distribution terminal 300 may include a camera and a distance measuring sensor (not shown). Instead of or in addition to the distribution terminal 300, the motion capture device 3020 and the controller 3030 may include a camera and a distance measuring sensor.

As described above, the distribution terminal 300 includes the communication IF 33, the input/output IF 34, and the touch screen 35 as an example of a mechanism for inputting information to the distribution terminal 300. The distribution terminal 300 may further include a camera and a distance measuring sensor, if necessary. Each of the above-described units as a mechanism for inputting can be regarded as an operation unit configured to receive an input operation of a user.

The operation unit may be configured with the touch screen 35. In this case, the distribution terminal 300 identifies and accepts the user's operation performed on the input unit 351 of the touch screen 35 as the user's input operation. Alternatively, when the operation unit includes the communication IF 33, the distribution terminal 300 identifies and receives a signal (for example, an output value) transmitted from the controller 3030 as a user input operation. Alternatively, when the operation unit is configured by the input/output IF 34, a signal output from an input device (not shown) different from the controller 3030 connected to the input/output IF 34 is specified and accepted as the user's input operation.

<Hardware components of each device>
The processor 10 controls the operation of the entire viewing terminal 100. The processor 20 controls the operation of the entire distribution server 200. The processor 30 controls the operation of the entire distribution terminal 300. The processors 10, 20, and 30 include a CPU (Central Processing Unit), an MPU (Micro Processing Unit), and a GPU (Graphics Processing Unit).

The processor 10 reads a program from the storage 12 described later and loads it into the memory 11 described later. The processor 20 reads a program from a storage 22 described later and loads it into a memory 21 described later. The processor 30 reads a program from a storage 32 described later and loads it into a memory 31 described later. The processor 10, the processor 20, and the processor 30 execute the expanded program.

Memories 11, 21, and 31 are main storage devices. The memories 11, 21, and 31 are configured by storage devices such as ROM (Read Only Memory) and RAM (Random Access Memory). The memory 11 provides a work area to the processor 10 by temporarily storing a program and various data read by the processor 10 from a storage 12 described later. The memory 11 also temporarily stores various data generated while the processor 10 operates according to a program. The memory 21 provides a work area to the processor 20 by temporarily storing various programs and data read by the processor 20 from a storage 22 described later. The memory 21 also temporarily stores various data generated while the processor 20 operates according to the program. The memory 31 provides a work area to the processor 30 by temporarily storing various programs and data read by the processor 30 from a storage 32 described later. The memory 31 also temporarily stores various data generated while the processor 30 operates according to the program.

In the present embodiment, the program executed by the processor 30 may be a distribution program for realizing live distribution of content. In the present embodiment, the program executed by the processor 10 may be a viewing program for realizing content reproduction.

In the present embodiment, the program executed by the processor 20 may be the distribution program described above or the viewing program described above. The processor 20 executes either or both of the distribution program and the viewing program in response to a request from the viewing terminal 100 or the distribution terminal 300. The distribution program and the viewing program may be executed in parallel.

That is, the distribution program may be a distribution program for realizing live distribution by cooperation between the distribution server 200 and the distribution terminal 300. Further, the viewing program may be a viewing program for realizing the reproduction of the content by the cooperation of the viewing terminal 100 and the distribution server 200.

The storages 12, 22, and 32 are auxiliary storage devices. The storages 12, 22, and 32 are configured by a storage device such as a flash memory or an HDD (Hard Disk Drive). Various data relating to the game is stored in the storages 12, 22, and 32.

The communication IF 13 controls transmission/reception of various data in the viewing terminal 100. The communication IF 23 controls transmission/reception of various data in the distribution server 200. The communication IF 33 controls transmission/reception of various data in the distribution terminal 300. The communication IFs 13, 23, and 33 control, for example, communication using a wireless LAN (Local Area Network), wired LAN, wireless LAN, or Internet communication via a mobile phone network, and communication using short-range wireless communication or the like. To do.

The input/output IF 14 of the viewing terminal 100 is an interface for the viewing terminal 100 to accept data input, and is an interface for the viewing terminal 100 to output data. The input/output IF 14 may input/output data via a USB (Universal Serial Bus) or the like. The input/output IF 14 may include, for example, a physical button of the viewing terminal 100, a camera, a microphone, a speaker, or the like. The input/output IF 24 of the distribution server 200 is an interface for the distribution server 200 to accept data input, and is an interface for the distribution server 200 to output data. The input/output IF 24 may include, for example, an input unit which is an information input device such as a mouse or a keyboard, and a display unit which is a device for displaying and outputting an image. The input/output IF 34 of the distribution terminal 300 is an interface for the distribution terminal 300 to accept data input, and is an interface for the distribution terminal 300 to output data. The input/output IF 34 includes, for example, an information input device such as a mouse, a keyboard, a stick, and a lever, a device for displaying and outputting an image such as a liquid crystal display, and peripheral devices (a microphone 3010, a motion capture device 3020, and a controller 3030). A connection may be included for sending and receiving data between.

The touch screen 15 of the viewing terminal 100 is an electronic component that combines the input unit 151 and the display unit 152. The touch screen 35 of the distribution terminal 300 is an electronic component that combines an input unit 351 and a display unit 352. The input units 151 and 351 are, for example, touch-sensitive devices, and are configured by, for example, touch pads. The display units 152 and 352 are configured by, for example, a liquid crystal display, an organic EL (Electro-Luminescence) display, or the like.

The input units 151 and 351 detect a position at which a user operation (mainly a physical contact operation such as a touch operation, a slide operation, a swipe operation, and a tap operation) on the input surface is detected, and information indicating the position is detected. Is provided as an input signal. The input units 151 and 351 may include a touch sensing unit (not shown). The touch sensing unit may be of any type such as a capacitance type or a resistance film type.

Although not shown, the viewing terminal 100 may include one or more sensors for specifying the holding posture of the viewing terminal 100. This sensor may be, for example, an acceleration sensor or an angular velocity sensor. When the viewing terminal 100 includes the sensor, the processor 10 can also specify the holding attitude of the viewing terminal 100 from the output of the sensor and perform the processing according to the holding attitude. For example, the processor 10 may perform vertical screen display for displaying a vertically long image on the display unit 152 when the viewing terminal 100 is held vertically. On the other hand, when the viewing terminal 100 is held sideways, a horizontally long image may be displayed on the display unit. In this way, the processor 10 may be capable of switching between the vertical screen display and the horizontal screen display according to the holding posture of the viewing terminal 100.

The camera 17 includes an image sensor and the like, and converts a light incident from a lens into an electric signal to generate a captured image.

The distance measuring sensor 18 is a sensor that measures the distance to the measurement target. The distance measuring sensor 18 includes, for example, a light source that emits pulse-converted light and a light receiving element that receives the light. The distance measuring sensor 18 measures the distance to the measurement object by the light emission timing from the light source and the light reception timing of the reflected light generated by the light emitted from the light source reflected on the measurement object. The distance measuring sensor 18 may include a light source that emits light having directivity.

<Functional configuration of distribution system 1>
FIG. 3 is a block diagram showing the functional configurations of the viewing terminal 100, the distribution server 200, and the distribution terminal 300 included in the distribution system 1. Each of the viewing terminal 100, the distribution server 200, and the distribution terminal 300 has a functional configuration required to function as a general computer (not shown) and a functional configuration required to realize a known function in a game. Configurations may be included.

The viewing terminal 100 has a function as an input device that receives an input operation of a user and a function as an output device that outputs a game image or sound. The viewing terminal 100 functions as the control unit 110 and the storage unit 120 in cooperation with the processor 10, the memory 11, the storage 12, the communication IF 13, the input/output IF 14, and the like.

The distribution server 200 communicates with the distribution terminal 300 and the viewing terminal 100. The distribution server 200 has a function of acting as an intermediary when distributing content data from the distribution terminal 300 to the viewing terminal 100. The distribution server 200 functions as the control unit 210 and the storage unit 220 in cooperation with the processor 20, the memory 21, the storage 22, the communication IF 23, the input/output IF 24, and the like.

The distribution terminal 300 has a function of creating content and performing live distribution of the content to the viewing terminal 100 via the distribution server 200. The distribution terminal 300 functions as the control unit 310 and the storage unit 320 in cooperation with the processor 30, the memory 31, the storage 32, the communication IF 33, the input/output IF 34, and the like.

(Data stored in the storage unit of each device)
The storage unit 120 stores the viewing program 131 and the viewer information 132 of the viewer corresponding to the viewing terminal 100. The storage unit 320 stores a distribution program 331 and distributor information 332 of a distributor corresponding to the distribution terminal 300. The storage unit 220 stores a viewing program 131, viewer information 132, a distribution program 331, and distributor information 332. The storage unit 220 collectively stores the distributor information 332 collected from each distribution terminal 300. The storage unit 220 collectively stores the viewer information 132 collected from each viewing terminal 100.

The viewing program 131 is a program executed by the viewing terminal 100 and the distribution server 200. On the other hand, the distribution program 231 is a program executed by the distribution terminal 300 and the distribution server 200.

It should be noted that execution of the viewing program 131 and the distribution program 331 by the distribution server 200 is not essential. For example, when the viewing terminal 100 can execute the viewing program 131 by itself to reproduce the content, the distribution server 200 does not have to execute the viewing program 131. Further, for example, when the distribution terminal 300 can execute live distribution by executing the distribution program 331 independently, the distribution server 200 does not have to execute the distribution program 331.

The viewer information 132 is data that the control unit 110 refers to when executing the viewing program 131. The viewer information 132 is data managed for each viewer. For example, the viewer information 132 may include the name of the viewer, that is, the name of the account of the viewer corresponding to the viewing terminal 100. Further, the viewer information 132 may include a content browsing history, a message transmission/reception history, and the like in the account of the viewer corresponding to the viewing terminal 100.

The distributor information 332 is data that the control unit 210 refers to when executing the distribution program 231. The distributor information 332 is data managed for each distributor. For example, the distributor information 332 may include information indicating the name of the distributor account to which the distribution terminal 300 corresponds. Further, the distributor information 332 includes various kinds of information related to the distributed contents such as the distribution history of the contents, the number of times the distributed contents are reproduced, etc. in the account of the distributor corresponding to the distribution terminal 300, and the history of message transmission and reception. May be.

(Functional configuration of distribution server 200)
The control unit 210 centrally controls the distribution server 200 by executing the distribution program 231 stored in the storage unit 220. For example, the control unit 210 receives the content from the distribution terminal 300 in real time, and delivers the content live to the viewing terminal 100. The control unit 210 communicates with the viewing terminal 100 and the distribution terminal 300 to send and receive various types of information as necessary.

The control unit 210 functions as the transmission/reception unit 211, the server processing unit 212, and the data management unit 213 according to the description of the distribution program 231. The control unit 210 can also function as other functional blocks (not shown) in order to support live distribution of content from the distribution terminal 300 and reproduction of the content by the viewing terminal 100.

The transmission/reception unit 211 receives various data and requests from the distribution terminal 300. For example, the transmission/reception unit 211 receives a part or all of the distributor information 332 from the distribution terminal 300. The transmission/reception unit 211 transmits various data and requests to the distribution terminal 300.

The transmitting/receiving unit 211 also receives various data and requests from the viewing terminal 100. For example, the transmission/reception unit 211 receives a live distribution request from the viewing terminal 100. The transmission/reception unit 211 transmits various kinds of information to the viewing terminal 100. For example, the transmission/reception unit 211 delivers the content live to the viewing terminal 100.

The server processing unit 212 performs arithmetic processing necessary for live distribution and reproduction of content. The server processing unit 212 instructs the transmission/reception unit 211 to transmit various data. The server processing unit 212 instructs the data management unit 213 to add, update, or delete various information.

The data management unit 213 manages the data stored in the storage unit 220. The data management unit 213 executes addition, update, or deletion of various information included in the distributor information 332 and the viewer information 132 according to an instruction from the server processing unit 212. The data management unit 213 may read at least one of the distributor information 332 and the viewer information 132 from the storage unit 220 according to an instruction from the server processing unit 212.

(Functional configuration of the viewing terminal 100)
The control unit 110 executes the viewing program 131 stored in the storage unit 120 to centrally control the viewing terminal 100. For example, the control unit 110 executes content reproduction in accordance with the viewing program 131 and the user's operation. Further, the control unit 110 communicates with the distribution server 200 as necessary to transmit and receive information during reproduction of content.

The control unit 110 functions as the operation reception unit 111, the reproduction unit 112, the display control unit 113, and the communication unit 114 according to the description of the viewing program 131. The control unit 110 can also function as other functional blocks (not shown) in order to reproduce the content.

The operation receiving unit 111 detects and receives a user's input operation on the input unit 151. The operation reception unit 111 determines what input operation has been performed based on the user's action on the console via the touch screen 15 and the other input/output IF 14, and outputs the result to each element of the control unit 110. To do.

For example, the operation receiving unit 111 receives an input operation on the input unit 151, detects the coordinates of the input position of the input operation, and specifies the type of the input operation. The operation receiving unit 111 identifies, as the type of input operation, for example, a touch operation, a slide operation, a swipe operation, a tap operation, or the like. Further, the operation receiving unit 111 detects that the touch input is released from the touch screen 15 when the continuously detected input is interrupted.

The reproduction unit 112 executes processing related to content reproduction. The reproducing unit 112 receives content data from the distribution server 200 via the communication unit 114 in real time. The reproduction unit 112 includes a space definition unit 112A, a background control unit 112B, an object control unit 112C, a camera control unit 112D, and a drawing unit 112E.

The space definition unit 112A defines a virtual space based on the content data. A specific example of the virtual space and a definition method will be described in detail later.

The background control unit 112B arranges the background image designated by the content data in the virtual space. The type of the background image is not particularly limited, but may be, for example, a landscape image obtained by photographing 360 degrees around a specific existing point.

The object control unit 112C arranges the object designated by the content data in the virtual space at the position designated by the data in the designated posture.

The camera control unit 112D arranges a virtual camera in the virtual space. The virtual camera defines the position and angle of the virtual viewpoint of the user in the virtual space.

The drawing unit 112E draws at least the object and the background image included in the field of view of the virtual camera in the virtual space as a moving image or a still image. Hereinafter, a moving image or a still image in which an object and a background image included in the visual field of the virtual camera are drawn is simply referred to as “image of visual field”. The drawing unit 112E may draw not only the visual field area but also the background image and the object of the entire virtual space.

The display control unit 113 causes the display unit 152 to display the image of the visual field area drawn by the drawing unit 112E of the reproduction unit 112 as content. The display control unit 113 may control and draw a 2D or 3D game object (UI object) for constructing a UI (user interface). For example, the display control unit 113 may superimpose the UI object required for various operations for viewing the content on the content, or may display the UI object on the screen of the display unit 152 together with the content. The UI object is, for example, an icon, a button, a list, a menu screen, or the like. The UI object is a tool for notifying the viewer of various information from the viewing terminal 100 during the reproduction of the content on the viewing terminal 100.

The communication unit 114 transmits various data and requests to the distribution server 200. For example, the communication unit 114 transmits a distribution request for the content data and a part or all of the viewer information 132 to the distribution server 200. The communication unit 114 receives various kinds of information related to live distribution from the distribution server 200. For example, the communication unit 114 receives the content data and part or all of the distributor information 332 from the distribution server 200.

(Functional configuration of distribution terminal 300)
The control unit 310 centrally controls the distribution terminal 300 by executing the distribution program 331 stored in the storage unit 320. For example, the control unit 310 creates content data according to the operations of the distribution program 331 and the distributor, and supplies the content data to the viewing terminal 100 via the distribution server 200. The control unit 310 communicates with the distribution server 200 to send and receive information.

The control unit 310 functions as the operation reception unit 311, the distribution unit 312, the display control unit 313, and the communication unit 314 according to the description of the distribution program 331. The control unit 310 can also function as another functional block (not shown) according to the property of the content data to be created, that is, the property of the content.

The operation receiving unit 311 detects and receives an input operation of the operator on the input unit 351. The operation reception unit 311 determines what input operation is performed from the action exerted by the operator on the console via the touch screen 35 and the other input/output IF 34, and outputs the result to each element of the control unit 310. Output. The details of the function of the operation receiving unit 311 are almost the same as those of the operation receiving unit 111 in the viewing terminal 100.

The distribution unit 312 executes various processes related to content distribution. For example, the delivery unit 312 creates content data. The method of creating the content data will be described in detail later.

The distribution unit 312 may create a screen for the distributor to assist the distributor in operating the content creation. Hereinafter, the screen for the distributor is also referred to as a distribution screen. The structure of the distribution screen is not particularly limited. For example, the delivery unit 312 may create a delivery screen that can list various components that can be included in the content, such as various objects such as avatars, background images, and various parameters. The distribution screen is displayed on the touch screen 35 via the display control unit 313.

Further, the distribution unit 312 may create and output various notifications to the distributor. For example, the distribution unit 312 may notify the distributor of the viewer comment received via the distribution server 200.

The display control unit 313 outputs, to the display unit 352 of the touch screen 35, a display screen on which the processing results executed by the above-described elements are reflected. The details of the function of the display control unit 313 are almost the same as those of the display control unit 113 in the viewing terminal 100. For example, the display control unit 313 may display UI objects required for various operations for live distribution on the screen of the display unit 352.

The communication unit 314 transmits various data and requests to the distribution server 200. For example, the communication unit 314 transmits the content and a part or all of the distributor information 332 to the distribution server 200. The communication unit 314 receives various kinds of information related to live distribution from the distribution server 200. For example, the communication unit 314 receives a part or all of the viewer information 132 from the distribution server 200.

The functions of the viewing terminal 100, the distribution server 200, and the distribution terminal 300 shown in FIG. 3 are merely examples. For example, the distribution server 200 may include at least a part of the functions included in the viewing terminal 100 or the distribution terminal 300. Further, for example, the viewing terminal 100 or the distribution terminal 300 may include at least a part of the functions included in the distribution server 200.

Further, the viewing terminal 100 may cooperate with other components other than the above-described components, or other devices connected to the viewing terminal 100. For example, the viewing terminal 100 may cause another component or another device to execute a part of the processing in the viewing terminal 100. In addition, the distribution terminal 300 may cooperate with other constituent elements other than the above-described constituent elements, or other devices connected to the distribution terminal 300. For example, the distribution terminal 300 may cause another component or another device to execute a part of the processing in the distribution terminal 300.

The computer that executes the distribution program in the present embodiment may be any of the distribution server 200, the distribution terminal 300, and other devices, or may be realized by a combination of a plurality of these devices. Further, the computer that executes the viewing program in the present embodiment may be any of the distribution server 200, the viewing terminal 100, and another device, or may be realized by a combination of these devices. In addition, the distribution system 1 uses other devices other than the viewing terminal 100, the distribution server 200, and the distribution terminal 300 as constituent elements of the distribution system 1, and causes the other devices to execute part of the processing in the distribution system 1. Good.

<Flow of distribution processing>
FIG. 4 is a sequence diagram showing an example of the flow of processing relating to live distribution and reproduction of content.

In step S100, the distribution unit 312 creates content data. The method of creating the content data will be described later.

In step S102, the distribution unit 312 transmits the content data to the distribution server 200 in real time via the communication unit 314. After that, the distribution unit 312 continues the live distribution of the content, for example, until the operation reception unit 311 receives an input operation instructing the end of the live distribution.

In step S104, the transmission/reception unit 211 of the distribution server 200 receives the content data in real time. The transmission/reception unit 211 may temporarily or non-temporarily store the received content data in the storage unit 220. The distribution server 200 waits until it receives a live distribution request from the viewing terminal 100.

In step S106, the operation accepting unit 111 of the viewing terminal 100 accepts an input operation for instructing the viewer to watch the content. When the operation receiving unit 111 receives the instruction, the reproducing unit 112 executes step S108.

In step S108, the reproduction unit 112 requests distribution of content via the communication unit 114.

In step S110, the server processing unit 212 of the distribution server 200 receives the distribution request via the transmission/reception unit 211. Upon receiving the request, the server processing unit 212 executes the process of step S112.

In step S112, the server processing unit 212 delivers the content data received from the delivery terminal 300 to the viewing terminal 100 in real time. After that, the server processing unit 212 continues the distribution of the content data until, for example, the transmitting/receiving unit 211 stops receiving the content data from the distribution terminal 300, or until the viewing terminal 100 notifies the end of the viewing.

In step S114, the communication unit 114 of the viewing terminal 100 receives the content data in real time.

In step S116, the reproduction unit 112 executes a reproduction process based on the content data that the communication unit 114 receives at any time. The reproduction process means a series of processes related to the reproduction of the content indicated by the content data. After that, the reproducing unit 112 continues to perform the content display process until it is determined YES in step S118 or YES in step S120.

In step S118, control unit 110 determines whether operation reception unit 111 has received an input operation instructing the end of viewing. If YES in step S118, that is, if the operation receiving unit 111 receives an input operation for instructing the end of viewing, the reproducing unit 112 executes step S122.

In step S122, the reproduction unit 112 ends the reproduction of the content. That is, the reproduction unit 112 ends the reproduction process. Further, the reproduction unit 112 notifies the distribution server 200 of the end of viewing through the communication unit 114. When receiving the notification, the server processing unit 212 of the distribution server 200 stops the live distribution to the viewing terminal 100.

On the other hand, if NO in step S118, that is, if the operation receiving unit 111 has not received the input operation for instructing the end of viewing, the control unit 110 further executes the process of step S120.

In step S120, the control unit 110 determines whether the live distribution has ended. For example, the control unit 110 may determine that the live distribution has ended when the communication unit 114 has finished receiving the content. If NO in step S120, control unit 110 continues the content display process in step S116, and repeats the determination process from step S118. On the other hand, if YES in step S120, the reproducing unit 112 executes the process of step S122. That is, the reproduction unit 112 ends the reproduction of the content.

(Creating content)
FIG. 5 is a diagram showing how the distributor creates content data. The distributor 701 creates content data by performing various inputs to the distribution terminal 300.

The content data includes at least definition parameters for constructing a virtual space, information for designating a background image, information for designating a character object indicating a avatar of a distributor, and setting parameters for a virtual camera. I'm out. Hereinafter, the “character object indicating the avatar” is simply referred to as the “avatar”.

For example, the distribution unit 312 includes information indicating the size of the virtual space and the coordinate axis system in the content data. Further, for example, the delivery unit 312 determines a character object indicating the avatar of the deliverer, and determines the facial expression and motion (motion) of the avatar in real time. Then, the distribution unit 312 includes the information regarding the determined avatar in the content data.

Further, for example, the distribution unit 312 acquires a background image and includes the background image in the content data. Note that the distribution unit 312 may include, in the content data, not the background image itself, but information that enables the background image to be specified or acquired later in the viewing terminal 100.

Note that the motion data of the model 702 that is a part of the distributor may be input to the distribution terminal 300. For example, as a type of the motion capture device 3020, an HMD may be attached to the model 702, and the position and orientation of the head of the model 702 may be detected by the HMD. Then, the information indicating the position and posture of the head transmitted from the HMD may be input to the distribution terminal 300 as motion data. Then, the distribution unit 312 may include the motion data of the model 702 in the content data as data that specifies the motion of the avatar. Thereby, as will be described later, when the content is reproduced on the viewing terminal 100, the movement indicated by the motion data can be reflected in the movement of the avatar. For example, the head of the avatar can be moved according to the movement of the head of the model 702. The motion data is data obtained by detecting the movement of the body of the model 702 via the motion capture device 3020.

Further, for example, the distributor 701 may input his/her voice 703 to the distribution terminal 300 via the microphone 3010. Then, the distribution unit 312 of the distribution terminal 300 may include the input voice data as the voice of the avatar in the content data. Alternatively, the distribution unit 312 may include a voice having the same content as the input voice data in the content data as the voice of the avatar. The “voice having the same content” means the voice 703 itself, a voice obtained by performing a predetermined voice processing on the voice 703, or a voice in which the content of the voice 703 is mechanically uttered by reading software or the like.

The distribution unit 312 collects these pieces of data input in real time as content data at any time, and transmits the content data to the distribution server 200 via the communication unit 314. Thereby, the information including the content data is delivered live from the delivery terminal 300 to the viewing terminal 100 through the delivery server 200 as an intermediary. Hereinafter, the information including the content data will be referred to as first information.

(Flow of content display processing)
FIG. 6 is a diagram showing an example of the flow of the reproduction process shown in step S116 of FIG.

In step S200, the reproduction unit 112 of the viewing terminal 100 acquires the first information via the communication unit 114.

In step S201, the space definition unit 112A defines a virtual space. The space definition unit 112A defines a spherical virtual space, for example, based on the definition parameters included in the content data.

In step S202, the background control unit 112B develops the background image in the defined virtual space. Specifically, the background control unit 112B pastes one background image specified by the content data in the virtual space so as to cover the entire spherical surface of the space. By the processing of S200 to S202, a virtual space in which the background image is developed is constructed.

In step S204, the object control unit 112C arranges the first object in the virtual space. The first object is an object indicating the avatar of the distributor. The object control unit 112C determines the type of the first object, the model data, the arrangement position, the posture, and the like based on the content data, and arranges the first object in the virtual space.

In step S206, the object control unit 112C arranges the second object in the virtual space. The second object is an object provided so as to shield a specific area of the background image attached to the virtual space, as described later. The second object may be a solid or a plane.

The size and shape of the second object are not limited. For example, the second object may be an object that serves as a base on which the first object that is an avatar stands, or may be ceiling lighting in a virtual space. In other words, the first object may be arranged on the second object, that is, in contact with the second object. However, it is desirable that the second object has a size and a shape that do not impair the original interest of the content in the live distribution.

The object control unit 112C determines the type of the second object, the model data, the placement position, the posture, and the like based on the content data, and places the second object in the virtual space.

In step S207, the object control unit 112C operates the first object according to the motion data included in the first information.

In step S208, the camera control unit 112D arranges the virtual camera in the virtual space. At this time, the camera control unit 112D arranges the virtual camera at a position different from the center point of the virtual space. An example of the arrangement position of the virtual camera will be shown later.

In step S210, the drawing unit 112E calculates the visual field area of the virtual camera from the definition parameter of the virtual space and the position and angle of the virtual camera.

In step S212, the drawing unit 112E draws an object, an image, or the like that appears in the vicinity of the calculated visual field area in the virtual space. In other words, the drawing unit 112E renders the image of the calculated visual field area in the virtual space.

In step S214, the display control unit 113 causes the display unit 152 or the like to display the image of the visual field area obtained by the rendering of the drawing unit 112E.

As described above, the content data is sequentially distributed from the distribution server 200. Therefore, the space definition unit 112A, the background control unit 112B, and the object control unit 112C, in the content data acquired by the communication unit 114, define the virtual space itself, the background image, the position of the object, the facial expression, the movement, and the virtual camera. When there is a change in the position, angle, or the like, the change is reflected in the virtual space each time. As a result, the content based on the content data is reproduced and displayed in real time on the viewing terminal 100.

Of the processing shown in FIG. 6, the processing of step S206 is not essential. That is, the object control unit 112C may not arrange the second object in the virtual space. Further, the object control unit 112C may arrange objects other than the first and second objects in the virtual space. The background control unit 112B may paste an image other than the background image in the virtual space.

According to the above processing, the visual field area of the virtual camera arranged at a position different from the center point is drawn, so that the center of the virtual space can be shown to the user. Therefore, the interest of the content in the live distribution is improved.

(Example of virtual space)
The state (A) of FIG. 7 is a diagram showing an example of a virtual space constructed in the distribution system. The virtual space 600 has a spherical structure centering on the center point 601 and covering the entire 360-degree direction connecting the north pole 602 and the south pole 603. In the state (A) of FIG. 7, a hemisphere in the upper half of the horizon 604 in the virtual space 600 is illustrated for the sake of simplicity. In the actual virtual space, the background image and the object can be arranged in the lower hemisphere as well. In addition, the latitude lines and the longitude lines shown in the virtual space 600 are for the purpose of description, and the latitude lines and the longitude lines may be omitted when the actual virtual space is defined and constructed.

In the virtual space 600, an XYZ coordinate system whose origin is the center point 601 is defined. The XYZ coordinate system is, for example, parallel to the real coordinate system. The horizontal direction, the vertical direction (vertical direction), and the front-back direction in the XYZ coordinate system are defined as the X axis, the Y axis, and the Z axis, respectively. Therefore, the X axis of the XYZ coordinate system (horizontal direction) is parallel to the x axis of the real coordinate system, the Y axis of the XYZ coordinate system (vertical direction) is parallel to the y axis of the real coordinate system, and the X axis of the XYZ coordinate system is The Z axis (front-back direction) is parallel to the z axis of the real coordinate system.

When the space definition unit 112A defines the celestial spherical virtual space 600 as shown in the state (A) of FIG. 7, the background control unit 112B inside the virtual space 600 is a background image that is a 360-degree omnidirectional image. Paste 630.

The object control unit 112C arranges the first object 610 of the avatar in the virtual space 600. Hereinafter, the first object is also referred to as an avatar. In addition, the object control unit 112C may arrange the second object 620 in the virtual space 600. In the illustrated example, the second object 620 is a base object on which the avatar 610 stands.

The camera control unit 112D arranges the virtual camera 640 in the virtual space 600. Here, the camera control unit 112D arranges the virtual camera 640 at an arbitrary position other than the center point of the virtual space 600 and the vicinity thereof at an arbitrary angle. The position and angle of the virtual camera 640 are specified by the content data.

The state (B) of FIG. 7 is a diagram showing an example of an image of the visual field area of the virtual camera shown in the state (A). In the example of the state (A) of FIG. 7, the camera control unit 112D arranges the virtual camera 640 at the same height as the face of the avatar 610. It can be said that the view area of the virtual camera 640 is an area of the virtual space 600 that can be viewed by the viewer via the viewing terminal 100. That is, it can be said that the position and angle of the virtual camera 640 are the same as the viewpoint of the viewer in the virtual space 600. In the case of the virtual camera 640 in the state (A) of FIG. 7, as shown in the state (B) of FIG. 7, an image 700 of the visual field area from the viewpoint of viewing the avatar 610 from almost right side is drawn and displayed on the display unit 152. ..

In this way, the reproduction unit 112 constructs the virtual space 600 by the cooperation of the space definition unit 112A, the background control unit 112B, and the object control unit 112C. Then, the camera control unit 112D arranges the virtual camera 640, and the drawing unit 112E draws the image 700 of the visual field area from the viewpoint of the virtual camera 640. Then, the display control unit 113 displays the drawn visual field image 700 on the display unit.

Here, the second object 620 is an object on which the avatar 610 stands, and an area (specific area) behind the second object 620 as viewed from the virtual camera 640 is an area shielded from the virtual camera 640. In other words, if the second object 620 is arranged in accordance with the position of a region (specific region) that the viewer wants to hide, such as a reflection part of the camera that captured the background image, the region is shielded by the second object 620. And can be hidden from the viewer.

(Another example of virtual space)
The object control unit 112C may arrange the second object so as to overlap the north pole, the south pole, or the horizon of the virtual space in the field of view from the virtual camera.

The state (A) of FIG. 8 is a diagram showing another example of the virtual space constructed in the distribution system. The state (B) of FIG. 8 is a diagram showing an example of an image of the visual field area of the virtual camera shown in the state (A). In the example of the state (A) of FIG. 8, the object control unit 112C hides the background image 630 near the north pole when the north pole and the peripheral area (specific area) enter the visual field area of the virtual camera 640. The second object 621 is arranged at such a predetermined position.

The background image 630 is an image that covers the entire spherical surface of the virtual space 600, that is, an omnidirectional image. When the omnidirectional image is pasted on the inner surface of the sphere in the virtual space 600, the background image may be distorted in at least one of the north pole 602, the south pole 603, and the horizon 604 of the sphere. By disposing the second object 621 in the vicinity of the region where the distortion of the background image may occur (specific region), the distortion of the background image can be made inconspicuous.

For example, even when the camera angle is such that the avatar 610 is looked up like the position and angle of the virtual camera 640 in the same figure, as shown in the state (B) of FIG. Since the peripheral area (specific area) is shielded by the second object 621, the distortion of the background image can be hidden from the viewer.

(Another example of virtual space)
It should be noted that the object control unit 112C may arrange the second object so as to cover the specific area in the background image in the visual field from the virtual camera.

The state (A) of FIG. 9 is a diagram showing another example of the virtual space constructed in the distribution system. The state (B) of FIG. 9 is a diagram showing an example of an image of the visual field area of the virtual camera shown in the state (A). In the example of the state (A) of FIG. 9, the object control unit 112C arranges the second object 622 vertically from the horizon 604 as illustrated.

Since the background image 630 is an omnidirectional image, when it is created, distortion may occur in a part of the region depending on the shape and size of the original image that is the source of the omnidirectional image. When an area (specific area) where such image distortion may occur is displayed by the virtual camera 640, the second object 622 is arranged so as to shield the area. Therefore, it is possible to prevent the viewer from seeing the distortion of the background image.

For example, as shown in the state (B) of FIG. 9, since it is possible to make the specific region occluded by the second object 622, the distortion of the background image 630 cannot be seen in the image 900 of the visual field region. You can

(Flow until viewing live stream)
The operation of selecting live distribution and starting viewing by the viewing terminal 100 may be performed by a GUI using a map.

FIG. 10 is a diagram showing an example of a display screen of the viewing terminal 100. As shown in the figure, the display screen 1700 displays a map based on the map data and an area 1701 (gray circle in the figure) where the live distribution can be viewed. Here, the map data may be stored in the storage unit 120 of the viewing terminal 100, or may be appropriately acquired from an external server or the like. Further, on the display screen 1700, a message 1702 indicating the date and time of the live distribution, contents, and the like, and an icon 1703 for instructing to start the live distribution may be displayed in association with the area 1701. Further, the display screen 1700 may display an icon 1704 indicating the current position of the viewing terminal 100, that is, the current position of the user. Further, on the display screen 1700, an object 1705 such as a building or landscape corresponding to the map data may be displayed.

The information on the current position of the viewing terminal 100 can be obtained when the viewing terminal 100 includes a GPS (Global Positioning System) receiver (not shown) and receives GPS radio waves. The GPS receiver transmits the position usage of the current position indicated by the obtained radio waves to the control unit 110, and the control unit 110 acquires this.

Further, the control unit 110 periodically receives information from the distribution server 200 from the distribution server 200, such as information indicating an area in which the live distribution can be viewed, information regarding the live distribution such as a time period when the live distribution is distributed, and information indicating the status of the live distribution. To the information displayed on the display screen 1700 and the display screen 1800.

FIG. 11 is a diagram showing an example of the user's first input operation on the display screen shown in FIG. 10. The first input operation is an input operation for designating a certain point on the map on the display screen displaying the map. In the illustrated example, when the user (icon 1703) is in the area 1701 where the live distribution can be viewed on the display screen 1800, when an input operation for selecting the icon 1703 for instructing the start of the live distribution is performed, The input operation is accepted as the first input operation.

When the operation receiving unit 111 receives the first input operation, the control unit 110 identifies the live distribution and receives the content data from the distribution server 200. When the content data is received, the reproduction unit 112 executes the processing shown in FIG. 6, that is, the construction of the virtual space, the placement of the virtual camera, and the drawing of the view image. This allows the user to watch the live distribution. In other words, when the distance between the position on the map of the viewing terminal 100 indicated by the position information of the current position and the point designated by the first input operation is within the predetermined range, the space definition unit of the reproduction unit 112. 112A and the background control unit 112B cooperate to build a virtual space for lively distributed content.

It should be noted that in the live distribution, the distribution time period may be fixed. Further, as shown in FIGS. 10 and 11, the content of the message 1702 may be changed depending on the state of live distribution, for example, whether or not live distribution is in progress.

FIG. 12 shows a display screen of the viewing terminal 100 when viewing live distributed content. When the user performs an input operation of selecting the icon 1703, viewing of the live distribution is started, and the display screen 1900 is displayed on the viewing terminal 100. As shown, the display screen 1900 includes at least the avatar 1901 of the distributor. On the display screen 1900, as the background image 1902, an image obtained by photographing a landscape in the physical space around 360 degrees around the point where the distributor avatar 1901 is arranged is displayed. Then, the avatar 1901, the second object 1903, and the like may be displayed at positions that shield a specific area of the background image.

Here, as shown in the figure, the second object 1903 is an object on the base on which the avatar 1901 stands, and shields a region (specific region) below the second object 1903 from the virtual camera. In other words, if the second object 1903 is arranged in accordance with the position of the area (specific area) that the viewer wants to hide, such as the reflection part of the camera that captured the background image, the area is shielded by the second object 1903. And can be hidden from the viewer.

[Embodiment 2]
The distribution terminal 300 may acquire the image of the play screen of the game played by the distributor using the game terminal included in the game system. Then, the reproducing unit 112 may construct a virtual space in which at least the omnidirectional image, the first object indicating the avatar of the distributor, and the third object displaying the image of the game play screen are arranged. In the present embodiment, the position of the virtual camera may be a position in the virtual space different from the center point of the virtual space, and the position may include the center point in the visual field region. Other positions may be used. The content data may be “live content” that is delivered live, or may be “recorded content” that is recorded data. Hereinafter, “live content” and “recorded content” are collectively referred to simply as “content”.

(System overview)
FIG. 13 is a diagram showing an outline of the distribution system 2 according to this embodiment. FIG. 13 differs from the distribution system 1 shown in FIG. 1 in that the game terminal 400 is included.

The game terminal 400 is a terminal for a distributor to play a game. The game terminal 400 transmits the image of the play screen of the game to the distribution terminal 300 at any time. Thereby, the distributor can create the content data of the content including the play screen when creating the content data in the distribution terminal 300.

The game terminal 400 may be a game system including the game terminal 400 together with another game terminal or a game server (not shown). That is, even when the distributor plays a game using a certain game terminal 400 included in the game system, a play image of the game is transmitted from the game terminal 400 or the game server to the distribution terminal 300. Good.

(Virtual space)
The state (A) of FIG. 14 is a diagram showing another example of the virtual space constructed in the distribution system 2. In the virtual space 1000 shown in the figure, a background image 1030 is pasted, and an avatar 1010 and a virtual camera 1040 are arranged. Although not shown in the figure, the second object may be arranged in the virtual space 1000 in this embodiment as well. Further, in the example of the figure, a chair object 1060 on which an avatar sits is also arranged.

In the present embodiment, the object control unit 112C arranges the third object 1021 that displays the image of the play screen of the game played by the distributor in the virtual space 1000. For example, as illustrated, the third object 1021 may be an object imitating a television, a personal computer, or the like. An image 1050 of the game play screen is displayed in at least a partial area of the third object 1021. Note that the object control unit 112C may arrange a third object capable of displaying the play screen, or the third object and the play screen may be displayed so that the play screen is apparently displayed on the third object. The image may be pasted or arranged so as to be superimposed when viewed from the virtual camera 1040.

The state (B) of FIG. 14 is a diagram showing an example of an image of the visual field area of the virtual camera shown in the state (A). In the example of the state (A) of FIG. 14, the camera control unit 112D arranges the virtual camera 1040 at the same height as the head of the avatar 1010 and behind the avatar 1010. In the case of the virtual camera 1040 in the state (A) of FIG. 14, as shown in the state (B) of FIG. 14, the field of view of the avatar 1010 looking at the play screen displayed on the third object 1021 is seen from directly behind. An image of the area is drawn and displayed on the display unit 152.

(Switching virtual camera 1)
In response to the reception of the specific second input operation in the viewing terminal 100, the camera control unit 112D sets the position and angle of the virtual camera 1040 to a position substantially similar to the viewpoint of the distributor corresponding to the avatar. You may switch to an angle.

The specific method for switching the position and angle of the virtual camera is not particularly limited. For example, the camera control unit 112D may move the position and the angle of the virtual camera arranged in the virtual space. Further, the camera control unit 112D may temporarily delete the virtual camera arranged in the virtual space and rearrange the virtual camera at the position and the angle after the switching.

Alternatively, a plurality of virtual cameras may be arranged in different positions and angles in the virtual space, and the virtual cameras used by the camera control unit 112D may be sequentially switched. Then, for example, in the drawing unit 112E, it may be determined which virtual camera field-of-view image is to be rendered and rendering is performed.

The state (A) of FIG. 15 is a diagram showing the virtual space after the second input operation is performed and the position and angle of the virtual camera are switched. When the second input operation is accepted, the camera control unit 112D switches the virtual camera to a position substantially the same as the viewpoint of the avatar 1010, that is, a position and an angle substantially the same as the distributor's viewpoint with respect to the play screen 1050. In the state (A) of FIG. 15, the view area of the avatar 1010 is indicated by α. Although the avatar 1010 and the virtual camera 1040 are superimposed in the actual virtual space 1000, the virtual camera 1040 is not shown for the sake of easy viewing of the drawing.

The state (B) of FIG. 15 is a diagram showing an example of an image of the visual field area of the virtual camera shown in the state (A). In the case of the viewpoint of the avatar 1010 shown in the state (A) of FIG. 15, the area α in the figure is the image of the visual field area. Therefore, as shown in the state (B) of FIG. 15, the up screen of the third object 1021 and the play screen 1050 is drawn and displayed as an image of the visual field area.

Thereby, the virtual camera 1040 can be switched to a position and an angle at which an area substantially similar to the field of view of the distributor is displayed in accordance with the second input operation of the user of the viewing terminal 100. Therefore, the user of the viewing terminal 100, that is, the viewer can view the video traced from the distributor's viewpoint according to the input operation of the user. Therefore, it is possible to increase the degree of freedom of camera work of the content, and thus to enhance the interest of the content.

(Switching virtual camera 2)
The camera control unit 112D may arrange the fourth object 1041 indicating the belongings of the avatar 1010 in the virtual space 1000. Then, in response to the reception of the specific third input operation in the viewing terminal 100, the camera control unit 112D sets the position and angle of the virtual camera 1040 to a position substantially similar to the position and orientation of the fourth object. And the angle may be switched.

The state (A) of FIG. 16 is a diagram illustrating the virtual space after the third input operation is performed and the position and the angle of the virtual camera are switched. When the third input operation is accepted, the camera control unit 112D switches the virtual camera to the position and angle according to the position and orientation of the fourth object 1041 that is the property of the avatar 1010. In the illustrated example, the belongings of the avatar 1010 are the camera objects 1041 held by the avatar 1010. In the state (A) of FIG. 15, the fourth object 1041 faces the face of the avatar 1010. Although the camera object 1041 and the virtual camera 1040 are superimposed in the actual virtual space 1000, the virtual camera 1040 is not shown for the sake of easy viewing of the drawing.

The state (B) of FIG. 16 is a diagram showing an example of an image of the visual field area of the virtual camera shown in the state (A). As shown in the state (B) of FIG. 16, the image 1300 of the visual field area from the viewpoint of the fourth object 1041 reflects the direction in which the fourth object is facing, that is, the direction of the face of the avatar 1010.

Accordingly, the virtual camera can be switched to a position and an angle at which the visual field image is viewed from a viewpoint substantially similar to the viewpoint of the property of the avatar, according to the third input operation of the user of the viewing terminal 100. Therefore, the user of the viewing terminal 100, that is, the viewer can view the image from the viewpoint of the belongings of the avatar according to the input operation of the user. Therefore, it is possible to increase the degree of freedom of camera work of the content, and thus to enhance the interest of the content.

(Adding a virtual camera)
The camera control unit 112D may arrange a plurality of virtual cameras. In addition, the camera control unit 112D sets a second virtual camera (that is, a second virtual viewpoint) different from the virtual camera 1040 to the first object in response to the specific fourth input operation accepted by the viewing terminal 100. You may arrange|position at the position and angle which the specific part of is reflected. Here, the specific part of the avatar is, for example, the face or the hand of the avatar. Further, the drawing unit 112E may render an image including both an image in the visual field area of the virtual camera and an image in the visual field area of the second virtual camera.

The state (A) of FIG. 17 is a diagram showing the virtual space after the fourth input operation is performed and the position and angle of the virtual camera are switched. When the fourth input operation is accepted, the camera control unit 112D arranges the second virtual camera 1070 in the virtual space 1000 at a position and an angle where a specific part of the avatar 1010 is reflected. In the illustrated example, the second virtual camera 1070 is arranged at a position and an angle at which the face of the avatar 1010 is reflected.

The state (B) of FIG. 17 is a diagram showing an example of an image 1400 of the visual field regions of the virtual camera 1040 and the second virtual camera 1070 shown in the state (A). As shown in the state (B) of FIG. 16, the drawing unit 112E displays, as an image of the visual field area, an image 1080 of the visual field area of the second virtual camera 1070 by superimposing it on the image 1400 of the visual field area of the virtual camera 1040. .. Note that the image of the field of view of the second virtual camera 1070 may be displayed only in a part of the area instead of the entire area as illustrated.

Thereby, the specific portion of the avatar can be displayed on the second virtual camera 1070 in accordance with the fourth input operation by the user of the viewing terminal 100. Therefore, the viewer can simultaneously view both the image of the field of view by the virtual camera and the image of the specific part of the avatar by the second virtual camera according to the input operation of the viewer. Therefore, it is possible to increase the degree of freedom of camera work of the content, and thus to enhance the interest of the content.

In addition, when a predetermined input operation is accepted in the viewing terminal 100, the drawing unit 112E replaces and draws the image 1400 of the visual field area of the virtual camera 1010 and the image 1080 of the visual field area of the second virtual camera 1070. Good.

(Switching virtual cameras 3)
When the specific fifth input operation is received by the viewing terminal 100, the camera control unit 112D may switch the position and the angle of the virtual camera 1040 to the position and the angle at which the specific part of the first object appears.

The state (A) of FIG. 18 is a diagram showing the virtual space after the fifth input operation is performed and the position and angle of the virtual camera 1040 are switched. When the fifth input operation is accepted, the camera control unit 112D switches the position and angle of the virtual camera 1040 to the position and angle at which the specific part of the avatar 1010 is shown. In the state (A) of FIG. 18, the camera control unit 112D arranges the virtual camera 1040 at a position and an angle at which the face of the avatar 1010 can be seen from the front. The position and angle of the virtual camera 1040 are set relative to the position of the specific part of the avatar 1010. Therefore, when the position and the angle of the specific part of the avatar 1010 change, the camera control unit 112D moves or rearranges the virtual camera 1040 to the relative position according to the changed position and angle. That is, when a specific part of the avatar 1010 moves, the virtual camera 1040 also follows and moves so as to reach a predetermined relative position. In this way, even if the specific part of the avatar 1010 moves, the virtual camera 1040 also follows and moves, so that the specific part of the avatar 1010 can always be photographed.

The state (B) of FIG. 18 is a diagram showing an example of the image 1031 of the visual field area of the virtual camera 1040 shown in the state (A). As shown in the state (B) of FIG. 18, the virtual camera 1040 is in a position where the face of the avatar 1010 can be photographed from the front.

Thereby, the virtual camera 1040 can be switched to the position and the angle at which the specific part of the avatar is reflected in accordance with the fifth input operation of the user of the viewing terminal 100. Therefore, it is possible to increase the degree of freedom of camera work of the content, and thus to enhance the interest of the content.

[Modification of the game]
In the above description, the game is independent of the content distributed by the distribution system 2, and the game screen is included in the content when the game screen is loaded from the game terminal 400 to the distribution terminal 300. ..

However, in the distribution system 2, the game terminal 400 and the distribution terminal 300 may be integrated. In this case, the distribution unit 312 may provide a game for the distributor to play in the virtual space via the avatar. An example of such a game is a VR game realized by using an HMD. In this case, the distribution unit 312 may execute various processes related to the progress of the game to create the game screen/distribution screen. In this case, the game play screen included in the content data is the content itself including the avatar of the distributor.

FIG. 19 is a diagram showing another example of the image of the visual field area of the virtual camera in the distribution system 2. On the game screen 1600 shown in FIG. 19, contents as a game screen including at least the avatar 1010 of the distributor are displayed. In the illustrated example, an enemy character 1610 is further arranged on the game screen 1600, and is in competition with an avatar 1010 operated by the distributor (in which the motion of the distributor is reflected).

On the game screen 1600, the avatar 1010 is the player character. In the illustrated example, the avatar 1010 corresponding to the distributor may move on the game screen 1600 according to the motion of the distributor (model 702 or the like). That is, the movement of the distributor who is the target of the motion data acquisition may be reflected as it is on the avatar 1010. For example, as described above, when the model 702 is wearing the HMD, the head direction of the avatar 1010 may change according to the movement of the head of the model 702.

Further, a virtual camera that automatically tracks may be provided at a position facing the line of sight of the avatar 1010 corresponding to the distributor. With this virtual camera, it is possible to reliably capture the line of sight of the avatar 1010 corresponding to the distributor. Therefore, it is possible to prevent the dissatisfaction of the viewer that the avatar 1010 corresponding to the distributor does not look at himself. A mode for displaying the image from the virtual camera may be provided separately. For example, the virtual camera may be arranged in the virtual space, and when the viewer taps the virtual camera, the mode may be shifted to the above mode.

[Modification]
The distribution system 1 or 2 combines the arrangement method and the arrangement position of the second object described in the first embodiment, the position and angle switching of the virtual camera described in the second embodiment, and the arrangement of the second virtual camera. You may execute.

For example, in the distribution system 1, when the operation receiving unit 111 receives the second input operation or the third input operation, the object control unit 112C of the viewing terminal 100, as described in the second embodiment, the position of the virtual camera. And the angle may be switched. Further, in the distribution system 1, when the operation receiving unit 111 receives the fourth input operation, the object control unit 112C of the viewing terminal 100 arranges the second virtual camera and renders it as described in the second embodiment. The unit 112E may draw images of the visual field areas of both the virtual camera and the second virtual camera.

In addition, in the examples of the first and second embodiments, the viewing terminal 100 executes the construction of the virtual space, the placement of the virtual camera, and the drawing of the image of the viewing area based on the content data created by the distribution terminal 300. .. However, the distribution unit 312 of the distribution terminal 300 according to the distribution systems 1 and 2 may execute the construction of the virtual space, the arrangement of the virtual cameras, and the drawing of the image of the visual field area. In other words, the distribution unit 312 of the distribution terminal 300 may execute the processes of steps S201 to S212 in FIG. 6, and the drawn image of the visual field area may be distributed as content to the viewing terminal 100 via the distribution server 200. ..

In this case, the switching of the virtual cameras described in the second embodiment can be realized by the following method, for example. That is, this is a method in which the distribution unit 312 of the distribution terminal 300 draws the entire virtual space in advance or draws an image of the visual field area of the virtual camera at all possible positions and angles. In this case, when the second input operation, the third input operation, the fourth input operation, or the fifth input operation is executed in the viewing terminal 100, the viewing terminal 100 informs the distribution server 200 that the operation is performed. To be As described above, the distribution server 200 stores the drawn images corresponding to the images of all the visual field regions, and the server processing unit 212 displays the image of the visual field region according to the content of the operation transmitted from the viewing terminal 100. , To the viewing terminal 100. Thereby, the viewpoint of the virtual camera can be switched according to the input operation on the viewing terminal 100.

[Example of software implementation]
The control block of each control block of the control units 110, 210, and 310 may be realized by a logic circuit (hardware) formed in an integrated circuit (IC chip) or the like, or may be realized by software.

In the latter case, the information processing apparatus including at least one of the control units 110, 210, and 310 includes a computer that executes a program instruction that is software that realizes each function. The computer includes, for example, one or more processors and a computer-readable recording medium that stores the program. Then, in the computer, the processor reads the program from the recording medium and executes the program to achieve the object of the present invention. As the processor, for example, a CPU (Central Processing Unit) can be used. As the recording medium, a "non-transitory tangible medium" such as a ROM (Read Only Memory), a tape, a disk, a card, a semiconductor memory, a programmable logic circuit, or the like can be used. Further, a RAM (Random Access Memory) for expanding the program may be further provided. The program may be supplied to the computer via any transmission medium (communication network, broadcast wave, etc.) capable of transmitting the program. Note that one aspect of the present invention can also be realized in the form of a data signal embedded in a carrier wave, in which the program is embodied by electronic transmission.

The present invention is not limited to the above-described embodiments, but various modifications can be made within the scope of the claims, and embodiments obtained by appropriately combining the technical means disclosed in the different embodiments Is also included in the technical scope of the present invention.

[Appendix]
The contents according to one aspect of the present invention are listed below.

(Item 1) The program (viewing program 131) has been described. According to an aspect of the present disclosure, a program includes a processor (10) and a memory (11) as a viewing terminal (100) for a distributor to view content to be distributed in real time using a distribution terminal (300). , A program executed by a computer, the program being information relating to contents to a processor, and being information distributed at any time from a distribution terminal, including at least motion data in which motion of a distributor's body is detected. A step of receiving the first information (S200), a step of constructing a virtual space in which a background image is expanded (S201 to S202), and a step of arranging a first object which is an avatar corresponding to a distributor in the virtual space. (S204), in response to receiving the first information during the real-time distribution, a step of operating the first object according to the motion data (S207), and the center of the virtual space in the virtual space. An image obtained by rendering: a step of arranging the virtual viewpoint at a position different from the point and including the center point in the visual field area (S208); a step of rendering an image of the virtual viewpoint visual field area (S212); Is displayed on the display unit (S214).

According to the above processing, the visual field area of the virtual viewpoint arranged at a position different from the center point is drawn, so that the center of the virtual space can be shown to the user. Therefore, the interest of the content in the live distribution is improved.

(Item 2) In (Item 1), the program may cause the processor to further execute a step (S206) of arranging the second object in the virtual space so as to occlude a specific region of the background image. ..

When a background image covering the virtual space is created, a part of the background image may be distorted depending on the shape and size of the original original image. According to the above-described processing, for example, when displaying the image of the visual field region of the virtual viewpoint in the region where such distortion of the background image may occur, the second object is arranged so as to shield the region. Therefore, it is possible to prevent the viewer from showing the distortion of the background image.

(Item 3) In (Item 2), the step of arranging the first object may arrange the first object on the second object.

According to the above processing, when the viewer focuses on the first object, the area in which the image distortion may occur is blocked by the second object. Therefore, it is possible to prevent the viewer from showing the distortion of the background image.

(Item 4) In any one of (Item 1) to (Item 3), the program further causes the processor to display a map based on the map data, and a step for designating a certain point on the map. The step of accepting a 1-input operation and the step of executing and constructing a virtual space are based on the first input operation, and the background image is an image of the surroundings of the position in the real space corresponding to the point. It may be.

According to the processing described above, it is possible to live-distribute an image with an actual landscape as a background at a specific point. Therefore, the interest of the content in the live distribution is improved.

(Item 5) In (Item 4), the program further causes the processor to execute the step of acquiring position information indicating the current position of the viewing terminal, and the step of constructing is on the map of the viewing terminal indicated by the position information. The virtual space may be constructed when the distance between the position at and the point designated by the first input operation is within a predetermined range.

According to the above processing, the live can be viewed by actually going near the designated point and then executing the first operation. Therefore, the interest of live distribution can be improved.

(Item 6) In the item of any one of (item 1) to (item 5), the program determines the position of the virtual viewpoint and the position of the virtual viewpoint in response to the second input operation accepted by the viewing terminal. The step of changing the angle to a position and an angle that are substantially the same as the viewpoint of the distributor corresponding to the first object is executed, and the step of rendering is performed by changing the position and the angle to a position that is substantially the same as the viewpoint of the distributor. An image of the field of view may be rendered after the position and angle have been switched.

According to the above processing, the virtual viewpoint can be switched to a position and an angle at which an area substantially similar to the field of view of the distributor is projected according to the second input operation by the user of the viewing terminal. Therefore, the user of the viewing terminal, that is, the viewer can view the video in which the distributor's viewpoint is traced according to the input operation of the user. Therefore, according to the processing described above, the degree of freedom in camera work for live distribution can be increased, and thus the interest of live distribution can be improved.

(Item 7) In any one of (Item 1) to (Item 6), the program arranges a fourth object indicating the belongings of the avatar in the virtual space in the processor, and in the viewing terminal, In response to the acceptance of the three-input operation, the step of switching the position and angle of the virtual viewpoint to the position and angle substantially similar to the position and orientation of the fourth object, and the step of rendering, 4 The image of the visual field region may be rendered after the position and angle of the virtual viewpoint are switched by the step of switching to the position and angle substantially similar to the position and orientation of the object.

According to the above-described processing, the virtual viewpoint can be switched to the position and the angle at which the visual field image is projected from a viewpoint substantially similar to the viewpoint of the avatar's inventory, according to the third input operation of the user of the viewing terminal. Here, the belongings of the avatar are, for example, camera objects held by the avatar. According to the above-described processing, the user of the viewing terminal, that is, the viewer can view the image from the viewpoint of the avatar's belongings according to the input operation of the user. Therefore, it is possible to increase the degree of freedom of camera work for live distribution, and thus to enhance the interest of live distribution.

(Item 8) In any one of (Item 1) to (Item 7), the program is different from the virtual viewpoint in response to the processor receiving the fourth input operation at the viewing terminal. The step of arranging the two virtual viewpoints at the position and the angle at which the specific part of the first object is reflected and performing the rendering includes both the image of the visual field area of the virtual viewpoint and the image of the visual field area of the second virtual viewpoint. An image including may be rendered.

According to the above process, the specific part of the avatar can be projected from the second virtual viewpoint in accordance with the fourth input operation by the user of the viewing terminal. Here, the specific part of the avatar is, for example, the face or the hand of the avatar. According to the above-described process, the user of the viewing terminal, that is, the viewer simultaneously receives both the image of the visual field area from the virtual viewpoint and the image of the specific part of the avatar from the second virtual viewpoint according to the input operation of the user. You can see. Therefore, it is possible to increase the degree of freedom of camera work for live distribution, and thus to enhance the interest of live distribution.

(Item 9) In the item of any one of (Item 1) to (Item 8), the program responds to the processor and further to the position of the virtual viewpoint and the position of the virtual viewpoint in response to the reception of the fifth input operation at the viewing terminal. The step of switching the angle to the position and the angle of the specific part of the first object is executed, and the step of rendering is performed by switching the position and the angle of the specific part of the first object to the position and the angle of the virtual viewpoint. The image of the field-of-view region after rendering may be rendered.

According to the above process, the virtual viewpoint can be switched to the position and the angle at which the specific part of the avatar appears in response to the fifth input operation by the user of the viewing terminal. Therefore, it is possible to increase the degree of freedom of camera work of the content, and thus to enhance the interest of the content.

(Item 10) The program (distribution program 331) has been described. According to an aspect of the present disclosure, a program is executed by a computer as a distribution terminal (200, 300) including a processor (20, 30) and a memory (21, 31) and used by a distributor for real-time distribution of content. The program is a step of constructing a virtual space in which a background image is developed in a processor (S201 to S202), and a step of arranging a first object which is an avatar corresponding to a distributor in the virtual space ( S204), a step of arranging a virtual viewpoint at a position different from the center point of the virtual space in the virtual space (S208), a step of rendering an image of a visual field of the virtual viewpoint (S212), The step of delivering the image as content to the viewing terminal in real time (S102, S112) is executed.

According to the above processing, the visual field area of the virtual viewpoint arranged at a position different from the center point is drawn, so that the center of the virtual space can be shown to the user. Therefore, the interest of the content in the live distribution is improved.

(Item 11) The method for the computer to execute the program has been described. According to an aspect of the present disclosure, the program (viewing program 131) is executed by a computer including a processor (10) and a memory (11). The method is a method in which a processor executes each step of the program described in (Item 1). The method according to (Item 11) has the same operational effects as the program according to (Item 1).

(Item 12) The information processing device has been described. According to an aspect of the present disclosure, the information processing apparatus controls the operation of the information processing apparatus (viewing terminal 100) by executing a memory (11) that stores a program according to (Item 1) and the program. And a processor (10) for The information processing device according to (Item 12) has the same operational effects as the program according to (Item 1).

1 distribution system, 4 network, 10, 20, 30 processor, 11, 21, 31 memory, 12, 22, 32 storage, 15, 35 touch screen, 17 camera, 18 distance measuring sensor, 100 viewing terminal, 110, 210, 310 control unit, 111, 311 operation reception unit, 112 reproduction unit, 112A space definition unit, 112B background control unit, 112C object control unit, 112D camera control unit, 112E drawing unit, 113, 313 display control unit, 114, 314 communication Section, 120, 220, 320 storage section, 131 viewing program, 132 viewer information, 332 distributor information, 151, 351 input section, 152, 352 display section, 200 distribution server, 211 transmitting/receiving section, 212 server processing section, 213 Data management unit, 231,331 distribution program, 300 distribution terminal, 312 distribution unit, 1020,3030 controller, 1030 storage medium, 3010 microphone, 3020 motion capture device, 13,23,33 communication IF,
14, 24, 34 I/O IF

Claims (12)

A program executed by a computer as a viewing terminal for a distributor to view content to be distributed in real time using a distribution terminal, the program including a processor and a memory,
The program causes the processor to
A step of receiving first information, which is information related to the content, which is information that is distributed from the distribution terminal at any time and that includes at least motion data in which the physical movement of the distributor is detected,
A step of constructing a virtual space in which a background image is expanded,
Placing a first object that is an avatar corresponding to the distributor in the virtual space;
Operating the first object according to the motion data in response to receiving the first information while the real-time delivery is performed;
Arranging a virtual viewpoint at a position in the virtual space different from the center point of the virtual space and including the center point in the visual field region,
Rendering an image of the field of view of the virtual viewpoint,
A program for causing the display unit to display the image obtained by the rendering. The program further causes the processor to:
The program according to claim 1, which causes a step of arranging a second object in the virtual space so as to cover a specific area of the background image. The program according to claim 2, wherein the step of arranging the first object arranges the first object on the second object. The program further causes the processor to:
Displaying a map based on the map data,
A step of receiving a first input operation for designating a certain point on the map,
The step of constructing constructs the virtual space based on the first input operation,
The program according to any one of claims 1 to 3, wherein the background image is an image captured around a position in a physical space corresponding to the point. The program further causes the processor to:
Performing a step of acquiring position information indicating the current position of the viewing terminal,
In the constructing step, if the distance between the position of the viewing terminal on the map indicated by the position information and the point designated by the first input operation is within a predetermined range, the virtual space is set. The program according to claim 4, which constructs. The program further causes the processor to:
In response to the second input operation being accepted at the viewing terminal, a step of switching the position and angle of the virtual viewpoint to a position and angle substantially similar to the viewpoint of the distributor corresponding to the first object is executed. Let
The rendering step renders an image of the visual field after the position and the angle of the virtual viewpoint are switched by the step of switching to a position and an angle that are substantially the same as the viewpoint of the distributor. The program according to any one of 1. The program further causes the processor to:
Arranging a fourth object indicating the belongings of the avatar in the virtual space;
Switching the position and angle of the virtual viewpoint to a position and angle substantially similar to the position and orientation of the fourth object in response to the reception of the third input operation at the viewing terminal. ,
The rendering step renders an image of the visual field area after the position and angle of the virtual viewpoint are switched by the step of switching to a position and an angle that are substantially the same as the position and orientation of the fourth object. The program according to any one of 1 to 6. The program further causes the processor to:
In response to the fourth input operation being accepted in the viewing terminal, a step of arranging a second virtual viewpoint different from the virtual viewpoint at a position and an angle at which a specific part of the first object is reflected,
The program according to any one of claims 1 to 7, wherein the rendering step renders an image including both an image of a visual field area of the virtual viewpoint and an image of a visual field area of the second virtual viewpoint. .. The program further causes the processor to:
In response to receiving a fifth input operation in the viewing terminal, executing a step of switching a position and an angle of the virtual viewpoint to a position and an angle at which a specific part of the first object appears.
The rendering step renders an image of the visual field after the position and angle of the virtual viewpoint are switched by the step of switching to a position and an angle at which a specific part of the first object is reflected. The program according to any one of 1. A program executed by a computer as a distribution terminal used by a distributor for real-time distribution of content, comprising a processor and a memory,
The program causes the processor to
A step of constructing a virtual space in which a background image is expanded,
Placing a first object that is an avatar corresponding to the distributor in the virtual space;
Arranging a virtual viewpoint at a position different from the center point of the virtual space in the virtual space,
Rendering an image of the field of view of the virtual viewpoint,
Delivering the image obtained by the rendering to the viewing terminal as content in real time. A method by which a computer executes a program,
The computer includes a processor and a memory,
A method in which the processor executes the steps of the program of claim 1. An information processing device,
The information processing device,
A memory for storing the program according to claim 1;
An information processing apparatus, comprising: a processor that controls the operation of the information processing apparatus by executing the program.

Images