JP2021099825A - Program, method, and information processing device - Google Patents

Abstract

To provide a program capable of improving amusement of content in live distribution, a method, and an information processing device.SOLUTION: In a distribution system comprising a distribution server, a distribution terminal, and a viewing terminal, for distributing content to a viewer in real time, a viewing program in a distribution unit of the distribution terminal causes a processor to execute the steps of: receiving first information (S200); constructing a virtual space (S201 and S202); arranging a first object (S204); operating the first object according to motion data (S207); arranging a virtual viewpoint of the virtual space in a position which differs from a center point of the virtual space and includes the center point in a visual field region (S208); and rendering an image of the visual field region at the virtual viewpoint (S212).SELECTED DRAWING: Figure 6

Description

The present invention relates to programs, methods, and information processing devices.

Conventionally, a technique for showing an omnidirectional image to a user has been developed. For example, there is a technique in which a 360-degree panoramic image is pasted inside a spherically modeled virtual space, and the panoramic image is cut out in the field of view of the virtual camera and displayed. In addition, a technique has been developed for live distribution of a state in which a distributor performs live music or a live game via an avatar as content.

“Facebook Social VR Demo --Oculus Connect 2016”, [online], YouTube (registered trademark), [Searched January 24, 2019], Internet (URL: https://www.youtube.com/watch?v=YuIgyKLPt3s) )

One aspect of the present invention is aimed at improving the enjoyment of content in live distribution.

In order to solve the above problems, the program according to one aspect of the present invention is executed by a computer as a viewing terminal for viewing content to be distributed in real time by a distributor using a distribution terminal, which includes a processor and a memory. It is a program to be done. The program includes a step of receiving the first information to the processor, which is information related to the content and is information including motion data at least detecting the movement of the distributor's body, which is distributed from the distribution terminal at any time, and a background image. The step of constructing the expanded virtual space, the step of arranging the first object which is the avatar corresponding to the distributor in the virtual space, and the step of receiving the first information during the real-time distribution. In response, a step of operating the first object according to the motion data and a step of arranging the virtual viewpoint in the virtual space at a position different from the center point of the virtual space and including the center point in the viewing area. , The step of rendering the image in the visual field area of the virtual viewpoint and the step of displaying the image obtained by the rendering on the display unit are executed.

According to one aspect of the present invention, it is possible to improve the enjoyment of content in live distribution.

It is a figure which shows the outline of the distribution system which concerns on Embodiment 1. FIG. It is a figure which shows the hardware configuration of the various devices included in the distribution system. It is a block diagram which shows the functional configuration of various devices included in the distribution system. It is a sequence diagram which shows an example of the processing flow which concerns on the live distribution and reproduction of the content in the distribution system. It is a figure which shows the state of the content creation by a distributor in the said distribution system. It is a figure which shows an example of the flow of the reproduction processing in the said distribution system. The state (A) is a diagram showing an example of a virtual space constructed in the distribution system. The state (B) is a diagram showing an example of an image of the field of view region of the virtual camera shown in the state (A). It is a figure which shows an example of the image of the visual field area of the virtual camera shown in FIG. 7. The state (A) is a diagram showing another example of the virtual space constructed in the distribution system. The state (B) is a diagram showing an example of an image of the field of view region of the virtual camera shown in the state (A). It is a figure which shows an example of the display screen of a viewing terminal. It is a figure which shows an example of the operation of the user with respect to the display screen shown in FIG. It is a figure which shows the display screen of the viewing terminal at the time of viewing live-delivered content. It is a figure which shows the outline of the distribution system which concerns on Embodiment 2. The state (A) is a diagram showing another example of the virtual space constructed in the distribution system. The state (B) is a diagram showing an example of an image of the field of view region of the virtual camera shown in the state (A). The state (A) is a diagram showing a virtual space after the second input operation is performed and the position and angle of the virtual camera are switched. The state (B) is a diagram showing an example of an image of the field of view region of the virtual camera shown in the state (A). The state (A) is a diagram showing a virtual space after the third input operation is performed and the position and angle of the virtual camera are switched. The state (B) is a diagram showing an example of an image of the field of view region of the virtual camera shown in the state (A). The state (A) is a diagram showing a virtual space after the fourth input operation is performed and the position and angle of the virtual camera are switched. The state (B) is a diagram showing an example of an image of the field of view region of the virtual camera shown in the state (A). The state (A) is a diagram showing a virtual space after the fifth input operation is performed and the position and angle of the virtual camera are switched. The state (B) is a diagram showing an example of an image of the field of view region of the virtual camera shown in the state (A). It is a figure which shows another example of the image of the field of view area of a virtual camera in the said distribution system.

The distribution system according to the present disclosure is a system for distributing content to viewers in real time. Hereinafter, the distribution system will be described with reference to the drawings. It should be noted that the present invention is not limited to the following description and examples in the drawings, and is indicated by the scope of claims, and all modifications within the meaning and scope equivalent to the scope of claims are included in the present invention. Is intended. In the following description, the same elements are designated by the same reference numerals in the description of the drawings, and duplicate description will not be repeated.

[Embodiment 1]
<Outline of operation of distribution system 1>
FIG. 1 is a diagram showing an outline of the distribution system 1 according to the present embodiment. The distribution system 1 includes one or more distribution servers 200, one or more distribution terminals 300, and one or more viewing terminals 100.

(Distribution terminal 300)
The distribution terminal 300 is a device that is a main body for live distribution of contents. The distribution terminal 300 is realized by, for example, a personal computer (PC) or a smartphone. Hereinafter, the user of the distribution terminal 300 will be referred to as a "distributor". The distribution terminal 300 creates content data in response to an input operation by the user of the distribution terminal 300. The content data is data for viewing the content created by the distributor on the viewing terminal 100. The distribution terminal 300 sequentially creates content data and distributes it to the distribution server 200 in real time. Hereinafter, the distribution of content data in real time is also referred to as "live distribution".

(Distribution server 200)
The distribution server 200 is a server that distributes content data received from the distribution terminal 300 in real time to the viewing terminal 100. The distribution server 200 is also a server that mediates the transmission and reception of various information between the viewing terminal 100 and the distribution terminal 300.

The distribution server 200 may collect various information related to live distribution from one or more viewing terminals 100 and transmit the various information to the distribution terminal 300. For example, the distribution server 200 may aggregate the number of viewing terminals 100 currently playing the content, that is, the current number of viewers, and transmit the content to the distribution terminal 300. Further, the distribution server 200 may aggregate the cumulative number of viewers from the start of distribution of the content and transmit it to the distribution terminal 300. Further, the distribution server 200 may specify the viewing terminal 100 that has transmitted the above-mentioned input information, and may transmit the information indicating the viewing terminal 100 to the distribution terminal 300.

(Viewing terminal 100)
The viewing terminal 100 is a device that receives content data in real time and generates and reproduces an image using the content data. In other words, the viewing terminal 100 is a device that renders and reproduces the content in real time. In the following description, the real-time rendering and playback of the content is also referred to simply as "playing the content". The viewing terminal 100 is realized by, for example, a PC or a smartphone. The viewing terminal 100 receives and reproduces content data from the distribution server 200 in response to an input operation by the user of the viewing terminal 100. Specifically, the viewing terminal 100 receives data such as motion as content data from the distribution server 200, and the viewing terminal 100 generates and reproduces a video (moving image) by real-time rendering using these data. .. Hereinafter, the user of the viewing terminal 100 will be referred to as a "viewer". The method of playing the content will be described in detail later.

The viewing terminal 100 may not only reproduce the content data, but may also be able to transmit the information specified or input by the input operation of the viewer to the distribution terminal 300 via the distribution server 200. .. For example, the viewing terminal 100 may transmit the comment input by the viewer to the distribution terminal 300 via the distribution server 200.

In this embodiment, an example in which the distribution server 200 and the distribution terminal 300 are separate devices will be described. However, the distribution server 200 and the distribution terminal 300 may be an integrated device. For example, one PC may have both a function as a distribution server 200 and a function as a distribution terminal 300.

<Hardware configuration of distribution system 1>
FIG. 1 is a diagram showing a hardware configuration of the distribution system 1. As shown in the figure, the distribution system 1 includes a distribution terminal 300, a distribution server 200, and a plurality of viewing terminals 100. The viewing terminal 100 and the distribution terminal 300 are connected to the distribution server 200 via the network 4.

(Network 4)
The network 4 is composed of various mobile communication systems constructed by the Internet and a wireless base station (not shown). Examples of this mobile communication system include so-called 3G and 4G mobile communication systems, LTE (Long Term Evolution), and wireless networks (for example, Wi-Fi (registered trademark)) that can be connected to the Internet by a predetermined access point. Be done.

(Distribution server 200)
The distribution server 200 (computer, information processing device) may be a general-purpose computer such as a workstation or a personal computer. The distribution server 200 includes a processor 20, a memory 21, a storage 22, a communication IF 23, and an input / output IF 24. These configurations included in the distribution server 200 are electrically connected to each other by a communication bus.

(Viewing terminal 100)
The viewing terminal 100 (computer, information processing device) may be a mobile terminal such as a smartphone, a feature phone, a PDA (Personal Digital Assistant), or a tablet computer. The viewing terminal 100 may be a playback device suitable for playing back the content. As shown in the figure, the viewing terminal 100 includes a processor 10, a memory 11, a storage 12, a communication interface (IF) 13, an input / output IF 14, and a touch screen 15 (display unit). These configurations included in the viewing terminal 100 are electrically connected to each other by a communication bus. The viewing terminal 100 may be provided with an input / output IF 14 to which a display (display unit) configured separately from the viewing terminal 100 main body can be connected instead of or in addition to the touch screen 15.

Further, as shown in FIG. 2, the viewing terminal 100 may be configured to be communicable with one or more controllers 1020. The controller 1020 establishes communication with the viewing terminal 100 according to a communication standard such as Bluetooth (registered trademark). The controller 1020 may have one or more buttons or the like, and transmits an output value based on a user's input operation to the buttons or the like to the viewing terminal 100. Further, the controller 1020 may have various sensors such as an acceleration sensor and an angular velocity sensor, and transmits the output values of the various sensors to the viewing terminal 100.

Further, the controller 1020 may be configured to be detachable from the viewing terminal 100. In this case, a coupling portion with the controller 1020 may be provided on at least one surface of the housing of the viewing terminal 100. When the viewing terminal 100 and the controller 1020 are connected by wire via the coupling portion, the viewing terminal 100 and the controller 1020 transmit and receive signals via wire.

As shown in FIG. 2, the viewing terminal 100 may accept the attachment of a storage medium 1030 such as an external memory card via the input / output IF14. As a result, the viewing terminal 100 can read the program and data recorded on the storage medium 1030. The program recorded on the storage medium 1030 is, for example, a viewing program.

The viewing terminal 100 may store the viewing program acquired by communicating with an external device such as the distribution server 200 in the memory 11 of the viewing terminal 100, or the viewing program acquired by reading from the storage medium 1030 in the memory. It may be stored in 11.

As described above, the viewing terminal 100 includes a communication IF 13, an input / output IF 14, and a touch screen 15 as an example of a mechanism for inputting information to the viewing terminal 100. Each of the above-mentioned parts as an input mechanism can be regarded as an operation part configured to accept a user's input operation.

For example, when the operation unit is composed of the touch screen 15, the viewing terminal 100 identifies and accepts the user's operation performed on the input unit 151 of the touch screen 15 as the user's input operation. Alternatively, when the operation unit is composed of the communication IF 13, the viewing terminal 100 identifies and accepts a signal (for example, an output value) transmitted from the controller 1020 as an input operation of the user. Alternatively, when the operation unit is composed of the input / output IF14, a signal output from an input device (not shown) different from the controller 1020 connected to the input / output IF14 is specified and accepted as a user input operation.

(Distribution terminal 300)
The distribution terminal 300 (computer, information processing device) may be a mobile terminal such as a smartphone, a PDA (Personal Digital Assistant), or a tablet computer. Further, the distribution terminal 300 may be a computer such as a server, a desktop personal computer, a laptop computer, or a tablet, and a group of computers in which these are combined.

The distribution terminal 300 may be a group of devices in which a display device, an input device, and an information processing device are connected so as to be suitable for live distribution. As shown in the figure, the distribution terminal 300 includes a processor 30, a memory 31, a storage 32, a communication interface (IF) 33, an input / output IF 34, and a touch screen 35 (display unit). These configurations included in the distribution terminal 300 are electrically connected to each other by a communication bus. The distribution terminal 300 may include an input / output IF 34 to which a display (display unit) configured separately from the distribution terminal 300 main body can be connected instead of or in addition to the touch screen 35.

Further, as shown in FIG. 1, the distribution terminal 300 is connected to peripheral devices such as one or more microphones 3010, one or more motion capture devices 3020, and one or more controllers 3030 via wireless or wired. , May be configured to be communicable. The wirelessly connected peripheral device establishes communication with the distribution terminal 300 according to a communication standard such as Bluetooth (registered trademark). Further, the device group including these peripheral devices may be regarded as one set of distribution terminals 300.

The microphone 3010 acquires the voice generated in the surroundings and converts it into an electric signal. The voice converted into an electric signal is transmitted to the distribution terminal 300 as voice data, and is received by the distribution terminal 300 via the communication IF 33.

The motion capture device 3020 tracks the motion (including the position and posture of the head, the facial expression, the movement of the mouth, etc.) of the tracking target (for example, a person), and transmits the output value as the tracking result to the distribution terminal 300. .. The motion data, which is an output value, is received by the distribution terminal 300 via the communication IF 33. The motion capture method of the motion capture device 3020 is not particularly limited. The motion capture device 3020 is a mechanism for capturing motion, such as a camera, various sensors, markers, a suit worn by a model (person), a signal transmitter, and an HMD (Head Mount Device), depending on the method adopted. Is selectively prepared.

The controller 3030 may have one or more physical input mechanisms such as buttons, levers, sticks, wheels and the like. The controller 3030 transmits an output value based on an input operation input to the input mechanism by the operator of the distribution terminal 300 to the distribution terminal 300. Further, the controller 3030 may have various sensors such as an acceleration sensor and an angular velocity sensor, and may transmit output values of the various sensors to the distribution terminal 300. The above output value is received by the distribution terminal 300 via the communication IF 33. In the following, a person who performs some input operation on the distribution terminal 300 by using an operation unit provided in the distribution terminal 300 or various input mechanisms communicably connected to the distribution terminal 300 is referred to as an operator. .. The operator includes a person who operates the distribution terminal 300 using the input unit 351 and the controller 3030, a voice actor who inputs voice through the microphone 3010, and moves via the motion capture device 3020. The model to be entered is also included.

The distribution terminal 300 may include a camera and a distance measuring sensor (not shown). Alternatively or in addition to the distribution terminal 300, the motion capture device 3020 and the controller 3030 may have a camera and a distance measuring sensor.

As described above, the distribution terminal 300 includes a communication IF 33, an input / output IF 34, and a touch screen 35 as an example of a mechanism for inputting information to the distribution terminal 300. The distribution terminal 300 may further include a camera and a distance measuring sensor, if necessary. Each of the above-mentioned parts as an input mechanism can be regarded as an operation part configured to accept a user's input operation.

The operation unit may be composed of the touch screen 35. In this case, the distribution terminal 300 identifies and accepts the user's operation performed on the input unit 351 of the touch screen 35 as the user's input operation. Alternatively, when the operation unit is composed of the communication IF 33, the distribution terminal 300 identifies and accepts a signal (for example, an output value) transmitted from the controller 3030 as an input operation of the user. Alternatively, when the operation unit is composed of the input / output IF34, a signal output from an input device (not shown) different from the controller 3030 connected to the input / output IF34 is specified as an input operation of the user and received.

<Hardware components of each device>
The processor 10 controls the operation of the entire viewing terminal 100. The processor 20 controls the operation of the entire distribution server 200. The processor 30 controls the operation of the entire distribution terminal 300. Processors 10, 20, and 30 include a CPU (Central Processing Unit), an MPU (Micro Processing Unit), and a GPU (Graphics Processing Unit).

The processor 10 reads a program from the storage 12 described later and expands it into the memory 11 described later. The processor 20 reads a program from the storage 22 described later and expands it into the memory 21 described later. The processor 30 reads a program from the storage 32 described later and expands it into the memory 31 described later. Processor 10, processor 20, and processor 30 execute the expanded program.

The memories 11, 21, and 31 are the main storage devices. The memories 11, 21, and 31 are composed of storage devices such as a ROM (Read Only Memory) and a RAM (Random Access Memory). The memory 11 provides a work area to the processor 10 by temporarily storing a program and various data read from the storage 12 described later by the processor 10. The memory 11 also temporarily stores various data generated while the processor 10 is operating according to the program. The memory 21 provides a work area to the processor 20 by temporarily storing various programs and data read from the storage 22 described later by the processor 20. The memory 21 also temporarily stores various data generated while the processor 20 is operating according to the program. The memory 31 provides a work area to the processor 30 by temporarily storing various programs and data read from the storage 32 described later by the processor 30. The memory 31 also temporarily stores various data generated while the processor 30 is operating according to the program.

In the present embodiment, the program executed by the processor 30 may be a distribution program for realizing live distribution of contents. In the present embodiment, the program executed by the processor 10 may be a viewing program for realizing the reproduction of the content.

In the present embodiment, the program executed by the processor 20 may be the above-mentioned distribution program or the above-mentioned viewing program. The processor 20 executes either or both of the distribution program and the viewing program in response to a request from the viewing terminal 100 or the distribution terminal 300 or the like. The distribution program and the viewing program may be executed in parallel.

That is, the distribution program may be a distribution program for realizing live distribution in collaboration with the distribution server 200 and the distribution terminal 300. Further, the viewing program may be a viewing program for realizing the reproduction of the content by the collaboration between the viewing terminal 100 and the distribution server 200.

Storages 12, 22, and 32 are auxiliary storage devices. The storages 12, 22, and 32 are composed of a storage device such as a flash memory or an HDD (Hard Disk Drive). Various data related to the game are stored in the storages 12, 22, and 32.

The communication IF 13 controls the transmission / reception of various data in the viewing terminal 100. The communication IF 23 controls the transmission and reception of various data on the distribution server 200. The communication IF 33 controls the transmission / reception of various data in the distribution terminal 300. The communication IFs 13, 23, and 33 control, for example, communication via a wireless LAN (Local Area Network), Internet communication via a wired LAN, a wireless LAN, or a mobile phone network, and communication using a short-range wireless communication or the like. To do.

The input / output IF 14 of the viewing terminal 100 is an interface for the viewing terminal 100 to accept data input, and an interface for the viewing terminal 100 to output data. The input / output IF 14 may input / output data via USB (Universal Serial Bus) or the like. The input / output IF 14 may include, for example, a physical button, a camera, a microphone, a speaker, or the like of the viewing terminal 100. The input / output IF24 of the distribution server 200 is an interface for the distribution server 200 to receive data input, and an interface for the distribution server 200 to output data. The input / output IF 24 may include, for example, an input unit that is an information input device such as a mouse or a keyboard, and a display unit that is a device that displays and outputs an image. The input / output IF 34 of the distribution terminal 300 is an interface for the distribution terminal 300 to receive data input, and an interface for the distribution terminal 300 to output data. The input / output IF34 includes, for example, information input devices such as a mouse, keyboard, stick, and lever, devices for displaying and outputting images such as a liquid crystal display, and peripheral devices (microphone 3010, motion capture device 3020, and controller 3030). It may include a connection for sending and receiving data between.

The touch screen 15 of the viewing terminal 100 is an electronic component that combines an input unit 151 and a display unit 152. The touch screen 35 of the distribution terminal 300 is an electronic component that combines an input unit 351 and a display unit 352. The input units 151 and 351 are, for example, touch-sensitive devices, and are configured by, for example, a touch pad. The display units 152 and 352 are composed of, for example, a liquid crystal display, an organic EL (Electro-Luminescence) display, or the like.

The input units 151 and 351 detect the position where the user's operation (mainly a physical contact operation such as a touch operation, a slide operation, a swipe operation, and a tap operation) is input to the input surface, and information indicating the position. Is provided as an input signal. The input units 151 and 351 may include touch sensing units (not shown). The touch sensing unit may adopt any method such as a capacitance method or a resistance film method.

Although not shown, the viewing terminal 100 may include one or more sensors for specifying the holding posture of the viewing terminal 100. This sensor may be, for example, an acceleration sensor, an angular velocity sensor, or the like. When the viewing terminal 100 includes a sensor, the processor 10 can also specify the holding posture of the viewing terminal 100 from the output of the sensor and perform processing according to the holding posture. For example, the processor 10 may be a vertical screen display in which a vertically long image is displayed on the display unit 152 when the viewing terminal 100 is held in the vertical direction. On the other hand, when the viewing terminal 100 is held horizontally, it may be a horizontal screen display in which a horizontally long image is displayed on the display unit. In this way, the processor 10 may be able to switch between the vertical screen display and the horizontal screen display according to the holding posture of the viewing terminal 100.

The camera 17 includes an image sensor and the like, and generates a captured image by converting the incident light incident from the lens into an electric signal.

The distance measuring sensor 18 is a sensor that measures the distance to the object to be measured. The ranging sensor 18 includes, for example, a light source that emits pulse-converted light and a light receiving element that receives the light. The distance measuring sensor 18 measures the distance to the object to be measured by the timing of light emission from the light source and the timing of receiving the reflected light generated by the light emitted from the light source being reflected by the object to be measured. The ranging sensor 18 may have a light source that emits light having directivity.

<Functional configuration of distribution system 1>
FIG. 3 is a block diagram showing the functional configurations of the viewing terminal 100, the distribution server 200, and the distribution terminal 300 included in the distribution system 1. Each of the viewing terminal 100, the distribution server 200, and the distribution terminal 300 has a functional configuration (not shown) required for functioning as a general computer, and functional functions required for realizing known functions in the game. May include configurations.

The viewing terminal 100 has a function as an input device that accepts a user's input operation and a function as an output device that outputs a game image or sound. The viewing terminal 100 functions as a control unit 110 and a storage unit 120 in cooperation with the processor 10, the memory 11, the storage 12, the communication IF 13, the input / output IF 14, and the like.

The distribution server 200 communicates with the distribution terminal 300 and the viewing terminal 100, and the distribution server 200 has a function of acting as an intermediary when distributing content data from the distribution terminal 300 to the viewing terminal 100. The distribution server 200 functions as a control unit 210 and a storage unit 220 in cooperation with the processor 20, the memory 21, the storage 22, the communication IF23, the input / output IF24, and the like.

The distribution terminal 300 has a function of creating content and delivering the content live to the viewing terminal 100 via the distribution server 200. The distribution terminal 300 functions as a control unit 310 and a storage unit 320 in cooperation with the processor 30, the memory 31, the storage 32, the communication IF 33, the input / output IF 34, and the like.

(Data stored in the storage unit of each device)
The storage unit 120 stores the viewing program 131 and the viewer information 132 of the viewer corresponding to the viewing terminal 100. The storage unit 320 stores the distribution program 331 and the distributor information 332 of the distributor corresponding to the distribution terminal 300. The storage unit 220 stores the viewing program 131, the viewer information 132, the distribution program 331, and the distributor information 332. The storage unit 220 collectively stores the distributor information 332 collected from each distribution terminal 300. The storage unit 220 collectively stores the viewer information 132 collected from each viewing terminal 100.

The viewing program 131 is a program executed by the viewing terminal 100 and the distribution server 200. On the other hand, the distribution program 231 is a program executed by the distribution terminal 300 and the distribution server 200.

It should be noted that the execution of the viewing program 131 and the distribution program 331 by the distribution server 200 is not essential. For example, when the viewing terminal 100 can realize the reproduction of the content by executing the viewing program 131 independently, the distribution server 200 does not have to execute the viewing program 131. Further, for example, when the distribution terminal 300 can realize live distribution by executing the distribution program 331 independently, the distribution server 200 does not have to execute the distribution program 331.

The viewer information 132 is data that the control unit 110 refers to when executing the viewing program 131. The viewer information 132 is data managed for each viewer. For example, the viewer information 132 may include the name of the viewer, that is, the name of the viewer account supported by the viewing terminal 100. Further, the viewer information 132 may include a content browsing history, a message transmission / reception history, and the like in the viewer account corresponding to the viewing terminal 100.

The distributor information 332 is data that the control unit 210 refers to when executing the distribution program 231. The distributor information 332 is data managed for each distributor. For example, the distributor information 332 may include information indicating the name of the account of the distributor corresponding to the distribution terminal 300. In addition, the distributor information 332 includes various information related to the distributed content such as the distribution history of the content, the number of playbacks of the distributed content, and the history of sending and receiving messages in the account of the distributor corresponding to the distribution terminal 300. You may be.

(Functional configuration of distribution server 200)
The control unit 210 comprehensively controls the distribution server 200 by executing the distribution program 231 stored in the storage unit 220. For example, the control unit 210 receives the content from the distribution terminal 300 in real time and delivers the content live to the viewing terminal 100. The control unit 210 communicates with the viewing terminal 100 and the distribution terminal 300 to transmit and receive various information as needed.

The control unit 210 functions as a transmission / reception unit 211, a server processing unit 212, and a data management unit 213 according to the description of the distribution program 231. The control unit 210 can also function as another functional block (not shown) in order to support live distribution of the content from the distribution terminal 300 and reproduction of the content by the viewing terminal 100.

The transmission / reception unit 211 receives various data and requests from the distribution terminal 300. For example, the transmission / reception unit 211 receives a part or all of the distributor information 332 from the distribution terminal 300. The transmission / reception unit 211 transmits various data and requests to the distribution terminal 300.

In addition, the transmission / reception unit 211 receives various data and requests from the viewing terminal 100. For example, the transmission / reception unit 211 receives a request for live distribution from the viewing terminal 100. The transmission / reception unit 211 transmits various information to the viewing terminal 100. For example, the transmission / reception unit 211 delivers the content live to the viewing terminal 100.

The server processing unit 212 performs arithmetic processing necessary for live distribution and playback of the content. The server processing unit 212 instructs the transmission / reception unit 211 to transmit various data. The server processing unit 212 instructs the data management unit 213 to add, update, or delete various types of information.

The data management unit 213 manages the data stored in the storage unit 220. The data management unit 213 adds, updates, or deletes various information included in the distributor information 332 and the viewer information 132 in response to an instruction from the server processing unit 212. The data management unit 213 may read at least one of the distributor information 332 and the viewer information 132 from the storage unit 220 according to the instruction of the server processing unit 212.

(Functional configuration of viewing terminal 100)
The control unit 110 comprehensively controls the viewing terminal 100 by executing the viewing program 131 stored in the storage unit 120. For example, the control unit 110 executes the reproduction of the content according to the operation of the viewing program 131 and the user. Further, the control unit 110 communicates with the distribution server 200 to send and receive information as needed during the reproduction of the content.

The control unit 110 functions as an operation reception unit 111, a reproduction unit 112, a display control unit 113, and a communication unit 114 according to the description of the viewing program 131. The control unit 110 can also function as other functional blocks (not shown) in order to realize the reproduction of the content.

The operation reception unit 111 detects and accepts a user's input operation with respect to the input unit 151. The operation reception unit 111 determines what kind of input operation has been performed from the action exerted by the user on the console via the touch screen 15 and other input / output IF 14, and outputs the result to each element of the control unit 110. To do.

For example, the operation receiving unit 111 receives an input operation for the input unit 151, detects the coordinates of the input position of the input operation, and specifies the type of the input operation. The operation reception unit 111 specifies, for example, a touch operation, a slide operation, a swipe operation, a tap operation, and the like as the types of input operations. Further, the operation reception unit 111 detects that the contact input is released from the touch screen 15 when the continuously detected input is interrupted.

The reproduction unit 112 executes a process related to the reproduction of the content. The playback unit 112 receives content data from the distribution server 200 in real time via the communication unit 114. The reproduction unit 112 includes a space definition unit 112A, a background control unit 112B, an object control unit 112C, a camera control unit 112D, and a drawing unit 112E.

The space definition unit 112A defines a virtual space based on the content data. Specific examples and definition methods of virtual space will be described in detail later.

The background control unit 112B arranges the background image specified by the content data in the virtual space. The type of the background image is not particularly limited, but for example, it may be a landscape image obtained by photographing 360 degrees around a specific existing point.

The object control unit 112C arranges the object specified by the content data in the virtual space at the position specified by the data and the posture specified by the data.

The camera control unit 112D arranges a virtual camera in the virtual space. The virtual camera defines the position and angle of the user's virtual viewpoint in the virtual space.

The drawing unit 112E draws at least an object and a background image included in the visual field area of the virtual camera in the virtual space as a moving image or a still image. Hereinafter, a moving image or a still image in which an object included in the visual field area of the virtual camera and a background image are drawn is simply referred to as an “image in the visual field area”. The drawing unit 112E may draw not only the visual field area but also the background image and objects of the entire virtual space.

The display control unit 113 causes the display unit 152 to display an image of the visual field region drawn by the drawing unit 112E of the reproduction unit 112 as content. The display control unit 113 may control and draw a 2D or 3D game object (UI object) for constructing a UI (user interface). For example, the display control unit 113 may superimpose UI objects necessary for various operations for viewing the content on the content or display the UI objects together with the content on the screen of the display unit 152. The UI object is, for example, an icon, a button, a list, a menu screen, or the like. The UI object is a tool for notifying the viewer of various information from the viewing terminal 100 during playback of the content on the viewing terminal 100.

The communication unit 114 transmits various data and requests to the distribution server 200. For example, the communication unit 114 transmits a distribution request for content data and a part or all of the viewer information 132 to the distribution server 200. The communication unit 114 receives various information related to live distribution from the distribution server 200. For example, the communication unit 114 receives the content data and a part or all of the distributor information 332 from the distribution server 200.

(Functional configuration of distribution terminal 300)
The control unit 310 comprehensively controls the distribution terminal 300 by executing the distribution program 331 stored in the storage unit 320. For example, the control unit 310 creates content data according to the operation of the distribution program 331 and the distributor, and supplies the content data to the viewing terminal 100 via the distribution server 200. The control unit 310 communicates with the distribution server 200 to send and receive information.

The control unit 310 functions as an operation reception unit 311, a distribution unit 312, a display control unit 313, and a communication unit 314 according to the description of the distribution program 331. The control unit 310 can also function as other functional blocks (not shown) depending on the nature of the content data to be created, that is, the nature of the content.

The operation reception unit 311 detects and accepts an operator's input operation with respect to the input unit 351. The operation reception unit 311 determines what kind of input operation has been performed on the console via the touch screen 35 and other input / output IF 34s from the action exerted by the operator, and outputs the result to each element of the control unit 310. Output. The details of the function of the operation reception unit 311 are almost the same as those of the operation reception unit 111 in the viewing terminal 100.

The distribution unit 312 executes various processes related to content distribution. For example, the distribution unit 312 creates content data. The method of creating the content data will be described in detail later.

The distribution unit 312 may create a screen for the distributor to support the operating of the content creation by the distributor. Hereinafter, the screen for the distributor is also referred to as a distribution screen. The configuration of the distribution screen is not particularly limited. For example, the distribution unit 312 may create a distribution screen capable of listing various components that can be included in the content, such as various objects such as avatars, background images, and various parameters. The distribution screen is displayed on the touch screen 35 via the display control unit 313.

In addition, the distribution unit 312 may create and output various notifications for the distributor. For example, the distribution unit 312 may notify the distributor of the viewer comment received via the distribution server 200.

The display control unit 313 outputs a display screen reflecting the processing results executed by each of the above-mentioned elements to the display unit 352 of the touch screen 35. The details of the function of the display control unit 313 are substantially the same as those of the display control unit 113 in the viewing terminal 100. For example, the display control unit 313 may display UI objects required for various operations for live distribution on the screen of the display unit 352.

The communication unit 314 transmits various data and requests to the distribution server 200. For example, the communication unit 314 transmits the content and a part or all of the distributor information 332 to the distribution server 200. The communication unit 314 receives various information related to live distribution from the distribution server 200. For example, the communication unit 314 receives a part or all of the viewer information 132 from the distribution server 200.

The functions of the viewing terminal 100, the distribution server 200, and the distribution terminal 300 shown in FIG. 3 are merely examples. For example, the distribution server 200 may include at least a part of the functions provided by the viewing terminal 100 or the distribution terminal 300. Further, for example, the viewing terminal 100 or the distribution terminal 300 may include at least a part of the functions provided by the distribution server 200.

Further, the viewing terminal 100 may cooperate with other components other than the above-mentioned components, or other devices connected to the viewing terminal 100. For example, the viewing terminal 100 may cause another component or another device to execute a part of the processing in the viewing terminal 100. Further, the distribution terminal 300 may cooperate with other components other than the above-mentioned components, or other devices connected to the distribution terminal 300. For example, the distribution terminal 300 may have other components or other devices execute a part of the processing in the distribution terminal 300.

The computer that executes the distribution program in the present embodiment may be any of the distribution server 200, the distribution terminal 300, and other devices, or may be realized by a combination of a plurality of these devices. Further, the computer that executes the viewing program in the present embodiment may be any of the distribution server 200, the viewing terminal 100, and other devices, or may be realized by a combination of a plurality of these devices. Further, the distribution system 1 uses a device other than the viewing terminal 100, the distribution server 200, and the distribution terminal 300 as a component of the distribution system 1, and causes the other device to execute a part of the processing in the distribution system 1. May be good.

<Flow of delivery process>
FIG. 4 is a sequence diagram showing an example of a processing flow related to live distribution and reproduction of content.

In step S100, the distribution unit 312 creates content data. The method of creating the content data will be described later.

In step S102, the distribution unit 312 transmits the content data to the distribution server 200 in real time via the communication unit 314. After that, the distribution unit 312 continues the live distribution of the content until, for example, the operation reception unit 311 receives an input operation instructing the end of the live distribution.

In step S104, the transmission / reception unit 211 of the distribution server 200 receives the content data in real time. The transmission / reception unit 211 may temporarily or non-temporarily store the received content data in the storage unit 220. The distribution server 200 waits until it receives a request for live distribution from the viewing terminal 100.

In step S106, the operation reception unit 111 of the viewing terminal 100 receives an input operation instructing the viewer to view the content. When the operation reception unit 111 receives the instruction, the reproduction unit 112 executes step S108.

In step S108, the reproduction unit 112 requests the distribution of the content via the communication unit 114.

In step S110, the server processing unit 212 of the distribution server 200 receives the distribution request via the transmission / reception unit 211. Upon receiving the request, the server processing unit 212 executes the process of step S112.

In step S112, the server processing unit 212 distributes the content data received from the distribution terminal 300 to the viewing terminal 100 in real time. After that, the server processing unit 212 continues the distribution of the content data until, for example, the transmission / reception unit 211 stops receiving the content data from the distribution terminal 300, or the viewing terminal 100 notifies that the viewing is completed.

In step S114, the communication unit 114 of the viewing terminal 100 receives the content data in real time.

In step S116, the reproduction unit 112 executes the reproduction process based on the content data received by the communication unit 114 at any time. The reproduction process means a series of processes related to the reproduction of the content indicated by the content data. After that, the reproduction unit 112 continues the content display process until it is determined to be YES in step S118 or YES in step S120.

In step S118, the control unit 110 determines whether or not the operation reception unit 111 has accepted the input operation instructing the end of viewing. If YES in step S118, that is, when the operation reception unit 111 receives the input operation instructing the end of viewing, the reproduction unit 112 executes step S122.

In step S122, the reproduction unit 112 ends the reproduction of the content. That is, the reproduction unit 112 ends the reproduction process. Further, the playback unit 112 notifies the distribution server 200 of the end of viewing via the communication unit 114. When the server processing unit 212 of the distribution server 200 receives the notification, the server processing unit 212 stops the live distribution to the viewing terminal 100.

On the other hand, if NO in step S118, that is, if the operation receiving unit 111 does not accept the input operation instructing the end of viewing, the control unit 110 further executes the process of step S120.

In step S120, the control unit 110 determines whether or not the live distribution has ended. For example, the control unit 110 may determine that the live distribution has ended when the reception of the content by the communication unit 114 ends. If NO in step S120, the control unit 110 continues the content display process in step S116 and repeats the determination process from step S118. On the other hand, if YES in step S120, the reproduction unit 112 executes the process of step S122. That is, the reproduction unit 112 ends the reproduction of the content.

(Creation of content)
FIG. 5 is a diagram showing a state in which the distributor creates content data. The distributor 701 creates content data by inputting various inputs to the distribution terminal 300.

The content data includes at least definition parameters for constructing a virtual space, information for specifying a background image, information for specifying a character object indicating a distributor's avatar, and setting parameters related to a virtual camera. I'm out. Hereinafter, the "character object indicating an avatar" is simply referred to as an "avatar".

For example, the distribution unit 312 includes information indicating the size of the virtual space and the coordinate axis system in the content data. Further, for example, the distribution unit 312 determines a character object indicating the avatar of the distributor, and determines the facial expression and motion of the avatar in real time. Then, the distribution unit 312 includes the information related to these determined avatars in the content data.

Further, for example, the distribution unit 312 acquires a background image and includes the background image in the content data. Note that the distribution unit 312 may include in the content data not the background image itself but information for making the background image identifiable or acquireable later on the viewing terminal 100.

The motion data of the model 702, which is a part of the distributor, may be input to the distribution terminal 300. For example, as a kind of motion capture device 3020, the model 702 may be equipped with an HMD, and the position and posture of the head of the model 702 may be detected by the HMD. Then, the information indicating the position and posture of the head transmitted from the HMD may be input to the distribution terminal 300 as motion data. Then, the distribution unit 312 may include the motion data of the model 702 in the content data as data for designating the motion of the avatar. As a result, when the content is played back on the viewing terminal 100 as described later, the movement indicated by the motion data can be reflected in the movement of the avatar. For example, the avatar's head can be moved according to the movement of the model 702's head. The motion data is obtained by detecting the movement of the body of the model 702 via the motion capture device 3020.

Further, for example, the distributor 701 may input his / her voice 703 into the distribution terminal 300 via the microphone 3010. Then, the distribution unit 312 of the distribution terminal 300 may include the input voice data as the voice of the avatar in the content data. Alternatively, the distribution unit 312 may include the voice having the same content as the input voice data in the content data as the voice of the avatar. The “voice of the same content” means the voice 703 itself, a voice obtained by subjecting the voice 703 to a predetermined voice processing, or a voice obtained by mechanically uttering the content of the voice 703 with a reading software or the like.

The distribution unit 312 collects these data input in real time as content data at any time and transmits the data to the distribution server 200 via the communication unit 314. As a result, information including content data is live-distributed from the distribution terminal 300 to the viewing terminal 100 through the distribution server 200 as an intermediary. Hereinafter, the information including the content data will be referred to as the first information.

(Flow of content display processing)
FIG. 6 is a diagram showing an example of the flow of the reproduction process shown in step S116 of FIG.

In step S200, the reproduction unit 112 of the viewing terminal 100 acquires the first information via the communication unit 114.

In step S201, the space definition unit 112A defines a virtual space. The space definition unit 112A defines a spherical virtual space based on, for example, definition parameters included in the content data.

In step S202, the background control unit 112B develops the background image in the defined virtual space. Specifically, the background control unit 112B attaches one background image designated by the content data to the virtual space so as to cover the entire spherical surface of the space. By the processing of S200 to S202, a virtual space in which the background image is developed is constructed.

In step S204, the object control unit 112C arranges the first object in the virtual space. The first object is an object indicating the avatar of the distributor. The object control unit 112C determines the type, model data, arrangement position, posture, and the like of the first object based on the content data, and arranges the first object in the virtual space.

In step S206, the object control unit 112C arranges the second object in the virtual space. The second object is an object provided so as to shield a specific area of the background image pasted in the virtual space, as will be described later. The second object may be a solid or a plane.

The size and shape of the second object is not limited. For example, the second object may be an object on which the first object, which is an avatar, stands, or may be ceiling lighting or the like in a virtual space. In other words, the first object may be placed on the second object, that is, in contact with the second object. However, it is desirable that the second object has a size and shape that does not impair the original interest of the content in the live-streamed content.

The object control unit 112C determines the type, model data, arrangement position, posture, and the like of the second object based on the content data, and arranges the second object in the virtual space.

In step S207, the object control unit 112C operates the first object according to the motion data included in the first information.

In step S208, the camera control unit 112D arranges the virtual camera in the virtual space. At this time, the camera control unit 112D arranges the virtual camera at a position different from the center point of the virtual space. An example of the placement position of the virtual camera will be shown later.

In step S210, the drawing unit 112E calculates the field of view area of the virtual camera from the definition parameters of the virtual space and the position and angle of the virtual camera.

In step S212, the drawing unit 112E draws an object, an image, or the like projected in the vicinity of the calculated visual field area in the virtual space. In other words, the drawing unit 112E renders the image of the calculated visual field area in the virtual space.

In step S214, the display control unit 113 causes the display unit 152 and the like to display an image of the visual field region obtained by rendering the drawing unit 112E.

As described above, the content data is sequentially distributed from the distribution server 200. Therefore, the space definition unit 112A, the background control unit 112B, and the object control unit 112C have the definition of the virtual space itself, the background image, the position of the object, the expression, the movement, and the virtual camera in the content data acquired by the communication unit 114. Whenever there is a change in the position, angle, etc. of, the change is reflected in the virtual space. As a result, the content based on the content data is reproduced and displayed in real time on the viewing terminal 100.

Of the processes shown in FIG. 6, the process of step S206 is not essential. That is, the object control unit 112C does not have to arrange the second object in the virtual space. Further, the object control unit 112C may arrange objects other than the first and second objects in the virtual space. Further, the background control unit 112B may paste an image other than the background image into the virtual space.

According to the above process, since the field of view area of the virtual camera arranged at a position different from the center point is drawn, the center of the virtual space can be shown to the user. Therefore, the interest of the content in the live distribution is improved.

(Example of virtual space)
The state (A) of FIG. 7 is a diagram showing an example of a virtual space constructed in the distribution system. The virtual space 600 has a spherical structure centered on the center point 601 and covering the entire 360-degree direction connecting the north pole point 602 and the south pole point 603. In the state (A) of FIG. 7, in order not to complicate the explanation, the hemisphere in the upper half of the virtual space 600 from the horizon 604 is illustrated. In the actual virtual space, background images and objects can be arranged in the lower hemisphere as well. Further, the parallels and meridians shown in the virtual space 600 are for illustration purposes, and the parallels and meridians may be omitted when defining and constructing the actual virtual space.

In the virtual space 600, an XYZ coordinate system with the center point 601 as the origin is defined. The XYZ coordinate system is, for example, parallel to the real coordinate system. The horizontal direction, vertical direction (vertical direction), and front-back direction in the XYZ coordinate system are defined as the X-axis, the Y-axis, and the Z-axis, respectively. Therefore, the X-axis (horizontal direction) of the XYZ coordinate system is parallel to the x-axis of the real coordinate system, and the Y-axis (vertical direction) of the XYZ coordinate system is parallel to the y-axis of the real coordinate system. The Z-axis (front-back direction) is parallel to the z-axis of the real coordinate system.

When the space definition unit 112A defines the celestial spherical virtual space 600 as shown in the state (A) of FIG. 7, the background control unit 112B is a background image which is a 360-degree omnidirectional image inside the virtual space 600. Paste 630.

The object control unit 112C arranges the first object 610 of the avatar in the virtual space 600. Hereinafter, the first object is also referred to as an avatar. Further, the object control unit 112C may arrange the second object 620 in the virtual space 600. In the illustrated example, the second object 620 is the base object on which the avatar 610 stands.

The camera control unit 112D arranges the virtual camera 640 in the virtual space 600. Here, the camera control unit 112D arranges the virtual camera 640 at an arbitrary angle at an arbitrary position, not at or near the center point of the virtual space 600. The position and angle of the virtual camera 640 are defined by the content data.

The state (B) of FIG. 7 is a diagram showing an example of an image of the field of view region of the virtual camera shown in the state (A). In the example of the state (A) of FIG. 7, the camera control unit 112D arranges the virtual camera 640 at the same height as the face of the avatar 610. It can be said that the field of view area of the virtual camera 640 is an area of the virtual space 600 that can be visually recognized by the viewer via the viewing terminal 100. That is, it can be said that the position and angle of the virtual camera 640 are the same as the viewpoint of the viewer in the virtual space 600. In the case of the virtual camera 640 in the state (A) of FIG. 7, as shown in the state (B) of FIG. 7, the image 700 of the visual field area from the viewpoint of viewing the avatar 610 from the side is drawn and displayed on the display unit 152. ..

In this way, the reproduction unit 112 constructs the virtual space 600 in cooperation with the space definition unit 112A, the background control unit 112B, and the object control unit 112C. Then, the virtual camera 640 is arranged by the camera control unit 112D, and the image 700 in the visual field region from the viewpoint of the virtual camera 640 is drawn by the drawing unit 112E. Then, the display control unit 113 displays the drawn image 700 of the visual field region on the display unit.

Here, the second object 620 is the base object on which the avatar 610 stands, and the area (specific area) behind the second object 620 when viewed from the virtual camera 640 is an area shielded from the virtual camera 640. In other words, if the second object 620 is arranged according to the position of the area (specific area) that the viewer wants to hide, such as the reflection part of the camera that captured the background image, that area is shielded by the second object 620. It can be made invisible to the viewer.

(Another example of virtual space)
The object control unit 112C may arrange the second object so as to overlap the north pole point, the south pole point, or the horizon of the virtual space in the field of view region from the virtual camera.

The state (A) of FIG. 8 is a diagram showing another example of the virtual space constructed in the distribution system. The state (B) of FIG. 8 is a diagram showing an example of an image of the field of view region of the virtual camera shown in the state (A). In the example of the state (A) of FIG. 8, the object control unit 112C hides the background image 630 in the vicinity of the north pole point when the north pole point and its peripheral area (specific area) enter the visual field area of the virtual camera 640. The second object 621 is arranged at a predetermined position.

The background image 630 is an image that covers the entire spherical surface of the virtual space 600, that is, an omnidirectional image. When the omnidirectional image is attached to the inner surface of the sphere of the virtual space 600, the background image may be distorted at at least one of the sphere at the north pole point 602, the south pole point 603, and the horizon 604. By arranging the second object 621 in the vicinity of the region (specific region) where such distortion of the background image can occur, the distortion of the background image can be made inconspicuous.

For example, even in the case of a camera angle that looks up at the avatar 610, such as the position and angle of the virtual camera 640 in the figure, as shown in the state (B) of FIG. Since the peripheral area (specific area) is shielded by the second object 621, it is possible to prevent the viewer from seeing the distortion of the background image.

(Another example of virtual space)
The object control unit 112C may arrange the second object so as to shield it from a specific area in the background image in the field of view area from the virtual camera.

The state (A) of FIG. 9 is a diagram showing another example of the virtual space constructed in the distribution system. The state (B) of FIG. 9 is a diagram showing an example of an image of the field of view region of the virtual camera shown in the state (A). In the example of the state (A) of FIG. 9, the object control unit 112C arranges the second object 622 vertically from the horizon 604 as shown in the figure.

Since the background image 630 is an omnidirectional image, some areas may be distorted at the time of creation depending on the shape and size of the original image that is the source of the omnidirectional image. When the virtual camera 640 projects an area (specific area) where such image distortion can occur, the second object 622 is arranged so as to shield the area. Therefore, it is possible to hide the distortion of the background image from the viewer.

For example, as shown in the state (B) of FIG. 9, since the specific area can be in a state of being shielded by the second object 622, the distortion of the background image 630 cannot be seen in the image 900 of the visual field area. Can be done.

(Flow until watching live streaming)
The operation of selecting live distribution and starting viewing by the viewing terminal 100 may be performed by a GUI using a map.

FIG. 10 is a diagram showing an example of a display screen of the viewing terminal 100. As shown in the figure, the display screen 1700 displays a map based on the map data and an area 1701 (gray circle in the figure) where the live distribution can be viewed. Here, the map data may be stored in the storage unit 120 of the viewing terminal 100, or may be appropriately acquired from an external server or the like. Further, on the display screen 1700, a message 1702 indicating the date and time, contents, etc. of the live distribution and an icon 1703 instructing the start of the live distribution may be displayed in association with the area 1701. Further, the display screen 1700 may display the current position of the viewing terminal 100, that is, the icon 1704 indicating the current position of the user. Further, the display screen 1700 may display an object 1705 such as a building or a landscape corresponding to the map data.

Information on the current position of the viewing terminal 100 can be obtained by equipping the viewing terminal 100 with a GPS (Global Positioning System) receiver (not shown) and receiving GPS radio waves. The GPS receiver transmits the position usage of the current position indicated by the obtained radio wave to the control unit 110, and the control unit 110 acquires this.

In addition, the control unit 110 periodically sends information indicating the area where the live distribution can be viewed, information related to the live distribution such as the time zone in which the live distribution is distributed, and information indicating the state of the live distribution from the distribution server 200. Is received and reflected in the information displayed on the display screen 1700 and the display screen 1800.

FIG. 11 is a diagram showing an example of a user's first input operation on the display screen shown in FIG. The first input operation is an input operation for designating a certain point on the map on the display screen displaying the map. In the illustrated example, on the display screen 1800, when the user (icon 1703) is in the area 1701 where the live distribution can be viewed and the input operation for selecting the icon 1703 instructing the start of the live distribution is performed, the input operation is performed. The input operation is accepted as the first input operation.

When the operation receiving unit 111 receives the first input operation, the control unit 110 identifies the live distribution and receives the content data from the distribution server 200. Upon receiving the content data, the reproduction unit 112 executes the processing shown in FIG. 6, that is, the construction of the virtual space, the arrangement of the virtual camera, and the drawing of the field image. This allows the user to watch the live stream. In other words, when the distance between the position on the map of the viewing terminal 100 indicated by the position information of the current position and the point specified by the first input operation is within a predetermined range, the space definition unit of the playback unit 112 The 112A and the background control unit 112B collaborate to build a virtual space for live-streamed content.

The time zone for live distribution may be fixed. Further, as shown in FIGS. 10 and 11, the content of the message 1702 may be changed depending on the state of live distribution, for example, whether or not live distribution is in progress.

FIG. 12 shows a display screen of the viewing terminal 100 when viewing live-distributed content. When the user performs an input operation for selecting the icon 1703, viewing of the live distribution is started, and the display screen 1900 is displayed on the viewing terminal 100. As shown, the display screen 1900 includes at least the distributor's avatar 1901. On the display screen 1900, as a background image 1902, an image of a landscape in real space at 360 degrees around a point on the map on which the distributor's avatar 1901 is arranged is displayed. Then, the avatar 1901, the second object 1903, and the like may be displayed at a position that shields a specific area of the background image.

Here, as shown in the figure, the second object 1903 is the base object on which the avatar 1901 stands, and the area (specific area) below the second object 1903 is shielded from the virtual camera. In other words, if the second object 1903 is arranged according to the position of the area (specific area) that the viewer wants to hide, such as the reflection part of the camera that captured the background image, that area is shielded by the second object 1903. It can be made invisible to the viewer.

[Embodiment 2]
The distribution terminal 300 may acquire an image of the play screen of the game being played by the distributor by using the game terminal included in the game system. Then, the reproduction unit 112 may construct a virtual space in which at least an omnidirectional image, a first object indicating the distributor's avatar, and a third object displaying an image of the game play screen are arranged. In the present embodiment, the position of the virtual camera may be a position in the virtual space different from the center point of the virtual space, and the position may include the center point in the visual field area. It may be in a position other than. Further, the content data may be "live content" that is delivered live, or may be "recorded content" that is recorded data. Hereinafter, "live content" and "recorded content" are collectively referred to simply as "content".

(System overview)
FIG. 13 is a diagram showing an outline of the distribution system 2 according to the present embodiment. FIG. 13 differs from the distribution system 1 shown in FIG. 1 in that it includes a game terminal 400.

The game terminal 400 is a terminal for the distributor to play the game. The game terminal 400 transmits an image of the play screen of the game to the distribution terminal 300 at any time. As a result, the distributor can create the content data of the content including the play screen when creating the content data on the distribution terminal 300.

The game terminal 400 may be a game system including the game terminal 400 together with another game terminal or a game server (not shown). That is, when a distributor plays a game using a game terminal 400 included in the game system, the play image of the game is transmitted from the game terminal 400 or the game server to the distribution terminal 300. Good.

(Virtual space)
The state (A) of FIG. 14 is a diagram showing another example of the virtual space constructed in the distribution system 2. A background image 1030 is attached to the virtual space 1000 shown in the figure, and an avatar 1010 and a virtual camera 1040 are arranged. Although not shown in the figure, the second object may be arranged in the virtual space 1000 also in the present embodiment. Further, in the example of the figure, the chair object 1060 on which the avatar sits is also arranged.

In the present embodiment, the object control unit 112C arranges the third object 1021 that displays the image of the play screen of the game being played by the distributor in the virtual space 1000. For example, as shown in the figure, the third object 1021 may be an object that imitates a television, a personal computer, or the like. An image 1050 of the game play screen is displayed in at least a part of the area of the third object 1021. The object control unit 112C may arrange a third object capable of displaying the play screen, or the third object and the play screen so that the play screen is apparently displayed on the third object. The image may be pasted or arranged so as to be superimposed when viewed from the virtual camera 1040.

The state (B) of FIG. 14 is a diagram showing an example of an image of the field of view region of the virtual camera shown in the state (A). In the example of the state (A) of FIG. 14, the camera control unit 112D arranges the virtual camera 1040 at the same height as the head of the avatar 1010 and behind the avatar 1010. In the case of the virtual camera 1040 in the state (A) of FIG. 14, as shown in the state (B) of FIG. 14, the avatar 1010 looking at the play screen displayed on the third object 1021 is viewed from directly behind. An image of the area is drawn and displayed on the display unit 152.

(Switching of virtual camera 1)
In response to the reception of a specific second input operation on the viewing terminal 100, the camera control unit 112D sets the position and angle of the virtual camera 1040 to a position and an angle substantially the same as the viewpoint of the distributor corresponding to the avatar. You may switch to the angle.

The specific method for switching the position and angle of the virtual camera is not particularly limited. For example, the camera control unit 112D may move the position and angle of the virtual camera arranged in the virtual space. Further, the camera control unit 112D may temporarily delete the virtual camera arranged in the virtual space and rearrange the virtual camera at the position and angle after switching.

Further, a plurality of virtual cameras may be arranged in the virtual space at different positions and angles, and the virtual cameras used by the camera control unit 112D may be sequentially switched. Then, for example, the drawing unit 112E may determine which virtual camera's field image to render and execute the rendering.

The state (A) of FIG. 15 is a diagram showing a virtual space after the second input operation is performed and the position and angle of the virtual camera are switched. When the second input operation is accepted, the camera control unit 112D switches the virtual camera to a position substantially similar to the viewpoint of the avatar 1010, that is, a position and angle substantially similar to the viewpoint of the distributor with respect to the play screen 1050. In the case of the state (A) of FIG. 15, the field of view region of the avatar 1010 is indicated by α. In the actual virtual space 1000, the avatar 1010 and the virtual camera 1040 are superimposed, but the virtual camera 1040 is not shown for the sake of easy viewing of the drawing.

The state (B) of FIG. 15 is a diagram showing an example of an image of the field of view region of the virtual camera shown in the state (A). In the case of the viewpoint of the avatar 1010 shown in the state (A) of FIG. 15, the region α in the figure is the image of the visual field region. Therefore, as shown in the state (B) of FIG. 15, the close-up screen of the third object 1021 and the play screen 1050 is drawn and displayed as an image of the visual field region.

Thereby, the virtual camera 1040 can be switched to a position and an angle for projecting a region substantially similar to the field of view of the distributor according to the second input operation of the user of the viewing terminal 100. Therefore, the user of the viewing terminal 100, that is, the viewer, can view the video that traces the viewpoint of the distributor according to his / her own input operation. Therefore, the degree of freedom of camera work of the content can be increased, and thus the interest of the content can be improved.

(Switching of virtual camera 2)
The camera control unit 112D may arrange the fourth object 1041 indicating the belongings of the avatar 1010 in the virtual space 1000. Then, in response to the reception of the specific third input operation by the viewing terminal 100, the camera control unit 112D sets the position and angle of the virtual camera 1040 to a position substantially the same as the position and orientation of the fourth object. And may be switched to angle.

The state (A) of FIG. 16 is a diagram showing a virtual space after the third input operation is performed and the position and angle of the virtual camera are switched. When the third input operation is accepted, the camera control unit 112D switches the virtual camera to a position and an angle according to the position and orientation of the fourth object 1041 which is the possession of the avatar 1010. In the illustrated example, the possession of the avatar 1010 is the camera object 1041 held by the avatar 1010. In the state (A) of FIG. 15, the fourth object 1041 faces the face of the avatar 1010. In the actual virtual space 1000, the camera object 1041 and the virtual camera 1040 are superimposed, but the virtual camera 1040 is not shown for the sake of easy viewing of the drawing.

The state (B) of FIG. 16 is a diagram showing an example of an image of the field of view region of the virtual camera shown in the state (A). As shown in the state (B) of FIG. 16, the image 1300 of the visual field region from the viewpoint of the fourth object 1041 reflects the direction in which the fourth object is facing, that is, the direction of the face of the avatar 1010.

Thereby, according to the third input operation of the user of the viewing terminal 100, the virtual camera can be switched to the position and angle at which the visual field image is projected from a viewpoint substantially similar to the viewpoint of the avatar's possession. Therefore, the user of the viewing terminal 100, that is, the viewer, can view the video from the avatar's possession viewpoint according to his / her own input operation. Therefore, the degree of freedom of camera work of the content can be increased, and thus the interest of the content can be improved.

(Addition of virtual camera)
The camera control unit 112D may arrange a plurality of virtual cameras. Further, the camera control unit 112D sets the second virtual camera (that is, the second virtual viewpoint) different from the virtual camera 1040 as the first object in response to the reception of the specific fourth input operation in the viewing terminal 100. It may be arranged at a position and an angle at which a specific part of the camera is reflected. Here, the specific part of the avatar is, for example, the face or hand of the avatar. Further, the drawing unit 112E may render an image including both an image in the field of view area of the virtual camera and an image in the field of view area of the second virtual camera.

The state (A) of FIG. 17 is a diagram showing a virtual space after the fourth input operation is performed and the position and angle of the virtual camera are switched. When the fourth input operation is accepted, the camera control unit 112D arranges the second virtual camera 1070 in the virtual space 1000 at a position and an angle at which a specific portion of the avatar 1010 is projected. In the illustrated example, the second virtual camera 1070 is arranged at a position and angle at which the face of the avatar 1010 is reflected.

The state (B) of FIG. 17 is a diagram showing an example of an image 1400 in the field of view of the virtual camera 1040 and the second virtual camera 1070 shown in the state (A). As shown in the state (B) of FIG. 16, the drawing unit 112E superimposes and displays the image 1080 of the visual field region of the second virtual camera 1070 on the image 1400 of the visual field region of the virtual camera 1040 as the image of the visual field region. .. As shown in the figure, the image of the field of view region of the second virtual camera 1070 may be displayed only in a part of the region instead of the whole.

As a result, a specific part of the avatar can be projected by the second virtual camera 1070 in response to the fourth input operation of the user of the viewing terminal 100. Therefore, the viewer can simultaneously view both the image of the visual field area by the virtual camera and the image of the specific part of the avatar by the second virtual camera according to his / her own input operation. Therefore, the degree of freedom of camera work of the content can be increased, and thus the interest of the content can be improved.

When a predetermined input operation is received by the viewing terminal 100, the drawing unit 112E replaces the image 1400 in the visual field area of the virtual camera 1010 with the image 1080 in the visual field area of the second virtual camera 1070 to draw. May be good.

(Switching of virtual camera 3)
When a specific fifth input operation is accepted by the viewing terminal 100, the camera control unit 112D may switch the position and angle of the virtual camera 1040 to the position and angle at which the specific portion of the first object is projected.

The state (A) of FIG. 18 is a diagram showing a virtual space after the fifth input operation is performed and the position and angle of the virtual camera 1040 are switched. When the fifth input operation is accepted, the camera control unit 112D switches the position and angle of the virtual camera 1040 to the position and angle at which the specific part of the avatar 1010 is projected. In the case of the state (A) of FIG. 18, the camera control unit 112D arranges the virtual camera 1040 at a position and an angle at which the face of the avatar 1010 can be seen from the front. The position and angle of the virtual camera 1040 are determined relative to the position of the specific portion of the avatar 1010. Therefore, when the position and angle of the specific portion of the avatar 1010 change, the camera control unit 112D moves or rearranges the virtual camera 1040 to a relative position according to the changed position and angle. That is, when the specific part of the avatar 1010 moves, the virtual camera 1040 also follows and moves so as to be in a predetermined relative position. In this way, even if the specific part of the avatar 1010 moves, the virtual camera 1040 also moves following the movement, so that the specific part of the avatar 1010 can always be photographed.

The state (B) of FIG. 18 is a diagram showing an example of the image 1031 of the visual field region of the virtual camera 1040 shown in the state (A). As shown in the state (B) of FIG. 18, the virtual camera 1040 is in a position where the face of the avatar 1010 can be photographed from the front.

As a result, the virtual camera 1040 can be switched to a position and an angle at which a specific part of the avatar is projected according to the fifth input operation of the user of the viewing terminal 100. Therefore, the degree of freedom of camera work of the content can be increased, and thus the interest of the content can be improved.

[Game variant]
In the above description, the game is independent of the content distributed by the distribution system 2, and the game screen is included in the content by incorporating the game screen from the game terminal 400 into the distribution terminal 300. ..

However, in the distribution system 2, the game terminal 400 and the distribution terminal 300 may be integrated. In this case, the distribution unit 312 may provide a game for the distributor to play in the virtual space via the avatar. Examples of such a game include a VR game realized by using an HMD. In this case, the distribution unit 312 may execute various processes related to the progress of the game to create a game screen and distribution screen. In this case, the game play screen included in the content data is the content itself including the distributor's avatar.

FIG. 19 is a diagram showing another example of an image in the field of view region of the virtual camera in the distribution system 2. The game screen 1600 shown in FIG. 19 displays content as a game screen including at least the distributor's avatar 1010. Further, in the illustrated example, the enemy character 1610 is arranged on the game screen 1600, and is playing against the avatar 1010 operated by the distributor (the operation of the distributor is reflected).

On the game screen 1600, the avatar 1010 is the player character. In the illustrated example, the avatar 1010 corresponding to the distributor may move on the game screen 1600 according to the movement of the distributor (model 702, etc.). That is, the movement of the distributor whose motion data is to be acquired may be reflected in the avatar 1010 as it is. For example, when the model 702 is equipped with the HMD as described above, the orientation of the head of the avatar 1010 may be changed according to the movement of the head of the model 702.

Further, a virtual camera that automatically tracks may be provided at a position facing the line-of-sight direction of the avatar 1010 corresponding to the distributor. According to this virtual camera, the line of sight of the avatar 1010 corresponding to the distributor can be reliably captured. Therefore, it is possible to prevent the viewer from being dissatisfied that the avatar 1010 corresponding to the distributor does not look at himself / herself. A mode for displaying the image from the virtual camera may be provided separately. For example, the virtual camera may be arranged in the virtual space, and when the viewer taps the virtual camera, the mode may be changed to the above mode.

[Modification example]
The distribution system 1 or 2 combines the arrangement method and arrangement position of the second object described in the first embodiment, the switching of the position and angle of the virtual camera described in the second embodiment, and the arrangement of the second virtual camera. You may do it.

For example, in the distribution system 1, the object control unit 112C of the viewing terminal 100 determines the position of the virtual camera as described in the second embodiment when the operation reception unit 111 receives the second input operation or the third input operation. And the angle may be switched. Further, in the distribution system 1, when the operation receiving unit 111 receives the fourth input operation, the object control unit 112C of the viewing terminal 100 arranges and draws the second virtual camera as described in the second embodiment. The unit 112E may draw an image in the field of view of both the virtual camera and the second virtual camera.

Further, in the examples of the first embodiment and the second embodiment, the viewing terminal 100 constructs a virtual space, arranges a virtual camera, and draws an image in the visual field area based on the content data created in the distribution terminal 300. .. However, the distribution unit 312 of the distribution terminal 300 according to the distribution systems 1 and 2 may execute the construction of the virtual space, the arrangement of the virtual cameras, and the drawing of the image in the visual field area. In other words, the distribution unit 312 of the distribution terminal 300 may execute the processes of steps S201 to S212 of FIG. 6 and distribute the drawn image of the visual field area as content to the viewing terminal 100 via the distribution server 200. ..

In this case, the switching of the virtual cameras described in the second embodiment can be realized by, for example, the following method. That is, the distribution unit 312 of the distribution terminal 300 draws the entire virtual space in advance, or draws an image of the field of view region of the virtual camera at all positions and angles that can be arranged. In this case, when the second input operation, the third input operation, the fourth input operation, or the fifth input operation is executed in the viewing terminal 100, the viewing terminal 100 informs the distribution server 200 that the operation has been performed. Be done. As described above, the distribution server 200 stores the drawn images corresponding to the images in all the visual field regions, and the server processing unit 212 stores the images in the visual field region according to the content of the operation transmitted from the viewing terminal 100. , Transmit to the viewing terminal 100. As a result, the viewpoint of the virtual camera can be switched according to the input operation on the viewing terminal 100.

[Example of realization by software]
The control block of each control block of the control units 110, 210, and 310 may be realized by a logic circuit (hardware) formed in an integrated circuit (IC chip) or the like, or may be realized by software.

In the latter case, the information processing apparatus including at least one control unit 110, 210, and 310 includes a computer that executes a program instruction, which is software that realizes each function. The computer includes, for example, one or more processors and a computer-readable recording medium that stores the program. Then, in the computer, the processor reads the program from the recording medium and executes it, thereby achieving the object of the present invention. As the processor, for example, a CPU (Central Processing Unit) can be used. As the recording medium, a "non-temporary tangible medium", for example, a ROM (Read Only Memory) or the like, a tape, a disk, a card, a semiconductor memory, a programmable logic circuit, or the like can be used. Further, a RAM (Random Access Memory) for expanding the above program may be further provided. Further, the program may be supplied to the computer via an arbitrary transmission medium (communication network, broadcast wave, etc.) capable of transmitting the program. It should be noted that one aspect of the present invention can also be realized in the form of a data signal embedded in a carrier wave, in which the above program is embodied by electronic transmission.

The present invention is not limited to the above-described embodiments, and various modifications can be made within the scope of the claims, and the embodiments obtained by appropriately combining the technical means disclosed in the different embodiments. Is also included in the technical scope of the present invention.

[Additional notes]
The contents relating to one aspect of the present invention are listed below.

(Item 1) The program (viewing program 131) has been described. According to certain aspects of the disclosure, the program serves as a viewing terminal (100), comprising a processor (10) and a memory (11), for the distributor to view content delivered in real time using the distribution terminal (300). , A program executed by a computer, the program is information related to the content to the processor, and is information including motion data at least detecting the movement of the distributor's body, which is distributed from the distribution terminal at any time. A step of receiving the first information (S200), a step of constructing a virtual space in which a background image is developed (S201 to S202), and a step of arranging a first object which is an avatar corresponding to a distributor in the virtual space. (S204), the step (S207) of operating the first object according to the motion data in response to receiving the first information during the real-time distribution, and the center of the virtual space of the virtual space. An image obtained by rendering a step (S208) of arranging a virtual viewpoint at a position different from a point and including a center point in the visual field area, a step of rendering an image of the visual viewpoint area of the virtual viewpoint (S212), and an image obtained by rendering. Is executed on the display unit (S214).

According to the above processing, since the visual field area of the virtual viewpoint arranged at a position different from the center point is drawn, the center of the virtual space can be shown to the user. Therefore, the interest of the content in the live distribution is improved.

(Item 2) In (Item 1), the program may cause the processor to further perform a step (S206) of arranging a second object in the virtual space so as to shield a specific area of the background image. ..

When creating a background image that covers a virtual space, a part of the background image may be distorted depending on the shape and size of the original original image. According to the above processing, for example, when displaying an image of the visual field region of the virtual viewpoint in a region where such distortion of the background image can occur, the second object is arranged so as to shield the region. Therefore, it is possible to prevent the viewer from seeing the distortion of the background image.

(Item 3) In (Item 2), in the step of arranging the first object, the first object may be arranged on the second object.

According to the above process, when the viewer pays attention to the first object, the second object shields the area where the image can be distorted. Therefore, it is possible to prevent the viewer from seeing the distortion of the background image.

(Item 4) In any one of (Item 1) to (Item 3), the program further specifies to the processor a step of displaying a map based on map data and a certain point on the map. 1 The step of accepting the input operation and the step of executing and constructing the virtual space are constructed based on the first input operation, and the background image is an image taken around the position in the real space corresponding to the point. There may be.

According to the above processing, it is possible to live-stream an image against the background of an actual landscape at a specific point. Therefore, the interest of the content in the live distribution is improved.

(Item 5) In (Item 4), the program causes the processor to further execute a step of acquiring the position information indicating the current position of the viewing terminal, and the step of constructing is on the map of the viewing terminal indicated by the position information. A virtual space may be constructed when the distance between the position in and the point specified by the first input operation is within a predetermined range.

According to the above process, the live can be seen by actually performing the first operation after actually going to the vicinity of the designated point. Therefore, it is possible to improve the interest of live distribution.

(Item 6) In any one of (Item 1) to (Item 5), the program sets the position of the virtual viewpoint and the position of the virtual viewpoint in response to the reception of the second input operation by the processor and the viewing terminal. The step of switching the angle to the position and angle substantially the same as the viewpoint of the distributor corresponding to the first object is executed, and the step of rendering is the step of switching to the position and angle substantially the same as the viewpoint of the distributor of the virtual viewpoint. The image in the field of view may be rendered after the position and angle have been switched.

According to the above process, the virtual viewpoint can be switched to a position and an angle for projecting a region substantially similar to the field of view of the distributor according to the second input operation of the user of the viewing terminal. Therefore, the user of the viewing terminal, that is, the viewer, can view the video that traces the viewpoint of the distributor according to his / her own input operation. Therefore, according to the above processing, the degree of freedom of the camera work of the live distribution can be increased, and thus the interest of the live distribution can be improved.

(Item 7) In any one of (Item 1) to (Item 6), the program arranges a fourth object indicating the belongings of the avatar in the processor and further in the virtual space, and in the viewing terminal, the fourth object. 3 In response to the reception of the input operation, the step of switching the position and angle of the virtual viewpoint to the position and angle substantially the same as the position and orientation of the fourth object, and the step of executing and rendering are the first steps. 4 The image of the visual field area may be rendered after the position and angle of the virtual viewpoint are switched by the step of switching to the position and angle substantially the same as the position and orientation of the object.

According to the above process, the virtual viewpoint can be switched to a position and an angle at which the visual field image is projected from a viewpoint substantially similar to the viewpoint of the avatar's possession, according to the third input operation of the user of the viewing terminal. Here, the possession of the avatar is, for example, a camera object held by the avatar. According to the above processing, the user of the viewing terminal, that is, the viewer, can view the video from the avatar's possession viewpoint according to his / her own input operation. Therefore, the degree of freedom of camera work for live distribution can be increased, and thus the interest of live distribution can be improved.

(Item 8) In any one of (Item 1) to (Item 7), the program differs from the virtual viewpoint in response to the reception of the fourth input operation by the processor and the viewing terminal. 2 The step of arranging the virtual viewpoint at the position and angle at which the specific part of the first object is projected is executed, and the rendering step is both the image of the visual field area of the virtual viewpoint and the image of the visual field area of the second virtual viewpoint. An image containing may be rendered.

According to the above process, the specific part of the avatar can be projected from the second virtual viewpoint in response to the fourth input operation of the user of the viewing terminal. Here, the specific part of the avatar is, for example, the face or hand of the avatar. According to the above processing, the user of the viewing terminal, that is, the viewer, simultaneously performs both the image of the visual field area by the virtual viewpoint and the image of the specific part of the avatar by the second virtual viewpoint according to his / her input operation. You can see it. Therefore, the degree of freedom of camera work for live distribution can be increased, and thus the interest of live distribution can be improved.

(Item 9) In any one of the items (item 1) to (item 8), the program sets the position of the virtual viewpoint and the position of the virtual viewpoint in response to the reception of the fifth input operation by the processor and the viewing terminal. The position and angle of the virtual viewpoint are switched by the step of switching the angle to the position and angle where the specific part of the first object is reflected and the step of switching to the position and angle where the specific part of the first object is reflected in the rendering step. After that, the image of the visual field area may be rendered.

According to the above process, the virtual viewpoint can be switched to the position and angle at which the specific part of the avatar is projected according to the fifth input operation of the user of the viewing terminal. Therefore, the degree of freedom of camera work of the content can be increased, and thus the interest of the content can be improved.

(Item 10) The program (delivery program 331) has been described. According to certain aspects of the disclosure, the program is executed by a computer as a distribution terminal (200, 300) that the distributor uses for real-time distribution of content, including processors (20, 30) and memory (21, 31). The program is a program, in which a step of constructing a virtual space in which a background image is developed (S201 to S202) and a step of arranging a first object, which is an avatar corresponding to a distributor, in the virtual space (S201 to S202). S204), a step of arranging the virtual viewpoint in the virtual space at a position different from the center point of the virtual space (S208), a step of rendering an image in the viewing area of the virtual viewpoint (S212), and rendering. The steps (S102, S112) of delivering the image as content to the viewing terminal in real time are executed.

According to the above processing, since the visual field area of the virtual viewpoint arranged at a position different from the center point is drawn, the center of the virtual space can be shown to the user. Therefore, the interest of the content in the live distribution is improved.

(Item 11) A method of executing a program by a computer has been described. According to certain aspects of the disclosure, the program (viewing program 131) is executed by a computer equipped with a processor (10) and a memory (11). The method is a method in which the processor executes each step of the program described in (Item 1). The method according to (item 11) has the same effect as the program according to (item 1).

(Item 12) The information processing apparatus has been described. According to a certain aspect of the present disclosure, the information processing device controls the operation of the information processing device (viewing terminal 100) by executing the program and the memory (11) for storing the program according to (item 1). The processor (10) is provided. The information processing apparatus according to (Item 12) has the same effect as the program according to (Item 1).

1 distribution system, 4 networks, 10,20,30 processors, 11,21,31 memory, 12,22,32 storage, 15,35 touch screen, 17 cameras, 18 ranging sensors, 100 viewing terminals, 110,210, 310 Control unit, 111,311 Operation reception unit, 112 playback unit, 112A space definition unit, 112B background control unit, 112C object control unit, 112D camera control unit, 112E drawing unit, 113,313 display control unit, 114,314 communication Unit, 120, 220, 320 Storage unit, 131 Viewing program, 132 Viewer information, 332 Distributor information, 151,351 Input unit, 152,352 Display unit, 200 Distribution server, 211 Transmission / reception unit, 212 Server processing unit, 213 Data management unit, 231,331 distribution program, 300 distribution terminal, 312 distribution unit, 1020, 3030 controller, 1030 storage medium, 3010 microphone, 3020 motion capture device, 13,23,33 communication IF, 14,24,34 input / output IF

Claims (12)

A program executed by a computer as a viewing terminal for viewing content delivered in real time by a distributor using a distribution terminal, which comprises a processor and memory.
The program is delivered to the processor.
A step of receiving the first information which is information related to the content and is information including at least motion data for detecting the movement of the body of the distributor, which is distributed from the distribution terminal at any time.
Steps to build a virtual space where the background image is expanded,
A step of arranging a first object, which is an avatar corresponding to the distributor, in the virtual space,
A step of operating the first object according to the motion data in response to receiving the first information during the real-time distribution.
A step of arranging the virtual viewpoint in the virtual space at a position different from the center point of the virtual space and including the center point in the visual field area.
The step of rendering the image in the visual field area of the virtual viewpoint and
A program for executing a step of displaying the image obtained by the rendering on a display unit. The program is added to the processor.
The program according to claim 1, wherein a step of arranging a second object in the virtual space is executed so as to shield a specific area of the background image. The program according to claim 2, wherein the step of arranging the first object is to arrange the first object on the second object. The program is added to the processor.
Steps to display a map based on map data,
The step of accepting the first input operation for designating a certain point on the map is executed.
The building step builds the virtual space based on the first input operation.
The program according to any one of claims 1 to 3, wherein the background image is an image taken around a position in real space corresponding to the point. The program is added to the processor.
The step of acquiring the position information indicating the current position of the viewing terminal is executed, and the step is executed.
The step to construct is the virtual space when the distance between the position of the viewing terminal on the map indicated by the position information and the point designated by the first input operation is within a predetermined range. 4. The program according to claim 4. The program is added to the processor.
In response to the reception of the second input operation in the viewing terminal, a step of switching the position and angle of the virtual viewpoint to a position and angle substantially the same as the viewpoint of the distributor corresponding to the first object is executed. Let me
Claims 1 to 5 that the rendering step renders an image of the visual field region after the position and angle of the virtual viewpoint are switched by a step of switching to a position and angle substantially similar to the viewpoint of the distributor. The program described in any one of the above. The program is added to the processor.
A step of arranging a fourth object indicating the belongings of the avatar in the virtual space,
In response to the reception of the third input operation in the viewing terminal, the step of switching the position and angle of the virtual viewpoint to a position and angle substantially the same as the position and orientation of the fourth object is executed. ,
The rendering step renders an image of the visual field region after the position and angle of the virtual viewpoint are switched by a step of switching to a position and angle substantially similar to the position and orientation of the fourth object. The program according to any one of 1 to 6. The program is added to the processor.
In response to the reception of the fourth input operation on the viewing terminal, the step of arranging the second virtual viewpoint different from the virtual viewpoint at the position and angle at which the specific part of the first object is projected is executed.
The program according to any one of claims 1 to 7, wherein the rendering step renders an image including both an image in the visual field region of the virtual viewpoint and an image in the visual field region of the second virtual viewpoint. .. The program is added to the processor.
In response to the reception of the fifth input operation in the viewing terminal, the step of switching the position and angle of the virtual viewpoint to the position and angle in which the specific part of the first object is projected is executed.
The rendering step renders the image of the visual field region after the position and angle of the virtual viewpoint are switched by the step of switching to the position and angle in which the specific portion of the first object is projected, claims 1 to 8. The program described in any one of the above. A program that includes a processor and memory and is executed by a computer as a distribution terminal used by a distributor for real-time distribution of content.
The program is delivered to the processor.
Steps to build a virtual space where the background image is expanded,
A step of arranging a first object, which is an avatar corresponding to the distributor, in the virtual space,
A step of arranging a virtual viewpoint in the virtual space at a position different from the center point of the virtual space,
The step of rendering the image in the visual field area of the virtual viewpoint and
A program that executes a step of delivering the image obtained by the rendering to a viewing terminal as content in real time. The way a computer runs a program
The computer comprises a processor and memory.
A method in which the processor executes each step of the program according to claim 1. It is an information processing device
The information processing device
A memory for storing the program according to claim 1 and
An information processing device including a processor that controls the operation of the information processing device by executing the program.

Images