JP2022089817A - Video application program, video object rendering method, video distribution system, video distribution server, and video distribution method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce load on a computer when distributing a video in which a plurality of characters are drawn.

SOLUTION: A video distribution system 11 comprises: a first image processing unit which renders a first avatar object reflecting movement of a host user and a second avatar object reflecting movement of a guest user who requests the host user to permit participation, in a drawing space of video; and a second image processing unit which uses object data for which processing load of drawing is lighter than that for the first avatar object and the second avatar object, to render a third avatar object corresponding to a guest user who requests the host user to permit participation.

SELECTED DRAWING: Figure 1

COPYRIGHT: (C)2022,JPO&INPIT

Description

There is an application for exception of loss of novelty

The present invention relates to a moving image application program, a moving object drawing method, a moving image distribution system, a moving image distribution server, and a moving image distribution method.

Conventionally, a system for distributing a moving image including a character moving image based on a user's movement has been known (see, for example, Patent Document 1). In this system, the moving image processing device acquires the detection result of the face and body movements of the first voice actor, and also acquires the detection result of the face and body movements of the second voice actor. Further, the moving image processing device performs rendering based on the three-dimensional information of the first character and the three-dimensional information of the second character based on the acquired detection result, and the moving image of the first character and the moving image of the second character generate the moving images. Synthesize the video of.

JP-A-2015-184689

However, if the movement of the user is applied to the three-dimensional model data of the character, the character is rendered together with the background, and the like for a plurality of characters, the processing load of the moving image generator becomes large. Therefore, it is required to reduce the load on the process of generating the co-starring moving image including a plurality of characters.

The present invention has been made in view of the above circumstances, and an object of the present invention is to reduce the load on a computer that draws or distributes a moving image including a plurality of objects reflecting a user's movement.

In the moving image application program for solving the above problems, the computer has a first character object that reflects the movement of the first user and a second that reflects the movement of the second user who has requested the first user to participate. The first character object and the second character are the first image processing unit that renders the character object in the drawing space of the moving image, and the third character object corresponding to the third user who has requested the first user to participate. It functions as a second image processing unit that renders using object data, which has a smaller drawing processing load than an object.

The video distribution system that solves the above problems includes a first character object that reflects the movement of the first user and a second character object that reflects the movement of the second user who has requested the first user to participate. The first image processing unit that renders in the drawing space of the moving image and the third character object corresponding to the third user who has requested the first user to participate are drawn more than the first character object and the second character object. It is provided with a second image processing unit for rendering using object data having a small processing load.

The video distribution server that solves the above problems is a first reception unit that receives a request for participation in a video including a first character object corresponding to the first user from a second user device used by the second user, and a third user. Receives animation configuration data including motion data reflecting the movement of the second user from the second reception unit that receives a request for participation in the moving image from the third user device used by the user and the second user device. 1 The receiving unit, the first transmitting unit that transmits the animation configuration data to the first user device used by the first user, the third user device, and the third user device draw more than the first character object. It is provided with a second receiving unit for receiving simple data for drawing object data having a small processing load, and a second transmitting unit for transmitting the simple data to the first user device and the second user device.

The video distribution method for solving the above problems includes a first reception step of accepting a request for participation in a video including a first character object corresponding to the first user from a second user device used by the second user, and a third user. A second reception step of receiving a request for participation in the moving image from the third user device used by the user, and a second reception step of receiving animation configuration data including motion data reflecting the movement of the second user from the second user device. 1 reception step, a first transmission step of transmitting the animation configuration data to the first user device and the third user device used by the first user, and drawing from the third user device rather than the first character object. A second reception step of receiving simple data for drawing object data having a small processing load, and a second transmission step of transmitting the simple data to the first user device and the second user device are executed. ..

According to the present invention, it is possible to reduce the load on the computer when distributing a moving image in which a plurality of characters are drawn.

The schematic diagram which shows 1st Embodiment of a moving image distribution system. The figure which shows the user management information stored in the server of the same embodiment. The schematic diagram which shows the positional relationship between the user apparatus and a user in the real space in the same embodiment. The schematic diagram which shows the arrangement of the avatar object in the virtual space in the same embodiment. The figure which shows an example of the distribution screen in the same embodiment. The figure which shows an example of the distribution screen in the same embodiment. The conceptual diagram which shows the structure of the participation waiting line in the same embodiment. The conceptual diagram which shows the position of the avatar object of a guest user in the same embodiment. The schematic diagram which shows the procedure of collaboration delivery in the same embodiment. A sequence chart showing the procedure of collaboration distribution in the same embodiment. A sequence chart showing the procedure of collaboration distribution in the same embodiment. The figure which shows an example of the distribution screen in the same embodiment. The figure which shows an example of the distribution screen in the same embodiment. The figure which shows an example of the distribution screen in the same embodiment. The figure which shows an example of the distribution screen in the same embodiment. The schematic diagram which shows the procedure of collaboration distribution in 2nd Embodiment of a moving image distribution system. A sequence chart showing the procedure of collaboration distribution in the same embodiment. A sequence chart showing the procedure of collaboration distribution in the same embodiment.

(First Embodiment)
Hereinafter, the first embodiment of the moving image distribution system will be described.
As shown in FIG. 1, the moving image distribution system 11 includes a user device 12 and a server 13. The moving image distribution system 11 is a system that displays a moving image on the user device 12 by transmitting and receiving data between the plurality of user devices 12 and the server 13 via the network 14. A video application program for viewing and distributing a video is installed in the user device 12. Hereinafter, a user who distributes a video is referred to as a "distribution user", a user who views a video distributed by the distribution user is referred to as a "viewing user", and when these are not distinguished, they are simply referred to as a "user".

The user device 12 is a wearable computer such as a smartphone (multifunctional telephone terminal), a tablet terminal, a personal computer, a console game machine, and a head-mounted display, and an information processing device capable of reproducing moving images other than these devices.

The user device 12 includes a computer processor 20, a memory 21, a storage 22 (storage medium), and a communication interface (I / F) 23. The computer processor 20 is a computing device that loads an operating system or other program from the storage 22 or other storage into the memory 21 and executes an instruction fetched from the memory 21. The computer processor 20 is one or a plurality of arithmetic circuits such as a CPU (Central Processing Unit), a GPU (Graphic Processing Unit), and an NPU (Neural network Processing Unit). The communication I / F 23 can send and receive data to and from the server 13 and other user devices 12 via the network 14. The network 14 includes various networks such as a local area network and the Internet. The communication I / F 23 is implemented as hardware, software, or a combination thereof.

The memory 21 is a main storage device (recording medium) on which the computer processor 20 can directly read and write, and is composed of a semiconductor memory or the like. The storage 22 is an auxiliary storage device (recording medium), and is, for example, a storage medium such as a magnetic disk, an optical disk, or a semiconductor memory. The storage 22 can also be used in combination with a plurality of storages. The storage 22 stores the moving image application program 22A, the avatar data 22B for drawing the avatar object, and the object information 22C.

The moving image application program 22A is switched between a distribution mode in which the user himself distributes the moving image and a viewing mode in which the moving image distributed by another distribution user is viewed, according to the instruction of the user.

The avatar data 22B is three-dimensional model data for drawing an avatar object. The user device 12 acquires the avatar data 22B from the server 13 at a predetermined timing, such as when the moving image application program 22A is started. The avatar data 22B includes data for drawing the avatar object body, texture data applied to the avatar object body, and the like. The data for drawing the avatar object body includes polygon data, skeleton data (bone) for expressing the operation of the avatar object, and the like. The avatar data 22B includes data of a plurality of avatar object bodies. The texture data contains a plurality of part data applicable to the avatar object. For example, a plurality of parts data are prepared for each category such as "eyes", "eyebrows", "ears", and "clothes". The user selects one avatar object from a plurality of avatar objects. Also, select the part data and apply it to the avatar object body to create your own avatar object. The combination of the avatar object body and the part data selected by the user is recorded in the storage 22.

The object information 22C is data downloaded from the server 13 at a predetermined timing. The object information 22C is an object rendered in the virtual space and is information about an object other than the avatar object. Objects other than the avatar object include a gift object given by the viewing user to the avatar object or the distribution user. Gift objects include effect objects, wearing objects, and regular objects. The effect object is an object that affects the impression of the entire viewing screen of the distributed moving image, and is, for example, an object that imitates confetti, an object that imitates fireworks, and the like. The attached object is an object displayed on the display screen in association with a specific part of the avatar object. The attached object is, for example, an object that can be attached to an accessory (hat, glasses, etc.), clothes (T-shirt, etc.), a costume, and other avatar objects attached to the attached object. The attached object is provided to the avatar object by being selected by the distribution user after being provided by the viewing user. A normal object is a gift object other than a wearing object arranged as an object in a virtual space, and is, for example, an object imitating a stuffed animal or a bouquet.

Further, the object may be an object that performs physical behavior. Specifically, the object may simulate physical behavior such as falling due to gravity, motion due to application of force, and the like. A component (program) for simulating physical behavior is executably associated with a physical behavior object.

Further, the user device 12 includes a sensor unit 24, a speaker 25, a microphone 26, an operation unit 27, and a display 28. At least one of the sensor unit 24, the speaker 25, the microphone 26, the operation unit 27, and the display 28 may be provided as the same device as the computer processor 20, or may be provided so as to be connected to a device including the computer processor 20. You may.

The sensor unit 24 is a group of sensors that detect a face motion indicating a change in the facial expression of the user and a body motion indicating a change in the relative position of the user's body with respect to the sensor unit 24. Face motion includes movements such as blinking and opening and closing of the mouth. For example, the sensor unit 24 processes an infrared camera that detects infrared rays reflected on a user's face and an emitting unit that emits infrared rays toward the user, a camera that captures the user's face, and data captured by the camera. It has at least one of the image processing unit. The sensor unit 24 detects, for example, a change in facial expression from an initial state when detection is started, or a change in relative position. Alternatively, the sensor unit 24 may be a sensor other than this, or may be further combined with a sensor other than the above-mentioned sensor such as an acceleration sensor or a gyro sensor. Hereinafter, the face motion detection data and the body motion detection data are referred to as "motion data".

The speaker 25 converts voice data into voice and outputs it. The microphone 26 inputs the voice spoken by the user and converts it into voice data. The operation unit 27 is a touch panel integrated with the display 28, an operation button, a keyboard, a mouse, a controller, and the like provided in the housing of the user device 12. The display 28 outputs various images in response to the output instruction of the computer processor 20.

The computer processor 20 functions as an application management unit 20A, a motion data management unit 20B, and a display control unit 20C by executing the moving image application program 22A stored in the storage 22. The application management unit 20A manages the video application program. The application management unit 20A inputs a user's request input through the operation unit 27, a request from the server 13, and the like, and makes a request to the motion data management unit 20B or the display control unit 20C according to the contents of those requests. Output. Further, the application management unit 20A transmits a request from the motion data management unit 20B and the display control unit 20C to the server 13, and records various data received from the server 13 in the storage 22. As an example of the function realized by the application management unit 20A, there is a shooting function of capturing the entire screen of the moving image being distributed or a part thereof and saving the image data obtained as a "photograph".

The motion data management unit 20B acquires motion data from the sensor unit 24 and outputs it to the display control unit 20C.
The display control unit 20C applies motion data to the avatar data 22B in the distribution mode. Further, when the display control unit 20C displays an avatar object and an object other than the avatar object, the display control unit 20C renders the object including the avatar object. Rendering here refers to drawing processing including perspective projection, hidden surface erasing (rasterization), and the like. Rendering may be at least one of these processes. Further, in addition to the above processing, processing such as shading and texture mapping may be included. Further, the display control unit 20C generates an animation using the rendered image. Further, the display control unit 20C synthesizes (or) the animation and the GUI (Graphical User Interface) parts such as the posted message transmitted from the user device 12 of the viewing user, the notification transmitted from the server 13, and the operation buttons. The superimposed) moving image data is output to the display 28.

Further, the display control unit 20C encodes the audio data and the audio and video data input from the microphone 26. The application management unit 20A transmits the encoded data to the server 13.

Further, the display control unit 20C receives the moving image data and the audio data encoded from the server 13 in the viewing mode. Further, the display control unit 20C decodes the received data, synchronizes the moving image data and the audio data, and outputs the data to the display 28 and the speaker 25.

Next, the server 13 will be described. The server 13 is used by a service provider or the like that provides a service for distributing a moving image. The server 13 includes a computer processor 30, a memory 31, a storage 32, and a communication I / F 33. The computer processor 30 has the same configuration as the computer processor 20 of the user apparatus 12. The computer processor 30 may be configured by combining a plurality of arithmetic circuits. Further, the memory 31, the storage 32, and the communication I / F 33 have the same configuration as the user device 12. The server 13 may be composed of a plurality of devices, and the computer processor 30, the memory 31, the storage 32, and the communication I / F 33 may be distributed and provided in those devices.

The distribution program 32A is stored in the storage 32. The computer processor 20 functions as a distribution management unit 30A and a purchase processing unit 30B by executing the distribution program 32A.

The distribution management unit 30A stores various data received from the user device 12 in the storage 32, and outputs the request to the purchase processing unit 30B based on the purchase request received from the user device 12. Further, the distribution management unit 30A acquires the data requested from the user device 12 from the storage 32 or the like and transmits the data to the user device 12. Specifically, the distribution management unit 30A transmits a list of moving images being distributed in response to a request from the user device 12. When the distribution management unit 30A receives the identification information of the requested moving image from the list from the user device 12, the distribution management unit 30A transmits the distribution request to the user device 12 used by the distribution user of the requested moving image. In addition, the distribution management unit 30A receives a posted message or the like posted by the viewing user for the video being distributed. Further, the distribution management unit 30A transmits the received posted message to the user device 12 of the distribution user and the user device 12 of the viewing user. The posted message includes the content of the message, identification information of the viewing user (user's account name, etc.), posting date and time, and the like. Further, when the viewing user starts viewing the moving image, the distribution management unit 30A transmits, for example, a notification message of "user A has entered the room" to the distribution user's user device 12 and the viewing user's user device 12. The user device 12 displays the received notification message in the display field of the posted message.

The distribution management unit 30A receives a request for displaying an object on the moving image being viewed from the user device 12 used by the viewing user who views the moving image. The object for which the display request is made is an object displayed in the virtual space, and includes a gift object from the viewing user, an evaluation object that favorably evaluates the video, a support object for supporting the distribution user, and the like. A gift object makes a display request by paying a consideration associated with the viewing user. There are multiple types of consideration for making a display request. For example, a gift object (free object) that pays for points given in the video application by the user's actions such as viewing and video distribution, and a gift object (paid object) that pays for currency that can be used outside the video application. There is. The gift object may be one that pays for the display when it is displayed in response to the display request. The distribution management unit 30A transmits the display request of the gift object received from the user device 12 of the viewing user to the user device 12 of the distribution user together with the identification information of the gift object. The server 13 said, "User B presented fireworks" at a predetermined timing, such as when the display request for the gift object was received by the server 13, or when a predetermined operation was performed by the distribution user for the display request. Etc. are transmitted to the user device 12 of the distribution user and the user device 12 of the viewing user. The user device 12 displays a notification message regarding the received gift object in the display field of the posted message.

The purchase processing unit 30B performs the purchase processing of the object according to the operation of the user. The purchase process includes a purchase to obtain an object by paying for coins, points, tickets, etc. that can be used in the video application program. In addition, the purchase process may include the process of exchanging, selling, or transferring the object. Further, the purchase processing unit 30B may execute a lottery (gacha) in which a predetermined number of objects are selected from a plurality of objects by paying the consideration. The purchase processing unit 30B records the purchased object in the user device 12 or the server 13 as being owned by the user. Sales of purchaseable objects are distributed, for example, to distribution users and service providers.

Next, various data stored in the storage 32 of the server 13 will be described. User management information 32B, avatar data 32C, and object information 32D are stored in the storage 32.

The user management information 32B is information about a user who uses the moving image application program. The avatar data 32C is data for drawing an avatar object associated with each user, and is transmitted to the user device 12 in response to a request from the user device 12. The object information 32D is data for drawing a gift object, and is transmitted to the user device 12 in response to a request from the user device 12. The object information 32D includes data for drawing a gift object such as polygon data, as well as attribute information of the gift object.

The user management information 32B will be described in detail with reference to FIG. The user management information 32B is stored for each user. The user management information 32B includes user identification information (user ID and password), coins, points, and the like. The coin was purchased by the user. Points are given to users at predetermined timings such as when distribution or viewing is performed, when an event is participated, or when an object or the like is sold.

Next, with reference to FIG. 3, a method of distributing a moving image using the user device 12 will be described. The user device 12 used for distribution and viewing of moving images will be described as a smartphone. The distribution user 100 confronts the user device 12 so that at least his / her face is included in the detection range 101 of the sensor unit 24, and executes the distribution application program. At this time, the distribution user 100 puts the distribution application program into the distribution mode.

The user device 12 used by the distribution user 100 applies the motion data output from the sensor unit 24 to the avatar data 22B to generate an animation. The display 28 of the user device 12 displays a moving image including the generated animation and GUI parts. The angle of the virtual camera in the virtual space can be changed based on the user's operation. The position of the avatar object changes according to the relative distance L between the distribution user 100 and the sensor unit 24. The change in the relative distance L is detected in FIG. 4 as a change in the distance in the positive direction on the x-axis, the negative direction on the x-axis, the positive direction on the z-axis, and the negative direction on the z-axis.

The display 28 of the viewing user 102 displays a moving image based on the data transmitted from the user device 12 of the distribution user 100. The viewing user 102 performs an operation for presenting a gift object to the moving image, and the user device 12 transmits a display request for the gift object to the server 13. The server 13 transmits the display request of the gift object received from the user device 12 to the user device 12 of the distribution user 100. The user device 12 of the distribution user 100 transmits an animation in which the gift object is rendered to the server 13 based on the display request.

(motion)
Next, the collaboration distribution function (co-starring function) will be described. The viewing user 102 can participate in the collaboration distribution that displays the avatar object corresponding to the video being distributed. In participating in the collaboration distribution, the user device 12 used by the viewing user 102 transmits a participation request to the user device 12 used by the distribution user 100 who is distributing with the setting to allow the collaboration distribution via the server 13. When the distribution user 100 approves the participation request, the distribution user 100 becomes the host user 103 (see FIG. 3), the viewing user 102 who made the participation request becomes the guest user 104 (see FIG. 3), and the avatar object corresponding to the host user 103. And the avatar object corresponding to the guest user 104 are displayed in the video. It should be noted that the participation request cannot be made to the moving image distributed by the distribution user 100 who is not set to allow the collaboration distribution.

With reference to FIG. 4, a rendering method of the avatar object corresponding to the host user 103 and the avatar object corresponding to the guest user 104 will be described. The avatar object 110 corresponding to the host user 103 and the avatar object 111 of the guest user 104 are arranged at predetermined initial positions. For example, the avatar object 110 is arranged at a position on the left side when viewed from the virtual camera 118, and the avatar object 111 is arranged at a position on the right side when viewed from the virtual camera 118. As a result, the avatar object 110 is displayed on the left side of the screen and on the right side of the avatar object 111 even on the screen displayed on the display 28.

The avatar object 110 is set with a coordinate axis including a front direction 112 extending in the normal direction from the front thereof, a height direction 113, and a width direction 114. Similarly, the avatar object 111 is also set with a coordinate axis including a front direction 115, a height direction 116, and a width direction 117. The avatar objects 110 and 111 are drawn in a standing posture. Further, the avatar objects 110 and 111 can be displaced in the front direction 112 and 115 and the width direction 114 and 117. However, the movable range in which the avatar objects 110 and 111 can be displaced is limited in the virtual space, the host user's avatar object 110 is in the area on the left side of the virtual camera 118, and the guest user's avatar object 111 is in the virtual camera 118. This is the area on the right side. The positions of the avatar objects 110 and 111 cannot be swapped.

In collaboration distribution, there is a need for a viewing user or a distribution user who wants to take the same pose as a co-star or take a commemorative photo together with a gift object for collaboration distribution such as "Aigo Umbrella". In addition, various expressions are possible if the avatar objects 110 and 111 can be brought close to each other when expressing a feeling of familiarity with the co-starring partner or when acquiring image data as a commemorative photo by using the shooting function. It is ideal. However, it is not as easy to bring an avatar object in virtual space closer to an adjacent avatar object according to the user's actions, as in the case of approaching a nearby person in the real world.

For example, when the avatar objects 110 and 111 are rendered so that the front directions 112 and 115 are parallel to each other, it is difficult for one user to bring his or her avatar object closer to the avatar object of the other user. One of the reasons is that the sensor unit 24 has the highest accuracy for the movement of an object existing in the front, and the accuracy decreases as the distance from the front to the side increases. Therefore, even if the user performs an operation along the width direction (the z-axis positive direction and the z-axis negative direction in FIG. 3) such as tilting his / her body, it is not accurately reflected in the detection result of the sensor unit 24. There is. In that case, the avatar objects 110 and 111 do not operate as the user intended. Further, regardless of the accuracy of the sensor unit 24, if the user performs a complicated operation such as moving the head and the body other than the head at the same time, the avatar objects 110 and 111 may not operate as intended by the user. be.

In the present embodiment, the avatar objects 110 and 111 are rendered so that their frontal directions 112 and 115 are tilted toward the adjacent avatar objects 110 and 111. In other words, the avatar objects 110, 111 are rendered so that the vectors along their frontal directions 112, 115 approach each other as they approach the position of the virtual camera 118. More specifically, of the angles formed by the direction parallel to the front direction XC of the virtual camera 118 and the front directions 112 and 115 of the avatar objects 110 and 111, the size of the small angle θ is more than 0 ° and less than 90 °. Is. The angle θ is preferably 5 ° or more and 45 ° or less, and more preferably 10 ° or more and less than 30 °.

When the host user 103 and the guest user 104 move in the positive direction of the x-axis in FIG. 3 with respect to the user device 12 while maintaining the state of being in front of the user device 12 in the real space, the avatar objects 110 and 111 in the virtual space Moves along the frontal directions 112, 115. Since the frontal directions 112 and 115 of the avatar objects 110 and 111 are tilted toward the adjacent avatar object, the avatar objects 110 and 111 approach each other as they move. That is, since the host user 103 and the guest user 104 need only shorten the relative distance L between the host user 103 and the guest user 104 while maintaining the state of being located in front of the user device 12, the avatar objects 110 and 111 are brought close to each other. It is possible to make the avatar objects 110 and 111 perform the intended operation. On the contrary, when it is desired to move away from the co-star, the user may move away from the user device 12. As a result, it is possible to return to the original state from the state in which the avatar objects 110 and 111 are brought close to each other. It is also possible to express emotions such as "surprise" and "excitement (pull)" to the co-star. Therefore, since it is possible to easily perform an approaching motion and a moving motion with respect to the co-star, it is possible to make various expression methods in the moving image.

In FIG. 4, for convenience of explanation, the positions of the avatar objects 110 and 111 in the screen depth direction (X-axis negative direction) are the same, but the positions in the depth direction may be different. For example, the avatar object 110 may be located in front of the avatar object 111 when viewed from the virtual camera 118. The context of the avatar objects 110 and 111 may be maintained at all times. As a result, even if the avatar objects 110 and 111 approach each other and the parts of the body such as the arms appear to overlap on the screen, the part of the body of the avatar object 110 is always drawn in front of one of them. It is possible to suppress an uncomfortable expression such as a part of the body of the avatar object 110 passing through a part of the body of the other avatar object 111.

Further, the allowable lower limit value may be set in advance for the relative distance of the avatar objects 110 and 111 along the Z axis. Then, even if the user further approaches the user device 12 after the relative distance of the avatar objects 110 and 111 reaches the lower limit value, the avatar objects 110 and 111 may not approach.

FIG. 5 shows the collaboration distribution screen 130 in the initial state. On the collaboration distribution screen 130, the avatar object 110 of the host user 103 and the avatar object 111 of the guest user 104 are drawn in a state of being tilted (inwardly) toward each other while facing the front.

FIG. 6 is a collaboration distribution screen 131 when at least one of the host user 103 and the guest user 104 approaches the user device 12 while maintaining the position in front of the user device 12. The avatar objects 110 and 111 are drawn in close proximity. For example, when the host user 103 and the guest user 104 approach the user device 12, the avatar objects 110 and 111 approach the virtual camera 118. At this time, as shown in FIG. 6, the avatar objects 110 and 111 may be drawn large, and the sizes of the avatar objects 110 and 111 may not be changed.

Next, the collaboration distribution function that displays three or more avatar objects on the video screen will be described. In the collaboration distribution in which three or more avatar objects are displayed, the participation request is transmitted from each of the plurality of user devices 12 via the server 13, and the participation request is approved by the user device 12 used by the distribution user. Is started on. The viewing user's avatar object for which the participation request has been approved is added to the participation waiting line.

As shown in FIG. 7, the server 13 sequentially adds identification information 136, 137 of the avatar object (co-starring avatar) for which the participation request is approved to the participation waiting column 135. If the new participation request is approved, the identification information 138 of the new avatar object is added to the tail of the participation waiting column 135. Further, the server 13 causes the user apparatus 12 to render the avatar object corresponding to the first identification information 136 at a predetermined timing, and advances the order of the avatar objects waiting in the participation waiting column 135.

FIG. 8 schematically shows the display positions of the host user's avatar object 110 and the guest user's avatar objects 111, 140, 141 on the screen. Similar to FIGS. 5 and 6, the avatar object 111 of the first guest user is displayed at the first display position 145A on the right side of the avatar object 110 of the host user. The avatar object 110 displayed at the first display position 145A is drawn using the three-dimensional model data. Further, the avatar object 110 displayed at the first display position 145A reflects the motion data transmitted from the user device 12 used by the first guest user. That is, the avatar object 111 operates according to the movement of the body of the first guest user.

The avatar object 140 of the second guest user whose participation request is approved next to the first guest user is displayed at the second display position 145B on the left side of the rear of the avatar objects 110 and 111. The avatar object 141 of the third guest user whose participation request is approved next to the second guest user is displayed at the third display position 145C on the rear right side.

As the avatar objects 140 and 141 displayed at the second display position 145B and the third display position 145C, an image having a smaller drawing processing load than the avatar object 111 displayed at the first display position 145A is used. Hereinafter, the model data used for drawing the avatar object 110 of the host user and the avatar object 111 displayed in the first display position 145A is referred to as the first drawing data, and is referred to as the second display position 145B and the third display position 145C. The model data used to draw the displayed avatar objects 140 and 141 is called the second drawing data.

As the second drawing data, for example, two-dimensional data obtained by capturing the images of the avatar objects 140 and 141 is used. The two-dimensional data is created by the user device 12 of the host user. The user device 12 of the host user creates three-dimensional avatar object data using the avatar data 22B based on the identification information of the guest user's avatar object, and makes the data two-dimensional to obtain two-dimensional data. To create. The two-dimensional data may be transmitted from the user device 12 used by the guest user corresponding to the avatar objects 140 and 141, and may be generated by the viewing user user device 12 using the avatar data 22B. May be. Further, the second drawing data may be data whose processing load when displayed on the screen is smaller than that of the three-dimensional avatar object, and the three-dimensional model data or the two-dimensional polygon data in which the number of polygons is reduced may be used. It may be used.

The avatar object 111 displayed at the first display position 145A moves from the first display position 145A when the restriction condition is satisfied. At this time, the avatar object 111 exits by moving upward, but may be moved downward, right side, left side, back side of the screen, and front side of the screen to exit the screen. Alternatively, the moving avatar object 111 may be deleted without being displayed. The limiting condition is that the time from when the avatar object 111 is displayed at the first display position 145A until it moves reaches a predetermined fixed time. Alternatively, the limiting condition may be other than that. For example, a host user operates an operation object or the like on the screen with the user device 12, a guest user operates an operation object or the like on the screen with the user device 12, or a host user in a game in which a host user and a guest user compete against each other. Is to win or lose.

When the avatar object 111 moves upward from the first display position 145A and exits the screen, the order of the avatar objects waiting in the participation waiting row 135 is advanced, and the avatar object 140 displayed in the second display position 145B is displayed. , Is displayed at the first display position 145A. Further, the avatar object 141 displayed at the third display position 145C is displayed at the second display position 145B.

FIG. 9 is a conceptual diagram of collaboration distribution. When the collaboration distribution in which three or more avatar objects including the host user are displayed is started, the user device 12 of the participants of the collaboration distribution sends and receives various data such as motion data and voice data to each other and receives the data. Render based on the data. Specifically, the distribution user device 12A, which is a user device used by the distribution user, uses the server 13 for animation configuration data (hereinafter, simply referred to as configuration data 120) including motion data, camera data, gift object data, voice data, and the like. Send to. The camera data is data related to the virtual camera 118 set by the distribution user device 12A. The camera data is information about a shooting method of the virtual camera 118 that shoots an object in the virtual space, and includes the position and orientation of the virtual camera.

The gift object data is identification information of the gift object to be rendered, the position of the gift object, and the like. The voice data is voice data input by the distribution user device 12A via the microphone 26. Further, the configuration data 120 may include the identification information of the avatar object.

Further, the distribution user device 12A receives the configuration data 120 transmitted by the distribution user device 12B from the distribution user device 12B, which is a user device used by the first guest user.
The distribution user device 12B, which is a user device used by the first guest user, transmits the configuration data 120 to the server 13. Further, the distribution user device 12C, which is a user device used by the second guest user, does not transmit motion data, camera data, or gift object data. The distribution user device 12C transmits simple data 121 including voice data to the server 13.

When the distribution user devices 12A to 12C receive the configuration data 120 and the simple data 121 via the server 13, they perform rendering based on them and generate an animation. That is, an avatar object corresponding to the host user is generated based on the configuration data 120 transmitted from the distribution user device 12A, motion data is applied to the avatar object, and the avatar object is operated according to the movement of the host user. Generate an animation. Further, based on the configuration data 120 transmitted from the distribution user device 12B, an avatar object corresponding to the first guest user is generated, motion data is applied to the avatar object, and the avatar object is adjusted to the movement of the first guest user. Generate an animation that has been operated. The distribution user devices 12A and 12B appropriately transmit the identification data of the main body of the avatar object corresponding to the host user and the identification information of the texture data to the server 13. The distribution user devices 12A to 12C appropriately receive the information from the server 13, create an avatar object corresponding to the host user and the first guest user, and apply motion data to the avatar object.

Further, the distribution user device 12A encodes the generated moving image and transmits it to the server 13. The server 13 transmits the video data 122 encoded in the viewing user device 12D, which is the user device of the viewing user who is viewing the moving image.

(motion)
Next, with reference to FIGS. 10 and 11, a processing procedure when a host user and a plurality of guest users co-star will be described.

As shown in FIG. 10, first, the distribution user device 12A used by the host user starts distribution of a moving image (collaboration-permitted video) for which collaboration distribution (collaboration-permitted video) is permitted (step S10). The distribution user device 12B used by the first guest user and the distribution user device 12C used by the second guest user transmit a participation request to the server 13 based on the user's operation (steps S11 and S12). The participation application is sent together with the user's identification information. Further, here, a case where a participation request is transmitted together with the identification information of the avatar object will be described.

Based on the received participation request, the server 13 adds the identification information of the avatar object participating in the collaboration distribution to the participation waiting line (step S13). The avatar object contained in the waiting column is an avatar object that has not been rendered in the virtual space. When an avatar object added to the waiting line is rendered in virtual space, the identification information of that avatar is excluded from the waiting line.

The server 13 transmits the notification of the participation application to the distribution user device 12A together with the identification information of the viewing user who made the participation application (step S14). Upon receiving the participation request from the server 13, the distribution user device 12A displays the participation request approval screen on the display 28 and requests the selection of approval and denial (or standby) of the participation request. When the participation request is approved, the distribution user device 12A approves the participation request (step S15). At this time, the distribution user device 12A transmits the approved viewing user identification information to the server 13.

The server 13 adds the participant list to which the viewing user whose participation request has been approved is added to the distribution user devices 12B and 12C of the guest user, and transmits the participant list to the distribution user device 12A of the host user (step S16).

The distribution user device 12C of the second guest user refers to the participant list and determines whether or not the participation request is approved. When the distribution user device 12C determines that the participation request of the second guest user is approved, the distribution user device 12C transmits the simple data 121 to the server 13 (step S17). At this time, the identification information of the avatar object of the second guest user may be included in the simple data, or may be transmitted separately from the simple data.

The server 13 transmits the simple data 121 received from the distribution user device 12C of the second guest user to the distribution user device 12A of the host user and the distribution user device 12B of the first guest user (step S18).

Further, the distribution user device 12A of the host user transmits the configuration data 120 to the server 13 (step S19). The server 13 transmits the configuration data 120 received from the distribution user device 12A to the distribution user devices 12B and 12C of the guest user (step S20).

Further, the distribution user device 12B of the first guest user transmits the configuration data 120 to the server 13 (step S21). The server 13 transmits the configuration data 120 received from the distribution user device 12A to the distribution user device 12A of the host user and the distribution user device 12C of the second guest user (step S22).

The host user's distribution user device 12A displays and encodes an animation based on its own configuration data 120, configuration data 120 received from the distribution user device 12B, and simple data 121 received from the distribution user device 12C (step S23). ). Then, the encoded moving image data is transmitted to the viewing user device 12D (step S27).

Further, the distribution user device 12B displays an animation based on its own configuration data 120, the configuration data 120 received from the distribution user device 12A, and the simple data 121 received from the distribution user device 12C (step S25). Further, the distribution user device 12C displays an animation based on its own simple data 121 and the configuration data 120 received from the distribution user devices 12A and 12B (step S26).

The order of the timing at which the distribution user device 12C transmits the simple data 121 and the timing at which the distribution user devices 12A and 12B transmit the configuration data 120 is not particularly limited. Further, in step S16, the server 13 transmits the participant list to the distribution user device 12A of the host user, but this may be omitted.

Next, a case where the order of the guest users is changed will be described with reference to FIG.
The distribution user device 12B of the first guest user transmits an exit request when the restriction condition is satisfied (step S30). The exit request is sent with the identification information of the avatar object. Upon receiving the exit request, the server 13 updates the participation waiting line based on the identification information of the avatar object (step S31). That is, the order of the avatar objects waiting in the participation waiting line is advanced.

Further, the server 13 excludes the identification information of the avatar object corresponding to the first guest user who sent the exit request from the participant list and updates it, and transmits the updated participant list to the distribution user devices 12A to 12C ( Step S32, step S33). In the distribution user device 12A of the host user, an exit animation for exiting the avatar object excluded from the participant list from the screen is displayed on the display 28, and the exit animation is encoded together with voice data and the like (step S36). Then, the encoded data is transmitted to the viewing user device 12D (step S38). The viewing user device 12D receives the encoded data and displays the exit animation on the display 28 (step S39). When exiting the collaboration distribution, the user device 12 of the first guest user becomes the viewing user device 12D.

Further, the distribution user device 12B of the first guest user and the distribution user device 12C of the second guest user display an exit animation for exiting the avatar object corresponding to the first guest user from the screen based on the received participant list 28. Is displayed in (steps S34 and S35). Here, a case where only the second guest user is included in the participant list will be described.

When the avatar object of the first guest user leaves, the order of the second guest user is advanced. The distribution user device 12C of the second guest user displays the animation after the advance on the display 28 (step S40). In the animation after the advance, the avatar object corresponding to the second guest user is displayed at the first display position 145A, and the avatar object corresponding to the first guest user is deleted. Further, the first guest user is no longer a guest user by leaving the avatar object, and becomes a viewing user unless the participation request is made again (in FIG. 11, for convenience, the first guest in steps S34 and S35 and thereafter is also the first guest). User distribution User device 12B).

Further, the distribution user device 12C transmits the configuration data 120 to the server 13 (step S41). The server 13 transmits the configuration data 120 received from the distribution user device 12C to the distribution user device 12A of the host user (step S42).

The distribution user device 12A displays the animation after the carry on the display 28, and encodes the animation after the carry (step S43). Further, the distribution user device 12A transmits the encoded moving image data to the viewing user device 12D (step S44).

The distribution user device 12C of the second guest user displays the animation after the advance based on its own configuration data 120 and the configuration data 120 (not shown in FIG. 11) transmitted from the distribution user device 12A of the host user (not shown). Step S45). Further, in the user device 12 (delivery user device 12B) and the viewing user device 12D of the first guest user who became the viewing user due to the exit of the avatar object, after moving up based on the moving image data transmitted from the distribution user device 12A. Animation is displayed (step S46). When exiting the collaboration distribution, the user device 12 of the exiting user may display a home screen instead of a video viewing screen. That is, the user device 12 temporarily leaves the distribution room of the moving image that participated in the collaboration distribution. On the home screen, a list of videos being distributed is displayed, and you can start watching the videos by selecting the videos in the list.

In the example of FIG. 11, the case where the user following the second user does not participate in the participant list has been described, but the case where the participant list includes the identification information about the third guest user who participated in the collaboration distribution. Also transmits the configuration data 120 to the user device 12 corresponding to the third guest user in step S42. Further, the user apparatus 12 of the third guest user transmits the configuration data 120 or the simple data 121 to the server 13 in order.

Next, the screen displayed on the user apparatus 12 will be described with reference to FIGS. 12 to 15. FIG. 12 is a distribution screen 150 displayed on the user device 12 used by the distribution user. When the viewing user performs an operation for transmitting the participation request using the user device 12, the approval request window 152 is displayed on the distribution screen 150. In the approval request window 152, a wait button 154 for waiting (denying) approval and an approval button 153 for approving are displayed. When the approval button 153 is selected, the user device 12 adds the identification information of the avatar object corresponding to the user who requested the participation to the participation list.

FIG. 13 is a distribution screen 160 displayed on the user device 12 of the distribution user. A similar screen is displayed on the user device 12 used by the viewing user. The distribution screen 160 displays the avatar objects 111, 140, 141 of the three guest users. The avatar objects 110 and 111 are drawn as a three-dimensional model in the virtual space. Further, the motion data of the host user and the guest user are reflected in the avatar objects 110 and 111. The avatar objects 140 and 141 are two-dimensional data, and the motion data of the guest user is not reflected in the avatar objects 140 and 141.

As shown in FIG. 14, when the restriction condition for the avatar object 111 is satisfied, an exit animation in which the avatar object 111 moves to the upper part of the screen is displayed. As a result, the first display position 145A becomes vacant.

Then, as shown in FIG. 15, the animation after the advance is displayed. That is, the avatar object 140 displayed at the second display position 145B is erased or moved toward the upper part of the screen and exits from the screen. Then, the avatar object 140 displayed at the second display position 145B is displayed at the first display position 145A so as to move downward from the upper part of the screen. Further, the avatar object 141 displayed at the third display position 145C is arranged at the second display position 145B. If there is no waiting avatar object in the participation waiting row 135, the third display position 145C is vacant. It should be noted that the effect of the animation after the advance may be an effect other than the movement of the avatar object from the upper part to the lower part of the screen. The avatar object located at the first display position 145A is displayed as an animation of an object of three-dimensional data, and motion data is applied to the avatar object.

When the avatar object drawn by the two-dimensional model is newly drawn as the three-dimensional model behind the depth direction of the screen in this way, there is a sense of incongruity if the display mode suddenly changes in the screen. Therefore, the avatar object based on the two-dimensional model displayed at the second display position 145B is moved out of the screen and returned to the screen as an avatar object based on the three-dimensional model for drawing. As a result, it is possible to suppress giving a sense of discomfort to the viewing user and the distribution user.

Next, the effect of this embodiment will be described.
(1) The user device 12 participating in the collaboration distribution draws a moving image of an avatar object that reflects the movement of the host user and an avatar object that reflects the movement of the first guest user who has requested the host user to participate. Render to. Further, the user device 12 has a higher drawing processing load on the avatar object corresponding to the second guest user who has requested the host user to participate than the avatar object corresponding to the host user and the avatar object corresponding to the first guest user. Render with less object data. Therefore, since the load on the rendering process of the avatar object corresponding to the second guest user is reduced, the load on the drawing process can be reduced even when three or more avatar objects are drawn. .. Further, since the simple data 121 not including the motion data is transmitted from the user device 12 of the second guest user, the amount of data transmitted / received between the user devices 12 can be reduced.

(2) The avatar object corresponding to the second guest user is displayed in the moving image in a stationary state. Therefore, the load on the rendering process of the avatar object corresponding to the second guest user is reduced.

(3) The avatar object corresponding to the host user, the avatar object corresponding to the first guest user, and the avatar object corresponding to the second guest user are arranged in different areas. Therefore, it is known to the viewing user and the guest user that the avatar object corresponding to the second guest user is drawn in a display mode different from the avatar object corresponding to the host user and the avatar object corresponding to the first guest user. It will be easier to recognize. As a result, it is possible to suppress the user's misunderstanding that the factors that cause the avatar object corresponding to the second guest user to have a different display mode from the avatar object corresponding to the host user are a communication failure, a bug in the application program, and the like. can.

(4) When the avatar object drawn by the simple data 121 is newly drawn as a three-dimensional model, the avatar object is moved out of the screen and returned to the screen as an avatar object based on the three-dimensional model and drawn. As a result, it is possible to prevent the viewing user and the distribution user from giving a sense of discomfort due to the screen change due to the change of the performers in the collaboration distribution.

(5) The host user and the guest user can bring the avatar object corresponding to the host user and the avatar object corresponding to the guest user closer to each other by reducing the relative distance L with the user device 12. Therefore, various expressions are possible in the moving image.

(6) The user device 12 of the host user encodes the moving image data including the animation and transmits it to the server 13. Therefore, it is possible to reduce the load of the rendering process on the user device 12 of the viewing user.

(Second Embodiment)
Next, a second embodiment of the moving image distribution system will be described. In the first embodiment, the user device 12 of the host user encodes and transmits the moving image data, whereas in the present embodiment, the avatar object is rendered on the user device side of the viewer. Hereinafter, the same parts as those in the first embodiment are designated by the same reference numerals and detailed description thereof will be omitted.

As shown in FIG. 16, in the client-side rendering method in which the avatar object is rendered on the user device side of the viewer, the viewing user device 12D does not receive the encoded video data, but the configuration data 120 and the configuration data 120 from the server 13. The data 123 including the simple data 121 is received, and the object is rendered based on the data 123. The viewing user device 12D generates an animation using an image obtained by rendering an object, synthesizes a posted message or the like received from the server 13, and displays a moving image on the display 28.

(motion)
Next, with reference to FIGS. 17 and 18, a processing procedure when a host user and a plurality of guest users co-star will be described.

As shown in FIG. 17, the collaboration permission distribution is started (step S10), the guest user's participation request by the user device 12 (step S11, step S12), the addition to the participation waiting line by the server 13 (step S13), and the participation request. Notification (step S14) and participation list transmission (steps S14 to S16) are performed in the same manner as in the first embodiment. In the transmission of the participant list (step S16A), the server 13 also transmits the participant list to the viewing user device 12D.

Further, similarly to the first embodiment, the transmission of simple data (step S17, step S18) and the transmission of the configuration data 120 (steps S19 to S22) are performed between the participants consisting of the host user and the guest user.

The distribution user devices 12A to 12C display the animation based on the received configuration data 120 and the simple data 121 (steps S23, 25, 26). Further, the distribution user device 12A of the host user transmits the configuration data 120 and the simple data 121 to the viewing user device 12D via the server 13 (step S50). The viewing user device 12D renders based on the data 123 including the configuration data 120 and the simple data 121 received from the server 13, generates an animation, and displays it on the display 28 (step S51).

As shown in FIG. 18, the exit application (step S30), the update of the participation waiting line (step S31), and the participation list transmission (step S32) are performed in the same manner as in the first embodiment. Again, in the transmission of the participant list (step S33A), the server 13 also transmits the participant list to the viewing user device 12D.

Further, exit animations are displayed on the distribution user devices 12A to 12C participating in the collaboration distribution (steps S34 to S36). Further, the viewing user device 12D displays an exit animation based on the received participant list (step S37).

Further, the configuration data 120 transmitted from the user device 12 of the second guest user whose order has been advanced is also transmitted to the user device 12 of the viewing user in addition to the distribution user devices 12A to 12C participating in the collaboration distribution. Step S41 to S43).

Then, the distribution user devices 12A to 12C and the viewing user devices 12D render the object based on the configuration data 120 and the simple data 121, and display the animation after the advance (step S45, steps S55 to 56). When exiting the collaboration distribution, the user device 12 of the exiting user may display a home screen instead of a video viewing screen.

According to the second embodiment, in addition to the effects of (1) to (6) of the first embodiment, the following effects can be obtained.
(7) The user device 12 of the viewing user receives the animation configuration data from the server 13, renders the object, and generates the animation. Therefore, since the host user user device 12 does not need to encode the animation, the load on the host user user device 12 can be reduced.

The above embodiment can be modified and implemented as follows. The above embodiment and the following modified examples can be implemented in combination with each other within a technically consistent range.
-In the above embodiment, the avatar object is displaceable in the front direction 112, 115 and the width direction 114, 117 (see FIG. 4), but instead of or in addition to this, the height of the user's face. The positions of 113 and 116 in the height direction may be changed based on the change in position.

-In the above embodiment, the server 13 manages the order of the avatar objects to be rendered in the virtual space. Alternatively or additionally, the distribution user device 12A may manage the order of the avatar objects rendered in the virtual space. Alternatively, not only the distribution user device 12A but also each of the distribution user devices 12A to 12C may be managed.

-In the above embodiment, the server 13 manages the participant list and sends it to each user device 12. Alternatively or additionally, the distribution user device 12A may be managed.

-In the above embodiment, the user device 12 of the guest user corresponding to the avatar object displayed at the second display position and the third display position transmits the voice data. Instead, the user device 12 of the guest user corresponding to the avatar object displayed at the second display position and the third display position may transmit the second drawing data and may not transmit the voice data. According to this, the amount of communication between the user devices 12 via the server 13 can be further reduced.

-In the above embodiment, it is assumed that the video application program for viewing and distributing the video is installed in the user device 12. Instead of this, the application program for viewing the video and the application program for distributing the video may be different application programs. When distributing, the video is distributed using the distribution application program. When viewing, watch the video using the viewing application program. The display request of the gift object is made by using the viewing application program.

-In the above embodiment, the user device 12 renders the avatar object and the gift object. Alternatively or additionally, the server 13 may render the avatar object or gift object. Further, the server 13 may generate an animation using the image obtained by rendering the object and distribute it to the user device 12.

-The motion data management unit 20B is supposed to acquire detection data from the sensor unit 24 that detects changes in the user's facial expression, head movement, and the user's relative position with respect to the sensor unit 24, but at least one of these parameters is used. It may be the one to be acquired. In each case, the user device 12 may be used for the detection data that cannot be acquired by the sensor unit 24. Further, the motion data management unit 20B includes detection data for detecting movements other than the movement of the head in addition to or in place of at least one of the change in the facial expression of the user, the movement of the head, and the relative position of the user. Other detection data may be acquired.

-In the above embodiment, the movable range in which the avatar objects 110 and 111 can be displaced is limited, and the positions of the avatar objects 110 and 111 cannot be exchanged. Instead of this, the positions of the avatar objects 110 and 111 may be interchangeable according to the movements of the host user and the guest user.

-In the above embodiment, motion data may be applied to the avatar objects displayed at the second display position 145B and the third display position 145C. In this embodiment, the simple data 121 includes motion data.

Further, the avatar object displayed at the second display position 145B and the third display position 145C may be an avatar object based on the three-dimensional data, and motion data may be applied to the avatar object. In this aspect, all users other than the host user who participates in the collaboration distribution are the first guest users. Configuration data 120 is transmitted and received between the user equipment 12 of the first guest user, an object is rendered in each user equipment 12 based on its own configuration data 120 and the configuration data 120 of another user, and a moving image is generated. Ru.

-The virtual space displayed in the moving image may be an augmented reality (AR) space. For example, an avatar object, a gift object, or the like may be superimposed and displayed on a real-world image taken by the camera of the user device 12. In this aspect, the image captured by the camera of the user device 12 of the host user is used as the background image.

-In the above embodiment, the user's action is applied to the avatar object. However, from the viewpoint of reducing the processing load when generating a moving image, it is also possible to perform collaborative distribution in which videos of a plurality of users in the real world are combined on one screen. In this case, in the collaboration distribution, a part of the users displayed on one screen may be displayed as a moving image, and another user may be displayed as a still image.

-In the above embodiment, the user device 12 is an information processing device such as a smartphone, a mobile phone, a tablet terminal, a personal computer, a console game machine, a wearable computer such as a head-mounted display, and the like. Alternatively, the user device 12 may be a system composed of at least one of the information processing devices, an input device, and an output device. For example, in the user device 12, a console game machine including a computer processor 20 and the like, a sensor unit 24 provided separately from the console game machine, a controller as an operation unit 27, a speaker 25, a display 28, and the like can mutually transmit and receive data. System may be used. Further, in the user device 12, an information processing device such as a personal computer, a sensor unit 24 provided separately from the information processing device, a controller as an operation unit 27, a speaker 25, a wearable display as a display 28, and the like mutually exchange data. It may be a system that can send and receive. Alternatively, instead of the user device 12 that distributes the moving image, a system provided in the studio for distributing the moving image, which is provided with an information processing device, a sensor unit 24, an operation unit 27, a speaker 25, and a display 28, may be used. It may be used as a system for distributing moving images.

-In the above embodiment, the sensor unit 24 is provided on the user device 12 side, but at least a part of the sensor unit 24 may be attached to the body of the distribution user or provided in the vicinity of the distribution user. For example, the sensor unit 24 includes an emission unit that emits detection light such as infrared rays at predetermined intervals, and a light receiving unit that is attached to the body of the distribution user and receives the detection light and transmits the detection result to the computer processor 20. You may. Alternatively, the sensor unit 24 may be composed of a camera or the like that detects a marker attached to the body of the distribution user.
Next, the technical idea that can be grasped from the above embodiment and another example will be added below.
[1]
Computer,
The first image that renders the first character object that reflects the movement of the first user and the second character object that reflects the movement of the second user who requested the first user to participate in the drawing space of the moving image. Processing unit and
A second character object corresponding to the third user who has requested participation to the first user is rendered using object data having a smaller drawing processing load than the first character object and the second character object. An application program for movies that functions as an image processing unit.
[2]
The moving image application program according to [1], wherein the second image processing unit displays the third character object in a stationary state without reflecting the movement of the third user.
[3]
The moving image application according to [1] or [2], wherein the second image processing unit displays the third character object in an area different from the area including the first character object and the second character object. program.
[4]
The first image processing unit renders the second character object at the first display position.
The second image processing unit renders the third character object at the second display position and at the same time.
When the restriction condition regarding the second character object is satisfied, the first image processing unit causes the second character object to exit from the first display position and renders the third character object to the first display position. , The moving image application program according to any one of [1] to [3].
[5]
When the restriction condition regarding the second character object is satisfied, the first image processing unit causes the second character object to exit from the screen displayed on the display used by the user, and the second character object based on the three-dimensional model data. The moving image application program according to [4], which renders the above-mentioned first display position.
[6]
The first image processing unit is
The first character object and the second character object are rendered so that the vectors along their front directions are tilted toward the adjacent character objects.
Based on the amount of movement of the first user in the front direction and the amount of movement of the second user in the front direction, the first character object and the second character object are moved along their front directions [1]. ]-[5] The video application program according to any one of the items.
[7]
The second image processing unit draws the third character object using data in which information in the depth direction with respect to the virtual camera is reduced or omitted, according to any one of [1] to [6]. The video application program described.
[8]
The computer
A first animation generation unit that generates an animation based on motion data showing the movement of the first user, motion data showing the movement of the second user, and simple data for drawing the third character object.
The moving image application program according to any one of [1] to [7], which encodes the animation and further functions as an encoding unit for transmitting the encoded data to a server that distributes a moving image.
[9]
The computer
A configuration that receives motion data indicating the movement of the first user, motion data indicating the movement of the second user, and animation configuration data including simple data for drawing the third character object from a server that distributes a moving image. Data receiver and
Any of [1] to [8] that functions as a second animation generation unit that generates an animation including the first character object, the second character object, and the third character object based on the animation configuration data. The video application program described in Section 1.
[10]
The computer
The first image processing that renders the first character object that reflects the movement of the first user and the second character object that reflects the movement of the second user who requested the first user to participate in the drawing space of the moving image. Steps and
A second character object corresponding to the third user who has requested participation to the first user is rendered using object data having a smaller drawing processing load than the first character object and the second character object. How to draw a video object to perform image processing steps.
[11]
The first image that renders the first character object that reflects the movement of the first user and the second character object that reflects the movement of the second user who requested the first user to participate in the drawing space of the moving image. Processing unit and
A second character object corresponding to the third user who has requested participation to the first user is rendered using object data having a smaller drawing processing load than the first character object and the second character object. A video distribution system equipped with an image processing unit.
[12]
A first reception unit that receives a request for participation in a video containing a first character object corresponding to the first user from a second user device used by the second user.
A second reception unit that receives a request for participation in the video from a third user device used by the third user, and a second reception unit.
A first receiving unit that receives animation configuration data including motion data reflecting the movement of the second user from the second user device.
A first transmission unit that transmits the animation configuration data to the first user device used by the first user and the third user device, and
A second receiving unit that receives simple data for drawing object data having a smaller drawing processing load than the first character object from the third user device.
A video distribution server including a second transmission unit that transmits the simple data to the first user device and the second user device.
[13]
The first reception step of receiving a participation request from the second user device used by the second user to the video including the first character object corresponding to the first user, and
A second reception step of receiving the participation request to the moving image from the third user device used by the third user, and
A first reception step of receiving animation configuration data including motion data reflecting the movement of the second user from the second user device.
A first transmission step of transmitting the animation configuration data to the first user device and the third user device used by the first user, and
A second reception step of receiving simple data for drawing object data having a smaller drawing processing load than the first character object from the third user device.
A moving image distribution method for executing a second transmission step of transmitting the simple data to the first user device and the second user device.

11 ... Video distribution system,
12 ... User device 13 ... Server.

Claims (1)

Computer,
The first image that renders the first character object that reflects the movement of the first user and the second character object that reflects the movement of the second user who requested the first user to participate in the drawing space of the moving image. Processing unit and
A second character object corresponding to the third user who has requested participation to the first user is rendered using object data having a smaller drawing processing load than the first character object and the second character object. A video application program that functions as an image processing unit.

Images