JP7405351B2 - Video processing method, server device and computer program - Google Patents

Description

The technology disclosed in the present application relates to a video processing method for a video viewed by a viewing user, a server device and a computer program for controlling the video.

2. Description of the Related Art Video distribution services that distribute videos to terminal devices via networks have been known. In this type of video distribution service, the avatar (avatar object) of the distribution user who distributes the video is displayed, and the viewing user who watches the video and the distribution user can communicate through messages, chat, etc. A conducive environment is provided.

Furthermore, as a communication tool between viewing users and distribution users, Patent Document 1 describes electronically realizing the act (gifting) of providing an item acquired by a viewing user as a present to an avatar (distribution user) in a video. This is disclosed.

In addition, video distribution services such as REALITY (registered trademark) provide an environment in which viewing users can make gifts for each part of the avatar (distribution user) that is worn (e.g., head, neck, and hands). (Non-patent Document 1).

Note that the above-mentioned Patent Document 1 and Non-Patent Document 1 are incorporated in their entirety into this specification by reference.

Japanese Patent Application Publication No. 2012-120098

"REALITY", [online], [searched on August 27, 2019], Internet (URL: https://twitter.com/WFLE_REALITY)

The above-mentioned gifting creates a sense of intimacy and involvement in the viewing user with respect to the avatar through direct communication between the viewing user and the distribution user (avatar). However, in the case of gifting for each part of the avatar as described in Non-Patent Document 1, one viewing user makes a gift to the head of the avatar, and the gift based on the gift is sent to the head of the avatar. If a video containing an avatar attached to the avatar is distributed, other viewing users may gift the avatar's head for a specified period of time until the gift attached to the avatar's head is deleted from the video. , the gift based on the gifting will not be attached to the avatar's head. Therefore, in the case of such gifting control, only a limited number of gifting acts are reflected in the video, limiting gifting opportunities for viewing users, and inhibiting viewing users' desire to gift. There are challenges.

On the other hand, as a method to solve the above-mentioned problem, one viewing user gifts Gift A to the head of an avatar, and a video including the avatar with Gift A attached to the head based on the gifting is distributed. In such a case, an alternative method has also been proposed in which when another viewing user gifts gift B to the avatar's head, gift A is controlled to overwrite gift B immediately. However, in this case, for the viewing user who gifted Gift A, the gift that he gifted himself would suddenly be deleted from the video due to gifting by another viewing user, so the gift There are also issues such as increasing the sense of disappointment with the act of teasing.

Furthermore, gifting refers to a mode in which a viewing user purchases a gift by paying a fee to a service provider that provides a video distribution service, and then gifts the gift to a distribution user (avatar). Therefore, it can be said that increasing gifting opportunities by viewing users is a challenge for service providers.

Therefore, some embodiments disclosed in the present application provide a video processing method, a server device, and a computer program that increase gifting opportunities by viewing users.

A video processing method according to one aspect is a video processing method executed by one or more processors that execute computer-readable instructions, the video processing method being performed by a viewing user who views a video including an animation of an avatar object of a distribution user. a receiving step of receiving a request signal generated based on the operation and requesting that a first gift be given to the attachment site of the avatar object from the viewing user's terminal device via a communication line; When it is determined that the combination of the plurality of first gifts given to the attachment part of the avatar object satisfies a predetermined condition based on the request signal of the attachment part of the avatar object, The present invention includes a generation step of generating the video including an animation in which a second gift having characteristics common to the first gift is worn.

The server device according to one aspect includes one or more processors, and the processor generates a video image based on an operation of a viewing user who views a video including an animation of an avatar object of a distribution user, and generates a video image of a site where the avatar object is attached. A request signal requesting that a first gift be given to the user is received from the terminal device of the viewing user via a communication line, and the request signal is sent to the attachment site of the avatar object based on the plurality of request signals. If it is determined that the combination of the plurality of first gifts given by the user satisfies a predetermined condition, a second gift having characteristics common to the first gift is attached to the attachment site of the avatar object. The video including the attached animation is generated.

The server device according to one aspect includes one or more processors, and the processor generates a video image based on an operation of a viewing user who views a video including an animation of an avatar object of a distribution user, and generates a video image of a site where the avatar object is attached. A request signal requesting that a first gift be given to the user is received from the terminal device of the viewing user via a communication line, and the request signal is sent to the attachment site of the avatar object based on the plurality of request signals. If it is determined that the combination of the plurality of first gifts given by the avatar object satisfies a predetermined condition, a second gift having a feature common to the first gift at the attachment site of the avatar object. Such information is transmitted to the terminal device of the viewing user or the terminal device of the distribution user that generates the video including the animation in which the second gift is attached.

The computer program according to one embodiment is executed by one or more processors, and is generated based on an operation of a viewing user who views a video including an animation of an avatar object of a distribution user, and is generated based on an operation of a viewing user who views a video including an animation of an avatar object of a distribution user, a request signal requesting that a first gift be given to the user from the terminal device of the viewing user via a communication line; If it is determined that the combination of the plurality of given first gifts satisfies a predetermined condition, a second gift having characteristics common to the first gift is attached to the attachment site of the avatar object. The processor is configured to function to generate the moving image including the animated animation.

The computer program according to one embodiment is executed by one or more processors, and is generated based on an operation of a viewing user who views a video including an animation of an avatar object of a distribution user, and is generated based on an operation of a viewing user who views a video including an animation of an avatar object of a distribution user, a request signal requesting that a first gift be given to the user from the terminal device of the viewing user via a communication line; When it is determined that the combination of the plurality of first gifts given satisfies a predetermined condition, a second gift having a feature common to the first gift at the attachment site of the avatar object. The processor functions to transmit information to the terminal device or server device of the viewing user that generates the video including the animation in which the second gift is attached.

The computer program according to one embodiment is executed by one or more processors, and is generated based on an operation of a viewing user who views a video including an animation of an avatar object of a distribution user, and is generated based on an operation of a viewing user who views a video including an animation of an avatar object of a distribution user, A request signal requesting that a first gift be given to the avatar object is transmitted via a communication line, and a plurality of the first gifts are given to the attachment site of the avatar object based on the plurality of request signals. The video includes an animation in which a second gift having characteristics common to the first gift is attached to the attachment site of the avatar object when it is determined that the first gift combination satisfies a predetermined condition. In order to generate the gift, the processor is operated to receive information regarding the second gift from the distribution user's terminal device or server device.

The computer program according to one embodiment is executed by one or more processors, and is generated based on an operation of a viewing user who views a video including an animation of an avatar object of a distribution user, and is generated based on an operation of a viewing user who views a video including an animation of an avatar object of a distribution user, A request signal requesting that a first gift be given to the avatar object is transmitted via a communication line, and a plurality of the first gifts are given to the attachment site of the avatar object based on the plurality of request signals. The video includes an animation in which a second gift having characteristics common to the first gift is attached to the attachment site of the avatar object when it is determined that the first gift combination satisfies a predetermined condition. The processor functions to receive the information from the distribution user's terminal device or server device.

FIG. 1 is a block diagram showing an example of the configuration of a video distribution system according to an embodiment. FIG. 2 is a block diagram schematically showing an example of the hardware configuration of the terminal device (server device) shown in FIG. FIG. 3 is a block diagram schematically showing an example of the functions of the terminal device shown in FIG. 1. FIG. 4 is a block diagram schematically showing an example of the functions of the server device shown in FIG. 1. FIG. 5 is a diagram showing an example of a table managed by the database unit. FIG. 6A is a diagram showing an example of a table managed by the database unit, and is a diagram schematically showing an image when the determination unit makes a determination. FIG. 6B is a diagram illustrating an example of a table managed by the database unit, and is a diagram schematically illustrating an image when the determination unit makes a determination. FIG. 6C is a diagram illustrating an example of a table managed by the database unit, and is a diagram schematically illustrating an image when the determination unit makes a determination. FIG. 6D is a diagram showing an example of a table managed by the database unit, and is a diagram schematically showing an image when the determination unit makes a determination. FIG. 7 is a diagram showing an example of a confirmation screen displayed on the display unit of the distribution user's terminal device. FIG. 8 is a flow diagram showing an example of operations performed in the video distribution system shown in FIG. 1. FIG. 9 is a diagram illustrating an example of a video including an animation of a distribution user's avatar object. FIG. 10 is a diagram illustrating an example of a video including an animation of a state in which the first gift is attached to the attachment site of the distribution user's avatar object. FIG. 11 is a diagram schematically illustrating an example of an image in which a second gift is attached in place of the first gift at the attachment site of the distribution user's avatar object.

Hereinafter, various embodiments of the present invention will be described with reference to the accompanying drawings. Note that common components in the drawings are given the same reference numerals. Also, it should be noted that components depicted in one drawing may be omitted from another drawing for convenience of explanation. Furthermore, it should be noted that the attached drawings are not necessarily drawn to scale. Furthermore, the term application may also be referred to as software or a program, as long as it is a set of instructions to a computer that can be combined to achieve a certain result.

1. Configuration of video distribution system To put it simply, the video distribution system 1 disclosed in the present application allows a distribution user who distributes data and/or data regarding the distribution user's operations using the terminal device. It is possible to deliver a video containing an animation of an avatar object generated based on voice data related to utterances and singing to the terminal device of the viewing user who views the video via a communication line. .

FIG. 1 is a block diagram showing an example of the configuration of a video distribution system 1 according to an embodiment. As shown in FIG. 1, the video distribution system 1 includes a plurality of terminal devices 10 connected to a communication line (not shown) and one or more server devices 20 connected to the communication line. I can do it. Each terminal device 10 is connected to one or more server devices 20 via a communication line.

Note that the communication line may include, but is not limited to, a mobile phone network, a wireless LAN, a fixed telephone network, the Internet, an intranet, and/or Ethernet (registered trademark).

1-1. Terminal device 10
As shown in FIG. 1, the plurality of terminal devices 10 include one or more distribution user's terminal device 10 (here, terminal device 10A), one or more viewing user's terminal device 10 (here, terminal device 10A), and one or more viewing user's terminal device 10 (here, terminal device 10B to 10D). By having a common configuration, each terminal device 10 can be used as either a terminal device for distributing videos (terminal device 10 of a distribution user) or a terminal device for viewing videos (terminal device 10 of a viewing user). It is also possible.

When each terminal device 10 operates as a distribution user's terminal device 10, by executing the installed video distribution application, each terminal device 10 can collect data related to the distribution user's actions and/or utterances uttered by the distribution user. Acquire audio data related to singing, generate a video including an animation of a virtual character (avatar object) changed according to the acquired data, and send the video to the server device 20 (distribution server device 20C) via a communication line. can be sent to. The video is distributed via a communication line by the server device 20 (distribution server device 20C) to the terminal device 10 (viewing user's terminal device 10) for receiving the video, which executes the installed video viewing application. be done.

Furthermore, when each terminal device 10 operates as a distribution user's terminal device 10, the server device 20 controls the attachment site of the distribution user (avatar object) via the communication line based on the operation of the viewing user. When receiving a request signal requesting to give a gift (first gift) (gifting) from the terminal device 10 of the viewing user who views the video, each terminal device 10 transmits the request signal to the server device 20 (information processing server device). 20B) from the server device 20, it is possible to generate a video including an animation of the first gift being attached to the attachment site of the avatar object.

By the way, the attachment part of the avatar object refers to a part of the avatar object such as the avatar object's head, face, neck, right hand, left hand, waist, right foot, left foot, etc., and/or a part away from the avatar object (overhead, feet, (including the space surrounding the avatar object, such as the back).

In addition, in the above-mentioned gifting, a plurality of request signals requesting that the first gift be given to the attachment site of the avatar object (in the case where a plurality of request signals are transmitted from one viewing user terminal device 10, In the case where the request signal is transmitted from a plurality of viewing user terminal devices 10), the distribution user's terminal device 10 receives the avatar object from the server device 20 (information processing server device 20B). The video may be generated by receiving information requesting to generate a video including an animation of a second gift having the same characteristics as the first gift being attached to the attachment site of the gift. .

Note that a method for determining whether or not to generate a video including an animation of the second gift being attached to the attachment site of the avatar object will be described later, but this determination will be made by the server device 20 (described later) It is executed by the information processing server device 20B).

On the other hand, when each terminal device 10 operates as a terminal device for viewing videos (a viewing user's terminal device), each terminal device 10 operates as a terminal device of a distribution user by executing an installed video viewing application. The distributed video can be received via the server device 20 (distribution server device 20C).

Furthermore, when each terminal device 10 operates as a viewing user's terminal device, by executing a video viewing application, a gift is given to the wearing part of the distribution user (avatar object) based on the viewing user's operation. A request signal requesting to give (first gift) can be transmitted to the server device 20 (gift API server device 20A).

Furthermore, when each terminal device 10 operates as a viewing user's terminal device, by executing a video viewing application, information related to comments from the viewing user to the distribution user (avatar object) is transmitted to the server device 20 (comment API server device 20D).

Note that the video distribution application and the video viewing application described above can be installed and executed on each terminal device 10 as one integrated application or as separate applications.

Each terminal device 10 is any terminal device capable of performing such operations, including, but not limited to, a smartphone, a tablet, a mobile phone (feature phone), a personal computer, etc. It is something that can be done.

1-2. Server device 20
As shown in FIG. 1, the server device 20 can also be called a video distribution system in which a plurality of server devices, which will be described later, are combined.

The server device 20 can distribute the video transmitted from the distribution user's terminal device 10 to each viewing user's terminal device 10.

Further, the server device 20 can receive a request signal requesting a gift (first gift) to be given to the attachment site of the avatar object (gifting) from the terminal device 10 of each viewing user. . In addition, after transmitting information related to the request signal to the distribution user's terminal device 10, the gift is attached to the attachment site of the avatar object generated by the distribution user's terminal device 10 that receives the information. You can also receive videos that include animations.

In addition, in the above-mentioned gifting, a plurality of request signals requesting that the first gift be given to the attachment site of the avatar object (in the case where a plurality of request signals are transmitted from one viewing user terminal device 10, and (including cases where a request signal is transmitted from a plurality of viewing user terminal devices 10)), the server device 20 displays a state in which the second gift is attached to the attachment site of the avatar object. It is determined whether or not to generate a video including the animation, and when it is determined that the video is to be generated, information requesting generation of the video may be transmitted to the distribution user's terminal device 10. can. In addition, the server device 20 can receive from the distribution user's terminal device 10 a video including an animation of the second gift being attached to the attachment site of the avatar object.

Furthermore, the server device 20 can receive information related to comments from viewing users to distribution users (avatar objects) from the viewing users' terminal devices 10, and can transmit the comments to viewing users who view the same video. It can be distributed by

In order to perform such operations, the server device 20, in one embodiment, includes a gift API server device 20A, which are connected to each other via a communication line (including a wireless line and/or a wired line not shown). It can include an information processing server device 20B, a distribution server device 20C, and a comment API server device 20D.

The gift API server device 20A sends a request signal requesting that a gift (first gift) be given to the attachment site of the avatar object (gifting) based on the viewing user's operation to the terminal device of each viewing user. It can be received from 10. At this time, the gift API server device 20A collects identification information of the viewing user who performed the gifting act from the request signal (serial number of the terminal device 10 of each viewing user, ID information given to each viewing user, etc.), Information related to the gift that has been gifted (information about what kind of gift corresponds to which part of the avatar object it is attached to), and identification information of the distribution user who is the avatar object to be gifted (the terminal device 10 of the distribution user) (serial number, ID information given to the distribution user, etc.) and transmit the read information to the information processing server device 20B automatically or when receiving an inquiry from the information processing server device 20B. I can do it.

The information processing server device 20B receives the information read by the gift API server device 20A from the gift API server device 20A, either automatically or by making an inquiry on its own. Further, the information processing server device 20B can have a table (database) for each distribution user, and can store information related to gifts read by the gift API server device 20A in each table.

Furthermore, the information processing server device 20B refers to information related to gifts stored in the table, and identifies multiple gifting actions (multiple gifting actions by one viewing user, and multiple gifting actions by multiple viewing users). (including both gifting acts), whether to generate a video containing an animation of a state in which a second gift having characteristics in common with the first gift is attached to the attachment part of the avatar object. It can be determined whether or not. Note that this determination is made based on whether a combination of multiple gifting acts (multiple first gifts) satisfies a predetermined condition set in advance.

Furthermore, the information processing server device 20B refers to the information related to the gift stored in the table and requests to generate a video including an animation of the first gift being attached to the attachment site of the avatar object. information can be transmitted to the distribution user's terminal device 10. Similarly, if it is determined by the above-mentioned determination that a video including an animation of the second gift being attached is to be generated, an animation of the second gift being attached to the attachment part of the avatar object is included. Information requesting generation of a video can be sent to the distribution user's terminal device 10.

Furthermore, the information processing server device 20B refers to the viewing user's identification information and gift-related information stored in the table, and determines which viewing user has what type of attachment site (distribution user) of the avatar object. All or part of the information regarding whether a gift has been given can be transmitted to the comment API server device 20D.

Furthermore, if the information processing server device 20B determines that a combination of multiple gifting acts (multiple first gifts) does not satisfy a predetermined condition, the information processing server device 20B determines what kind of combination will result in the specified predetermined condition. After calculating the necessary conditions related to whether the conditions are satisfied, it is also possible to transmit the necessary conditions to the comment API server device 20D. Note that the details of the necessary conditions here will be described later.

The distribution server device 20C can distribute the video transmitted from the distribution user's terminal device 10 to each viewing user's terminal device 10.

The distribution server device 20C also receives a video including an animation of the first gift being attached to the attachment site of the avatar object from the distribution user's terminal device 10, and transmits the video to the terminal device 10 of each viewing user. can be delivered to.

Furthermore, the distribution server device 20C receives a video including an animation of the second gift being attached to the attachment site of the avatar object from the distribution user's terminal device 10, and transmits the video to the terminal device of each viewing user. It can be distributed to 10.

The comment API server device 20D can receive the viewing user's comment from the viewing user's terminal device 10, and can distribute the received comment to each viewing user who is viewing the same video. .

Additionally, the comment API server device 20D corresponds to all or part of the information received from the information processing server device 20B regarding which viewing user has given what gift to the attachment site (distribution user) of the avatar object. comments can be distributed to each viewing user who is viewing the same video.

Furthermore, the comment API server device 20D can also distribute comments corresponding to the above-mentioned requirements, received from the information processing server device 20B, to each viewing user who is viewing the same video.

In the example shown in FIG. 1, for the purpose of distributing the load, the server device 20 is configured to include a gift API server device 20A, an information processing server device 20B, a distribution server device 20C, and a comment API server device 20D. A configuration with four server devices is shown. However, at least one of these four server devices may be integrated with any of the remaining server devices.

2. Hardware Configuration of Each Device Next, an example of the hardware configuration of each of the terminal device 10 and the server device 20 will be described.

2-1. Hardware Configuration of Terminal Device 10 An example of the hardware configuration of each terminal device 10 will be described with reference to FIG. 2. FIG. 2 is a block diagram schematically showing an example of the hardware configuration of the terminal device 10 (server device 20) shown in FIG. (This is written in relation to the server device 20.)

As shown in FIG. 2, each terminal device 10 mainly includes a central processing unit 11, a main storage device 12, an input/output interface device 13, an input device 14, an auxiliary storage device 15, and an output device 16. and can include. These devices are connected to each other by a data bus and/or a control bus.

The central processing unit 11 is referred to as a "CPU" and performs calculations on instructions and data stored in the main storage device 12, and stores the results of the calculations in the main storage device 12. be. Further, the central processing unit 11 can control the input device 14, the auxiliary storage device 15, the output device 16, etc. via the input/output interface device 13. The terminal device 10 may include one or more such central processing units 11.

The main storage device 12 is called a “memory” and stores commands and commands received from the input device 14, auxiliary storage device 15, and communication line 30 (server device 20, etc.) via the input/output interface device 13. It stores data and the calculation results of the central processing unit 11. The main storage device 12 may include, but is not limited to, RAM (Random Access Memory), ROM (Read Only Memory), and/or flash memory.

The auxiliary storage device 15 is a storage device that has a larger capacity than the main storage device 12. The instructions and data (computer programs) that make up the aforementioned specific applications (video distribution applications, video viewing applications, etc.) and web browsers are stored and controlled by the central processing unit 11. Instructions and data (computer programs) can be transmitted to main memory 12 via input/output interface device 13 . The auxiliary storage device 15 can include, but is not limited to, a magnetic disk device and/or an optical disk device.

The input device 14 is a device that takes in data from the outside, and includes, but is not limited to, a touch panel, a button, a keyboard, a mouse, and/or a sensor. Sensors may include, but are not limited to, sensors including, but not limited to, one or more cameras, etc., and/or one or more microphones, etc., as described below.

The output device 16 can include, but is not limited to, a display device, a touch panel, a printer device, and the like.

In such a hardware configuration, the central processing unit 11 sequentially loads instructions and data (computer programs) constituting a specific application stored in the auxiliary storage device 15 into the main storage device 12, and By operating commands and data, the output device 16 is controlled via the input/output interface device 13, or other devices are controlled via the input/output interface device 13 and the communication line 30 (communication network 30). (For example, the server device 20 and other terminal devices 10, etc.) can transmit and receive various information.

Thereby, by executing the installed specific application, the terminal device 10 performs operations necessary to generate a video, operations necessary to transmit (distribute) and/or receive a video, and gifting actions. operations necessary for sending a comment (including various other operations described in detail later).

Note that the terminal device 10 may include one or more microprocessors and/or graphics processing units (GPUs) instead of or together with the central processing unit 11.

2-2. Hardware Configuration of Server Device 20 An example of the hardware configuration of each server device 20 will be described with reference to FIG. 2 as well. The hardware configuration of each server device 20 (gift API server device 20A, information processing server device 20B, distribution server device 20C, and comment API server device 20D) is, for example, the same as the hardware configuration of each terminal device 10 described above. It is possible to use the following. Therefore, reference numerals for components included in each server device 20 are shown in parentheses in FIG. 2.

As shown in FIG. 2, each server device 20 mainly includes a central processing unit 21, a main storage device 22, an input/output interface device 23, an input device 24, an auxiliary storage device 25, and an output device 26. and can include. These devices are connected to each other by a data bus and/or a control bus.

The central processing unit 21, the main memory 22, the input/output interface device 23, the input device 24, the auxiliary storage 25, and the output device 26 are the central processing unit 11, the main memory included in each terminal device 10 described above, respectively. The device 12, input/output interface device 13, input device 14, auxiliary storage device 15, and output device 16 can be substantially the same.

In such a hardware configuration, the central processing unit 21 sequentially loads instructions and data (computer programs) constituting a specific application stored in the auxiliary storage device 25 into the main storage device 22, and By operating commands and data, the output device 26 is controlled via the input/output interface device 23, or other devices are controlled via the input/output interface device 23 and the communication line 30 (communication network 30). (For example, each terminal device 10 etc.) It is possible to transmit and receive various information.

As a result, the server device 20 performs operations necessary to distribute the video transmitted from the terminal device 10 of each distribution user to the terminal device 10 of each viewing user, operations necessary to receive a streaming action, operations necessary to generate request information necessary to cause the terminal device 10 to generate a video, operations necessary to transmit (distribute) and/or receive a video. etc. (including various operations detailed below).

Note that the server device 20 may include one or more microprocessors and/or graphics processing units (GPUs) instead of or together with the central processing unit 21.

3. Functions of Each Device Next, a detailed example of the functions each of the terminal device 10 and the server device 20 have will be described.

3-1. Functions of Terminal Device 10 An example of the functions of the terminal device 10 will be described with reference to FIG. 3. FIG. 3 is a block diagram schematically showing an example of the functions of the terminal device 10 shown in FIG. 1.

As shown in FIG. 3, the terminal device 10 can mainly include a video generation section 100, a display section 110, a storage section 120, a user interface section 130, a gift storage section 140, and a communication section 150.

(1) Video generation section 100
The video generation unit 100 can generate a video including an animation of the distribution user's avatar object based on data regarding the distribution user's movements and/or audio data regarding speech and singing uttered by the distribution user. To achieve this, the video generation section 100 can include, for example, an acquisition section (not shown) and a processing section (not shown).

The acquisition unit includes one or more first sensors (not shown) that acquire data regarding the face, limbs, etc. (entire body) of the distribution user, and audio data regarding speech and/or singing uttered by the distribution user. and one or more second sensors (not shown) to obtain the .

In a preferred embodiment, the first sensor can include an RGB camera that images visible light and a near-infrared camera that images near-infrared rays. As such a camera, for example, one included in the True Depth camera of iPhone X (registered trademark) can be used. The second sensor can include a microphone that records audio.

First, regarding the first sensor, the acquisition unit images the distribution user's face, limbs, etc. using the first sensor placed close to the distribution user's face, limbs, etc. Thereby, the acquisition unit can generate data (for example, an MPEG file) in which an image acquired by the RGB camera is recorded over a unit time in association with a time code (a code indicating the time of acquisition). Further, the acquisition unit stores data (for example, a TSV file) recorded over a unit time by associating a predetermined number (for example, 51) of numerical values (for example, floating point numerical values) indicating the depth acquired by the near-infrared camera with the above-mentioned time code. [File format that records multiple pieces of data by separating data with tabs]) can be generated.

Regarding the near-infrared camera, specifically, a dot projector emits an infrared laser in a dot pattern onto the face and limbs of the broadcasting user, and the near-infrared camera emits the reflected infrared rays from the face and limbs of the broadcasting user. It captures the dots and generates an image of the infrared dots thus captured. The acquisition unit compares the image of the dot pattern emitted by the dot projector registered in advance and the image captured by the near-infrared camera, and calculates each point (each feature point) (for example, 51 points) in both images. The depth of each point (each feature point) (the distance between each point/feature point and the near-infrared camera) can be calculated using the positional shift at each point (each feature point). The acquisition unit can generate data recorded over a unit time by associating the numerical value indicating the depth calculated in this manner with the time code as described above.

Next, regarding the second sensor, the acquisition unit acquires audio related to speech and/or singing uttered by the distribution user using the second sensor placed close to the distribution user. Thereby, the acquisition unit can generate data (for example, an MPEG file) recorded over a unit time in association with the time code. In one embodiment, the acquisition unit uses the first sensor to acquire data regarding the distribution user's face and limbs, and at the same time uses the second sensor to acquire audio data regarding speech and/or singing uttered by the distribution user. can be obtained. In this case, the acquisition unit associates the image acquired by the RGB camera and the audio data related to speech and/or singing uttered by the distribution user using the second sensor with the same time code. Data recorded over a unit time (eg, an MPEG file) can be generated.

The acquisition unit generates data related to the distribution user's face, limbs, etc. (MPEG files, TSV files, etc.) and/or audio data related to speech and/or singing uttered by the distribution user (MPEG files, etc.). ) can be output to the processing unit.

Note that here, a case has been described in which the first sensor includes an RGB camera and an infrared camera, but the first sensor may, for example, detect infrared rays emitted by (A) an infrared laser and reflected on the face, limbs, etc. of the distribution user. (B) a plurality of RGB cameras that capture visible light; and (C) a single camera that captures visible light. In the case of (A) above, the acquisition unit can calculate the depth of each feature point on the distribution user's face, limbs, etc. using a method similar to that described above. In the case of (B) above, the acquisition unit can calculate the depth of each feature point on the distribution user's face, limbs, etc., using multiple images captured by multiple RGB cameras. . In the case of (C) above, the acquisition unit uses deep learning etc. to calculate the depth of each feature point on the distribution user's face, limbs, etc. from the image captured by a single camera. can do. In the case of (C) above, the acquisition unit calculates the depth of each feature point on the distribution user's face, limbs, etc. by analysis processing (image processing, etc.) on the image captured by a single camera. may be calculated.

The processing unit can generate a video including an animation of a virtual character (avatar object) based on data regarding the distribution user's face, limbs, etc. from the acquisition unit. Regarding the animation itself of the avatar object, the processing unit uses various information (for example, geometry information, bone information, texture information, shader information, blend shape information, etc.) stored in a character data storage unit (not shown) to It is also possible to generate a moving image of an avatar object by having a rendering unit that does not perform rendering execute rendering.

Note that any other well-known method may be used to generate a video including the animation of the avatar object of the distribution user based on data regarding the distribution user's movements and/or audio data regarding speech and singing uttered by the distribution user. It is also possible to use technology.

By the way, the video generation unit 100 sends request information to the server device 20 to the effect that the communication unit 150 (described later) generates a video including an animation of the first gift or the second gift being attached to the attachment site of the avatar object. When received from the information processing server device 20B, the request information can be received from the communication unit 150 and the video can be generated. In this case, the video generation unit 100 generates data related to the video including the animation of the avatar object generated in detail as described above, and data related to the first gift or the second gift (which part of the avatar object is attached). data regarding the shape, size, and color of the gift), and a video containing an animation of the first gift or the second gift being attached to the attachment site of the avatar object. can be generated. Specifically, information regarding which part of the avatar object the gift is to be attached to (attachment position information) is set in advance in the first gift and the second gift, and the gift is received from the viewing user. Alternatively, when a predetermined condition described below is satisfied, the first gift or the second gift can be displayed on the video at the attachment site based on the attachment position information. For example, if the first gift is a "red school bag," the wearing position information will be "back," and if the viewing user gifts a "red school bag," at least a portion of the avatar object's "back" will be displayed. The "red school bag" is displayed on the video so as to cover (or touch at least a part of the "back"). Similarly, if the second gift is a "large rainbow-colored fork", the wearing position information will be "left hand" (or "right hand"), and if the predetermined condition described below is satisfied for "left hand", The "large rainbow-colored fork" is displayed on the video so as to touch the "left hand" of the avatar object (so as to cover part of the "left hand"). Note that the position (position information) of each attachment part of the avatar object in the video is detected, for example, by a sensor such as a motion sensor described later. Therefore, when displaying a "red school bag" on the "back" of an avatar object, the position information of the "back" at that time is referred to, and the "red school bag" is displayed in the video corresponding to the "back". It will be displayed on top. In addition, while the displayed first gift and second gift are being displayed (duration described below), the displayed first gift and second gift are displayed on the video so that they move along with the movement of the attachment part of the avatar object to which they are attached. It may be displayed in Such a method of generating a video including an animation of the first gift or the second gift being attached to the attachment site of the avatar object is the method described in Patent No. 6431233 or other well-known methods. technology can be used.

(2) Display section 110
The display unit 110 displays various information required for distributing and/or viewing a video (including a video that includes an animation of the first gift or the second gift being attached to the attachment site of the avatar object). can be displayed. For example, the display unit 110 can display distributed videos, received videos, generated videos, and the like. Further, the display unit 110 can also display various comments distributed from the server device 20 to each terminal device 10 together with the received video. Furthermore, the gift information stored in the gift storage section 140 can be displayed as appropriate in accordance with the operation of the viewing user.

(3) Storage unit 120
The storage unit 120 stores various information required for distributing and/or viewing videos (including videos that include an animation of the first gift or the second gift being attached to the attachment site of the avatar object). Can be memorized.

(4) User interface section 130
The user interface unit 130 provides various functions necessary for distributing and/or viewing videos (including videos that include an animation of the first gift or the second gift being attached to the attachment site of the avatar object). Information can be input through operations by a distribution user or a viewing user. For example, the user interface unit 130 allows the viewing user to perform operations related to gifting for the avatar object (such as selecting a predetermined gift from a plurality of first gifts prepared in advance and selecting a command to give to the avatar object). The viewing user can input operation data indicating the contents (including the operation) and output the operation data to the communication unit 150. Here, the above-mentioned operation data can include, for example, information regarding what kind of gift (type of gift) the viewing user has gifted to the distribution user.

As another example, when one or more gifts are given by the viewing user, the user interface unit 130 transfers the gift (first gift or second gift) corresponding to the gift to the avatar object. When receiving an inquiry from the server device 20 regarding the attachment site, the distribution user can input operation data related to the answer to the inquiry and output the operation data to the communication unit 150.

(5) Gift storage section 140
When the video distribution application and the video viewing application are installed on the terminal device 10, the gift storage unit 140 can store gift information that is preset in these applications. Note that the gift information can include identification information such as the name, shape, and color of each gift, and the amount of each gift that will be charged to the viewing user when the viewing user makes a gift.

(6) Communication department 150
The communication unit 150 transmits various information necessary for distributing and/or viewing videos (including videos that include an animation of the first gift or the second gift being attached to the attachment site of the avatar object). , and the server device 20. For example, when the terminal device 10 operates as the terminal device 10 of the distribution user, the communication unit 150 transmits a video including an animation of the first gift or the second gift being attached to the attachment site of the avatar object. Request information for generation can be received from the server device 20 (information processing server device 20B), and the generated video can be transmitted to the server device 20 (distribution server device 20C). Similarly, when the terminal device 10 operates as the terminal device 10 of the distribution user and one or more gifts have been made by the viewing user, the gift corresponding to the gift (the first gift or An inquiry about attaching a second gift) to the attachment site of an avatar object is received from the server device 20, information on the inquiry is output to the display unit 110, and operation data related to the answer to the inquiry is displayed on the user interface. Information regarding the response received from the face unit 130 can be transmitted to the server device 20.

On the other hand, when the terminal device 10 operates as the viewing user's terminal device 10, the communication unit 150 can receive the video generated by the distribution user's terminal device 10 from the server device 20. Similarly, when the terminal device 10 operates as the viewing user's terminal device 10, information related to gifting from the viewing user to the distribution user (avatar object) and comments from the viewing user to the distribution user (avatar object) Information can be transmitted to the server device 20 (comment API server device 20D).

(7) Studio Unit A studio unit (not shown) including the above-described video generation section 100, display section 110, storage section 120, user interface section 130, gift storage section 140, and communication section 150 is used as the terminal device 10. You can also do that.

The studio unit can be implemented by an information processing device such as a personal computer, and, like the terminal device 10 and server device 20 described above, mainly includes a central processing unit, a main storage device, and an input/output interface. The device may include a device, an input device, an auxiliary storage device, and an output device. These devices are connected to each other by a data bus and/or a control bus. In addition, the studio unit functions as an information processing device that executes a specific installed application, thereby providing information on the behavior of the distribution user in the studio room (not shown) in which the studio unit is installed, and/or the distribution user. It is possible to generate a video including animation of the avatar object based on data related to speech and singing uttered by the user.

Most of the components of the video generation section 100 included in the studio unit are arranged in the studio room. Furthermore, in the studio room, distribution users perform various performances, and the studio unit detects the distribution users' movements, facial expressions, speech (including singing), and the like.

The distribution user's movements, facial expressions, speech (including singing), etc. are to be captured by various sensor groups included in the studio unit (video generation section 100). In this case, the distribution user does not necessarily have to be a human, and may be a robot capable of independent movement, for example. Further, the number of distribution users in the studio room may be one, or two or more.

The video generation section 100 in the studio unit includes various motion sensors (not shown) worn by the distribution user, a controller (not shown) held in the hand of the distribution user, and a mounting tool (not shown). and a camera (not shown) attached to the head of the broadcasting user via the broadcasting user. Further, the studio unit is provided with a microphone in the above-mentioned mounting device or somewhere in the studio room (for example, on the wall, floor, ceiling, etc.) in order to acquire the audio data of the distribution user. Additionally, the studio unit may have multiple base stations (not shown), tracking sensors (not shown), and displays (not shown) in addition to the configurations described above.

The aforementioned motion sensor can cooperate with the aforementioned base station to detect the location and orientation of the broadcast user. In one embodiment, the plurality of base stations are multi-axis laser emitters, and after emitting a synchronizing flashing light, one base station scans the laser light, e.g. around a vertical axis, and the other base stations is configured, for example, to scan a laser beam around a horizontal axis. The motion sensor is equipped with multiple optical sensors that detect the incidence of flashing light and laser light from the base station. It is possible to detect the incident angle of the laser beam detected by the sensor. The motion sensor may be, for example, Vive Tracker provided by HTC CORPORATION, or Vive Tracker provided by ZERO C SEVEN Inc. It may also be Xsens MVN Analyze provided by Microsoft.

The video generation section 100 in the studio unit acquires detection information indicating the position and orientation of each motion sensor calculated by the motion sensor. The motion sensor can detect the movement of the distribution user's body by detecting the position and orientation of the motion sensor by being attached to the distribution user's wrist, instep, waist, top of the head, or the like.

In one embodiment, a plurality of motion sensors are equipped with a large number of infrared LEDs, and light from the infrared LEDs is detected by an infrared camera installed on the floor or wall of a studio room. Position and orientation may also be detected. Furthermore, the position and orientation of the motion sensor may be detected by using a visible light LED instead of the infrared LED and detecting the light from the visible light LED with a visible light camera.

In one embodiment, a plurality of reflective markers may be used instead of a motion sensor. The reflective marker is attached to the distribution user using adhesive tape or the like. A configuration may be adopted in which the position and orientation of the reflective marker is detected by photographing the distribution user to whom the reflective marker is attached in this manner to generate photographic data, and performing image processing on this photographic data.

The controller outputs a control signal according to the distribution user's operation, and the video generation unit 100 acquires this.

The tracking sensor generates tracking information for determining virtual camera setting information for constructing a virtual space included in the video. The tracking information is calculated as a position in a three-dimensional orthogonal coordinate system and an angle around each axis, and the video generation unit 100 acquires the tracking information.

A camera attached to the distribution user's head is arranged so as to be able to capture an image of the distribution user's face. The camera acquires image data of the distribution user's face by continuously capturing images of the distribution user's face, and the video generation unit 100 acquires this image data from the camera. Note that the camera may be a 3D camera that can detect the depth of the distribution user's face.

The display functions as the display unit 110 and is placed in a position where it can be viewed by the distribution user, and the distribution user can acquire various information and view the video generated by the video generation unit 100 through the display. It is possible.

In addition to the components described above, the studio unit similarly includes the storage section 120, user interface section 130, gift storage section 140, and communication section 150 described above. Furthermore, the components of the studio unit described above are merely examples, and may include various other applicable components.

3-2. Functions of Server Device 20 An example of the functions of the server device 20 will be described with reference to FIGS. 4 to 7. FIG. 4 is a block diagram schematically showing an example of the functions of the server device 20 shown in FIG. 1. FIG. 5 is a diagram showing an example of a table managed by the database unit 230. 6A to 6D are diagrams showing examples of tables managed by the database unit 230, and are diagrams schematically showing an image when the determination unit 240 makes a determination. FIG. 7 is a diagram showing an example of a confirmation screen displayed on the display unit 110 of the distribution user's terminal device 10.

As shown in FIG. 4, the server device 20 can mainly include a storage section 200, a communication section 210, a gift processing section 220, a database section 230, a determination section 240, and a comment processing section 250.

(1) Storage unit 200
The storage unit 200 stores various information required for distributing and/or viewing videos (including videos that include an animation of the first gift or the second gift being attached to the attachment site of the avatar object). Can be memorized.

(2) Communication department 210
The communication unit 210 transmits various information required for distributing and/or viewing videos (including videos that include an animation of the first gift or the second gift being attached to the attachment site of the avatar object). , the terminal device 10 of the distribution user and/or the terminal device 10 of the viewing user. For example, the communication unit 210 receives a video (including a video that includes an animation of the first gift or the second gift being attached to the attachment site of the avatar object) from the terminal device 10 of the distribution user, and each It can be distributed to the viewing user's terminal device 10.

The communication unit 210 also receives information related to gifting from the viewing user to the distribution user (avatar object) and information related to comments from the viewing user to the distribution user (avatar object) from the viewing user's terminal device 10. Information related to gifting is output to the gift processing unit 220, and information related to comments is output to the comment processing unit 250.

Furthermore, the communication unit 210 transmits request information to the distribution user's terminal device 10 to generate a video including an animation of the first gift or the second gift being attached to the attachment site of the avatar object. be able to.

Furthermore, if multiple gifts have been made by the viewing user, and if the determination unit 240 (described later) determines that the first gift combination satisfies a predetermined condition, the second gift may be sent to the avatar object. A request signal related to an inquiry (confirmation) for obtaining confirmation from the distribution user regarding the attachment of the item to the attachment site is transmitted to the distribution user's terminal device 10, and information related to the answer to the inquiry is transmitted from the distribution user's terminal device 10. You can also receive it.

(3) Gift processing section 220
When the communication unit 210 outputs the information related to gifting (request signal related to gifting), the gift processing unit 220 outputs the identification information of the viewing user who performed the gifting act (the serial number of the terminal device 10 of each viewing user). number, ID information given to each viewing user, etc.), gift information (information about what kind of gift it is, which corresponds to which part of the avatar object it is attached to, and the name and shape of each gift)・Identification information such as color, etc.) and identification information of the distribution user who is the avatar object to be gifted (serial number of the terminal device 10 of the distribution user, ID information given to the distribution user, etc.), and the read information. is stored in the database unit 230 automatically or in response to an inquiry.

(4) Database section 230
The database unit 230 can receive various types of information read by the gift processing unit 220 from the gift processing unit 220 automatically or by inquiring the gift processing unit 220 at predetermined time intervals. Further, as shown in FIG. 5, the database unit 230 has a table (reference table) for each distribution user, and stores various information received from the gift processing unit 220 in each table of the corresponding distribution user. do.

Next, an example of information stored in the table will be explained with reference to FIG. 5.

The database unit 230 has tables 500 for each distribution user, and each table 500 is given identification information 501 of the distribution user (for example, distribution user ID information). Furthermore, the database unit 230 sequentially stores the gift information gifted in the corresponding table according to the distribution user identification information 501 received from the gift processing unit 220 (in FIG. 5, the order of gifting is taken as an example). However, for example, the first gift may be stored in the order in which the first gift is attached to the avatar object).

To explain in more detail, first, from the state where no gift is attached to the attachment part of the avatar object (initial state), there are gifts such as "red school bag", "diamond crown", and "silver fork" as shown in Figure 5. , "Golden Harisen,""GoldenRing," and "White Sneakers" in the order of gifting from one or more viewing users for the first gift, the attachment part of the avatar object is The first gifts are attached in this order (a video including an animation in which the first gifts are attached to the attachment site of the avatar object is generated). Therefore, the database unit 230 stores "red school bag,""diamondcrown,""silverfork,""goldenharisen,""goldring," and "white sneakers" in this order in table 500. (The order in which the gift was gifted or the order in which it was attached) is stored in the box 502 of the "List of attached gifts". Although not shown in FIG. 5, information regarding the attachment site of the avatar object corresponding to each first gift (the attachment site of the avatar object corresponding to "red school bag" is "back"), The identification information of the viewing user who gifted the first gift (for example, ID information of the viewing user) may be stored (displayed) in the box 502 so as to correspond to each first gift.

Next, after the six first gifts described above are attached to the attachment parts of the avatar object, "Diamond Crown", "Silver Fork", "Silver Ring", and "Copper Knife" are added. , "black backpack", "white school bag", "gold fork", "gold glasses", and "silver spoon" in the order of gifts related to the first gift from one or more viewing users. is executed. In this case, the aforementioned six first gifts have already been attached to the attachment parts of the avatar object corresponding to these gifts. In other words, there is a red school bag on the back, a diamond crown on the head, a silver fork on the left hand, a golden harisen on the right hand, and a golden harisen on the fingers. A ``gold ring'' and ``white sneakers'' are attached to both feet. On the other hand, the corresponding attachment parts overlap, such as a ``black backpack'' on the ``back'' and a ``silver ring'' on the ``finger.'' In such a case, the database unit 230 stores, at least temporarily, the further gifted first gifts in the "unattached gift list" box 503 in the order in which they were gifted. In this case, similar to box 502, information related to the attachment part of the corresponding avatar object and identification information of the viewing user who gifted each first gift are stored in box 503 in correspondence with each first gift. (display).

Based on the information stored as described above, the database unit 230 associates the type of first gift given to the attachment part of the avatar object with the number of first gifts given for each type. It can be said that I remember it. In other words, the database unit 230 uses the table 500 to manage (memorize) the number and type of first gifts given for each attachment site, and the information to be managed (stored) is determined In response to an inquiry from section 240, the information is also output to determination section 240.

Note that the first gift attached to the avatar object will be automatically deleted from the video after a predetermined period of time (for example, 1 minute or 2 minutes) has passed from the time it was attached to the avatar object. , is set in advance in the video distribution application and the video viewing application. Therefore, if a "black backpack" is gifted after a predetermined period of time has passed since the aforementioned "red school bag" was attached to the avatar object, the "black backpack" will not be attached to the avatar object. (In this case, the database 230 will store “black backpack” in box 502).

(5) Determination unit 240
When there are multiple gifting actions (including both multiple gifting actions by one viewing user and multiple gifting actions by multiple viewing users), the determination unit 240 determines the attachment site of the avatar object. It is determined whether or not to generate a video including an animation of a second gift being worn, which has characteristics common to the first gift. This determination is performed with reference to the table 500 managed by the database unit 230 (based on information output from the database unit 230), and the combination of the plurality of first gifts is determined by the video distribution application and the video viewing application. The determination is made based on whether or not a predetermined condition preset in the application is satisfied.

An example of determination will be described with reference to FIGS. 6A to 6D.

As described above with reference to FIG. 5, when focusing on the "head" of the attachment parts of the avatar object, the "diamond crown" is already attached. In this state, if a "diamond crown" is further gifted, the first gift combination will be two identical "diamond crowns" (see FIG. 6A). Here, if at least one of the aforementioned predetermined conditions is that two identical first gifts are present, then the aforementioned predetermined condition is satisfied that two "diamond crowns" are present. Such predetermined conditions can be arbitrarily set in advance in the video distribution application and the video viewing application. When the predetermined condition is satisfied in this way, the determination unit 240 includes an animation of a state in which a second gift having the same characteristics as the first gift "diamond crown" is attached to the "head" of the avatar object. It is determined that a video will be generated. In this case, the second gift may be one that changes the size, color, and/or at least part of the shape of the first gift, such as a "large diamond crown" or "100 carat diamond tiara." can be adopted as appropriate.

Next, as described above with reference to FIG. 5, if we pay attention to the "finger" of the attachment parts of the avatar object, a "gold ring" is already attached. In this state, if a "silver ring" is further gifted, the first gift combination will be a "gold ring" and a "silver ring" (see FIG. 6B). Here, if at least one of the predetermined conditions described above is that the two first gifts differing only in color come together, the combination satisfies the predetermined condition. When the predetermined condition is satisfied in this way, the determination unit 240 selects a second gift such as a "diamond ring" that has the same characteristics as the first gift "gold ring", as described with reference to FIG. 6A. It is determined that a video including an animation of the gift No. 2 being worn is to be generated.

Next, as described above with reference to FIG. 5, if we focus on the "back" of the avatar object's wearing parts, we can see that the "red school bag" is already being worn. In this state, if a "black backpack" and a "white school bag" are further gifted, the first gift combination is the three combinations of "red school bag", "black backpack", and "white school bag". (See Figure 6C). Here, if at least one of the above-mentioned predetermined conditions is that three first gifts are arranged for "back", the combination satisfies the predetermined condition. When the predetermined condition is satisfied in this way, the determination unit 240 determines the state in which the second gift, which has the same characteristics as the first gift "red school bag", is attached, as in the case described with reference to FIG. 6A. It is determined that a video containing the animation will be generated. In this case, the second gift may be, for example, "big wings." In this way, the second gift may have a feature in common with the first gift only by having the first gift and the second gift have the same attachment part. In addition, the second gift may be the same color, the same size, the same type of item (same type), etc. as the first gift. It can be said that they have common characteristics.

Next, as described above with reference to FIG. 5, when focusing on the "left hand" of the attachment parts of the avatar object, the "silver fork" is already attached. In this state, if a "silver fork", "copper knife", "gold fork", and "silver spoon" are further gifted, the first gift combination is two "silver forks". ”, “copper knife”, “gold fork”, and “silver spoon”, resulting in a total of five combinations (see FIG. 6D). Here, if at least one of the above-mentioned predetermined conditions is that the colors of gold, silver, and copper match in the "left hand" and the knives, forks, and spoons match, then the combination satisfies the predetermined conditions. When the predetermined condition is satisfied in this way, the determination unit 240 determines that the second gift, which has the same characteristics as the first gift "silver fork", is attached, as in the case described with reference to FIG. 6A. It is determined that a video including state animation is to be generated. In this case, the second gift may be, for example, a "large rainbow-colored fork."

As in the above example, the predetermined conditions described above can be set in any manner. If the predetermined conditions are simple (for example, the examples explained with reference to FIGS. 6A and 6B), many types of second gifts can appear, which increases the gifting motivation of viewing users. can be stimulated. On the other hand, even if the predetermined conditions are difficult to understand (for example, the example explained with reference to FIG. 6D), the viewing user may wonder what kind of first gifts should be combined to make the second gift appear. This arouses interest and curiosity. Therefore, opportunities for gifting by viewing users increase.

By the way, when the combination of the plurality of first gifts does not satisfy the predetermined condition, the determination unit 240 determines the necessary conditions for the combination to satisfy the predetermined condition (the number of gifts necessary to satisfy the predetermined condition). (type, number, etc.) can be calculated. The determining unit 240 compares the set predetermined condition and the combination of the plurality of first gifts, and determines what kind of first gift needs to be additionally gifted in order to satisfy the predetermined condition. Calculate the conditions for whether or not it exists. The calculated necessary conditions are output to the comment processing section 250. Here, a specific example of the necessary conditions will be explained. For example, in the case described above with reference to FIG. If only the gift item (completed) and the "black backpack" have been gifted, the necessary condition is "one first gift corresponding to the back of the avatar object". In the case of such requirements, the comment processing unit 250 synthesizes comment information such as "The gift will change if you put one more gift on your back" (displayed on the video). ) is transmitted to the distribution user's terminal device 10. Furthermore, in the case described above with reference to FIG. 6D, if the predetermined condition is that "the colors of gold, silver, and copper are the same for the left hand, and the knife, fork, and spoon are also aligned" as described above, then , if only "silver fork" (equipped), "gold fork", and "silver poon" have been gifted by the viewing user, the required condition is "a copper knife corresponding to the left hand of the avatar object" becomes. In the case of such requirements, the comment processing unit 250 synthesizes comment information such as "Currently looking for copper knife gifts!!" on the video and sends it to the distribution user. It is transmitted to the terminal device 10.

Further, when determining that the combination of the first gifts satisfies the predetermined condition, the determining unit 240 attaches the second gift to the attachment site of the avatar object (the second gift is attached to the attachment site of the avatar object). A request signal related to an inquiry for obtaining confirmation from the distribution user regarding the generation of a moving image including an animation can be output to the communication unit 210. The communication unit 210 transmits the request signal to the distribution user (the distribution user's terminal device 10), and receives information related to the answer to the inquiry from the distribution user's terminal device 10. In this case, specifically, a screen 600 as shown in FIG. 7 is displayed on the display unit 110 of the distribution user's terminal device 10. Note that "Do you want to combine?" in FIG. 7 can be replaced with "Do you want to attach the second gift to the avatar object?". In this way, if the distribution user agrees to attach the second gift to the avatar object (if the distribution user selects "Yes" in FIG. 7), the avatar object to which the second gift is attached Even if the viewing user further performs gifting on the attachment part of The determination unit 240 may instruct the comment processing unit 250 to send comment information such as “I can’t do that” or “I’m already wearing the second gift!!” to the terminal device 10 of the distribution user. .

The second gift attached to the avatar object persists within the video (the second gift exists within the video that includes an animation of the second gift being attached to the attachment site of the avatar object). ) The duration time (for example, 1 minute or 2 minutes) is set in advance, and the video distribution application and the video viewing application are configured so that the second gift is deleted from the video when the duration time elapses. It is set. The duration of the second gift may start from the time when the second gift is attached to the attachment site of the avatar object, or the time of the last gifting performed by the viewing user to satisfy a predetermined condition. The starting point may be a point in time (when a predetermined condition is met). In addition, in order to induce gifting by viewing users immediately after the second gift is deleted from the video, for example, "The second gift has disappeared" may be displayed immediately before the second gift is deleted from the video. The determining unit 240 may instruct the comment processing unit 250 to transmit comment information such as “We are recruiting new gifters!” to the distribution user's terminal device 10.

(6) Comment processing section 250
The comment processing unit 250 can receive comment information from the viewing user to the distribution user (avatar object) from the viewing user's terminal device 10 via the communication unit 210, and can receive comment information from the viewing user on the same It can be distributed to each viewing user who is viewing the video.

The comment processing unit 250 also refers to the table 500 managed by the database unit 230 and responds to all or part of the information regarding which viewing user has gifted what gift to the attachment site of the avatar object. It is also possible to create comment information and distribute the comment information to each viewing user who is viewing the same video.

Furthermore, the comment processing unit 250 refers to the table 500 managed by the database unit 230 to inform that the first gift and/or the second gift has been given to the attachment site of the avatar object. Alternatively, it is also possible to create comment information indicating that the device has been worn and distribute the comment information to each viewing user who is viewing the same video.

Furthermore, the comment processing unit 250 creates comment information corresponding to the above-mentioned requirements calculated by the determination unit, and distributes the comment information to each viewing user who is viewing the same video. You can also do it.

4. Overall Operation of Video Distribution System 1 Next, the overall operation performed in the video distribution system 1 having the above configuration will be described with reference to FIGS. 8 to 11. FIG. 8 is a flow diagram showing an example of operations performed in the video distribution system 1 shown in FIG. 1. FIG. 9 is a diagram illustrating an example of a video including an animation of a distribution user's avatar object. FIG. 10 is a diagram illustrating an example of a video including an animation of a state in which the first gift is attached to the attachment site of the distribution user's avatar object. FIG. 11 is a diagram schematically showing an example of an image in which a second gift is attached in place of the first gift at the attachment site of the distribution user's avatar object.

First, in step (hereinafter referred to as "ST") 300, the terminal device 10 (here, the terminal device 10A) starts and executes a video distribution application according to the operation of the distribution user (here, distribution user A).

Next, in ST301, the terminal device 10A generates a video including an animation of the distribution user A's avatar object based on data regarding the distribution user A's actions and/or audio data regarding speech and singing uttered by the distribution user. . As a result, as illustrated in FIG. 9, a moving image 401 including an animation of the avatar object 400 of the distribution user A is displayed on the display unit 110 of the terminal device 10A.

Further, the terminal device 10A transmits the generated video 401 to the distribution server device 20C of the server device 20. The distribution server device 20C distributes the video received from the terminal device 10A to the terminal device 10 of each viewing user who is running a video viewing application. As a result, the moving image 401 as illustrated in FIG. 9 is also displayed on the display unit 110 of the terminal device 10 of each viewing user.

Next, in ST302, the viewing user's terminal device 10 gives the first gift to the distribution user A (the part where the avatar object 400 is attached) based on the viewing user's operation on the user interface section 130. A request signal requesting this is sent to the server device 20 (gift API server device 20A).

Next, in ST303, the server device 20 (gift API server device 20A) that has received the request signal related to the first gift from the viewing user's terminal device 10 sends the identification information of the viewing user who performed the gifting act, the gifting At least the information related to the gift given and the identification information of the distribution user who is the avatar object to be gifted are read out. The various types of information that have been read are then stored in a table within the information processing server device 20B in the server device 20.

Next, in ST304, the server device 20 (information processing server device 20B) determines that the first gift has already been attached to each attachment site of the distribution user A's avatar object 400 at the time the information is stored in the table in ST303. Check whether it is installed or not. In ST304, the attachment part of the avatar object 400 gifted by the viewing user is, for example, the "head", and when the information related to the gifting is stored in the table in ST303, the first part is attached to the "head". If the gift is not attached, the server device 20 (information processing server device 20B) determines “NO” in ST304 and proceeds to ST310 by attaching a gift from the viewing user to the “head” of the avatar object 400. Information requesting generation of a video including an animation in which the first gift (for example, a "diamond crown") is attached is transmitted to the distribution user A's terminal device 10A.

On the other hand, in ST304, the attachment part of the avatar object 400 gifted by the viewing user is, for example, the "head", and when the information related to the gifting is stored in the table in ST303, the "head" is attached. If the "diamond crown" is already attached, the server device 20 (information processing server device 20B) sends a request signal to give the first gift ("diamond crown") to the "head". After understanding that there are a plurality of gifts and determining "YES" in ST304, a determination regarding the first gift combination is executed in ST320, which will be described later.

Next, as a result of the server device 20 determining “NO” in ST304 as described above, the terminal device 10A of the distribution user A is able to view the “head” of the avatar object 400 from the server device 20 (information processing server device 20B). Upon receiving information requesting to generate a video including an animation in which the first gift (for example, a "diamond crown") given by the user is attached, the terminal device 10A generates the video in ST310. generate. As a result, as illustrated in FIG. 10, the display unit 110 of the terminal device 10A displays an animation in which the first gift 410, which is a "diamond crown," is attached to the "head" of the avatar object 400 of the distribution user A. A video 402 including the following is displayed. Note that the video 402 in FIG. 10 shows an example in which the first gift 420, which is a "silver fork", is attached to the "left hand" of the avatar object 400 in the same steps.

Further, the terminal device 10A transmits the generated video 402 to the distribution server device 20C of the server device 20. In ST311, the distribution server device 20C distributes the video received from the terminal device 10A to the terminal device 10 of each viewing user who is running the video viewing application. As a result, the moving image 402 as illustrated in FIG. 10 is also displayed on the display unit 110 of the terminal device 10 of each viewing user, and the step when the server device determines "NO" in ST304 ends.

On the other hand, if the server device 20 determines "YES" in ST304 as described above, the determination unit 240 determines whether or not the first gift combination satisfies the predetermined condition. If the first gift combination satisfies the predetermined condition (“YES” in ST320), the server device 20 (determination unit 240 of the information processing server device 20B) sends the following information to distribution user A in ST321: Information related to an inquiry (confirmation) as to whether or not the second gift may be attached to the avatar object 400 is transmitted to the distribution user A's terminal device 10A. In response to this inquiry, when distribution user A sends an answer indicating that it is OK from distribution user A's terminal device 10A to server device 20, server device 20 (information processing device 20B) Information requesting generation of a video including an animation in which a second gift (for example, a "100 carat diamond tiara") is attached to a "head" is transmitted to the distribution user A's terminal device 10A. By the way, to supplement the specific example of the second gift with reference to FIG. When this is satisfied, a second gift 420Z called "Rainbow-colored Big Fork" is generated (synthesized). In this case, the first gift 420 and the second gift 420Z not only have the same attachment part of the corresponding avatar object 400, but also have the same shape, which is one of their characteristics. In this way, the first gift 420 and the second gift 420Z have some common characteristics.

Next, in ST322, the terminal device 10A of the distribution user A receives a second gift (for example, a "100 carat diamond tiara") from the server device 20 (information processing server device 20B) to the "head" of the avatar object 400. ), upon receiving information requesting to generate a video including an animation attached with the terminal device 10A, the terminal device 10A generates the video (not shown).

Further, the terminal device 10A transmits the generated video to the distribution server device 20C of the server device 20. In ST323, the distribution server device 20C distributes the video received from the terminal device 10A to the terminal device 10 of each viewing user who is running the video viewing application. As a result, the same video is displayed on the display unit 110 of the terminal device 10 of each viewing user, and if the server device determines "YES" in ST304 and "YES" in ST320, The step ends if

Note that in ST320, if the first gift combination does not satisfy the predetermined condition (“NO” in ST320), the server device 20 (determination unit 240 of the information processing server device 20B) determines the Necessary conditions for the combination to satisfy a predetermined condition (such as the type and number of gifts required to satisfy the predetermined condition) are calculated. Further, the calculated necessary conditions are converted into comment information and distributed to the terminal device 10 of each viewing user via the comment processing unit 250. Accordingly, similar comment information is also displayed on the display unit 110 of the terminal device 10 of each viewing user. In this way, the step in the case where "NO" is determined in ST320 ends.

The above steps are executed every time a viewing user performs a gifting action. Therefore, for every gifting action, a video including an animation of the first gift or the second gift being attached to each attachment part of the avatar object 400 can be generated and distributed in a timely manner.

5. Regarding Modifications In the above-mentioned embodiment, the generation of the video (including the video including the animation of the first gift or the second gift being attached to the attachment site of the avatar object) is performed by the terminal device 10 of the distribution user. However, in another embodiment, it may be executed by, for example, the server device 20 or the viewing user's terminal device 10. Further, the final video may be generated by each of the distribution user's terminal device 10, the server device 20, and the viewing user's terminal device 10, or a device other than these devices may generate a part of the video. may also generate a portion of the video. Further, in the other embodiment described above, a rendering unit (not shown) is provided in the server device 20 and/or the viewing user's terminal device 10, and the server device 20 and/or the viewing user's terminal device 10 is provided with a rendering unit (not shown). For example, so that the viewing user's terminal device 10 can generate a video (including a video that includes an animation of the first gift or the second gift being attached to the attachment site of the avatar object), In addition to the information related to the second gift and the information related to the second gift, only data related to the distribution user's actions and/or data related to the utterances and singing uttered by the distribution user are generated, and these data are transmitted to the server device 20. And/or it may be configured such that it is transmitted to the viewing user's terminal device 10.

In order to realize such a configuration, the functions of the server device 20, the viewing user's terminal device 10, and the distribution user's terminal device 10 are changed from the above-mentioned embodiment as appropriate. sell. For example, in one embodiment, the video generation unit 100 is provided in the terminal device 10, but it may be provided in the server device 20, or may be provided in the terminal device 10 or the viewing user's terminal device 10.

Further, in the above embodiment, the information processing server in the server device 20 refers to the table 500 that stores gift information and determines whether or not the first gift combination satisfies a predetermined condition by referring to the table 500. Although executed by the device 20B, in another embodiment, it may be executed by the distribution user's terminal device 10. Accordingly, although the determination unit 240 was included in the server device 20 in the above-described embodiment, it may also be included in the terminal device 10 of the distribution user.

Furthermore, in the above embodiment, based on the first gift combination given to each attachment part of the avatar object (for example, the first gift combination given to the "head"), the combination is However, in another embodiment, for example, based on a combination of a plurality of first gifts given to a plurality of attachment parts of an avatar object, the combination is determined to be a predetermined condition. It may be determined whether the conditions are satisfied. Specifically, a combination of a first gift given to the "head" and a first gift given to the "neck" of the avatar object, and a first gift given to the "left hand" of the avatar object. , a first gift given to the "right hand," a first gift given to the "left foot," and a first gift given to the "right foot."

Furthermore, in another embodiment, the above-mentioned predetermined conditions may be set to change every predetermined time (for example, every day, every week, every month).

Furthermore, as described with reference to FIG. 11 in the above-described embodiment, the second gift is set such that a predetermined gift is synthesized according to a preset combination of first gifts. However, in another embodiment, a plurality of types of second gifts are set to be combined according to a preset combination of first gifts, and the plurality of types of second gifts are combined. A second gift determined by the distribution user's selection from the gifts may be set to be attached to the avatar object.

Therefore, according to various embodiments, it is possible to provide a video processing method, a server device, and a computer program that increase gifting opportunities by viewing users.

6. Regarding Various Aspects The video processing method according to the first aspect is "a video processing method executed by one or more processors that execute computer-readable instructions, which includes an animation of an avatar object of a distribution user. A request signal generated based on an operation of a viewing user viewing a video and requesting that a first gift be given to the attachment site of the avatar object is transmitted from the viewing user's terminal device via a communication line. the receiving step, and when it is determined that the combination of the plurality of first gifts given to the attachment site of the avatar object satisfies a predetermined condition based on the plurality of request signals; and a step of generating the video including an animation in which a second gift having the same characteristics as the first gift is attached to the attachment site of the avatar object.

A video processing method according to a second aspect is the one in which "the animation is generated based on the action of the distribution user" in the first aspect.

In the video processing method according to the third aspect, in the first aspect or the second aspect, "the second gift has a size, a color, and/or at least a portion of the first gift. "changes in shape."

A video processing method according to a fourth aspect includes, in any one of the first to third aspects, "If the combination of the plurality of first gifts does not satisfy the predetermined condition, the combination The method further includes the step of transmitting comment information indicating the necessary conditions for satisfying the conditions to the terminal devices of the plurality of viewing users.

A video processing method according to a fifth aspect includes, in any one of the first to fourth aspects, "In response to the request signal received from a terminal device of a certain viewing user, the video processing method of the avatar object is Comment information indicating that the first gift has been attached to the attachment site and/or that the first gift has been attached to the attachment site of the avatar object is sent to the terminals of a plurality of viewing users. further comprising the step of transmitting the information to the device.

A video processing method according to a sixth aspect includes, in any one of the first to fifth aspects, "the determination as to whether or not the combination satisfies the predetermined condition is based on the attachment part of the avatar object." This is done based on a reference table that stores the types of the first gift given to the recipient and the number of gifts given for each type of first gift in association with each other.

A video processing method according to a seventh aspect includes, in any one of the first to sixth aspects, "If the combination satisfies the predetermined condition, the video processing method for the second gift is further comprising the step of setting a duration to last, and deleting the second gift from the video when the duration elapses.

A video processing method according to an eighth aspect is a video processing method according to any one of the first aspect to the seventh aspect, in which “the receiving step and the generating step are performed on the terminal device of the viewing user and/or the terminal device of the distribution user "is executed by a server device connected to the device via a communication line."

A video processing method according to a ninth aspect is one in which "the server device determines that the combination satisfies the predetermined condition" in the eighth aspect.

A video processing method according to a tenth aspect is that in any of the first to seventh aspects, "the generation step is executed by a terminal device of the distribution user."

A video processing method according to an eleventh aspect is that in the tenth aspect, "it is determined by the server device or the terminal device of the distribution user that the combination satisfies the predetermined condition."

A video processing method according to a twelfth aspect is that in any of the first to seventh aspects, "the generation step is executed by the viewing user's terminal device."

A video processing method according to a thirteenth aspect is the method according to the twelfth aspect, in which "it is determined by the server device or the distribution user's terminal device that the combination satisfies the predetermined condition."

A video processing method according to a fourteenth aspect includes, in any one of the first to thirteenth aspects, “the attachment part of the avatar object is separated from a part of the avatar object and/or the avatar object; It is something that lies in the parts of the body.

A video processing method according to a fifteenth aspect is the method according to any one of the first to fourteenth aspects, in which "the processor is a central processing unit (CPU), a microprocessor, or a graphics processing unit (GPU). ” is a thing.

A video processing method according to a sixteenth aspect is one in which "the communication line includes the Internet" in any of the first to fifteenth aspects.

A server device according to a seventeenth aspect includes “one or more processors, wherein the processor generates the avatar object based on an operation of a viewing user who views a video including an animation of an avatar object of a distribution user. A request signal requesting that a first gift be given to the attachment site of the avatar object is received from the terminal device of the viewing user via a communication line, and the attachment of the avatar object is performed based on the plurality of request signals. If it is determined that the combination of the plurality of first gifts given to parts satisfies a predetermined condition, a second gift having characteristics common to the first gift is given to the attachment part of the avatar object. "generate the video including the animation in which the gift is attached."

The server device according to the eighteenth aspect is characterized in that the server device includes “one or more processors, and the processor generates the avatar object based on the operation of the viewing user who views the video including the animation of the avatar object of the distribution user. A request signal requesting that a first gift be given to the attachment site of the avatar object is received from the terminal device of the viewing user via a communication line, and the attachment of the avatar object is performed based on the plurality of request signals. If it is determined that the combination of the plurality of first gifts given to parts satisfies a predetermined condition, a second gift having characteristics common to the first gift is given to the attachment part of the avatar object. information regarding the second gift to the terminal device of the viewing user or the terminal device of the distribution user that generates the video including the animation in which the second gift is attached.

A server device according to a nineteenth aspect is one in which “the processor is a central processing unit (CPU), a microprocessor, or a graphics processing unit (GPU)” in the seventeenth aspect or the eighteenth aspect. .

A server device according to a twentieth aspect is one in which “the communication line includes the Internet” in any one of the seventeenth to nineteenth aspects.

The computer program according to the twenty-first aspect is “generated by being executed by one or more processors based on the operation of a viewing user who views a video including an animation of an avatar object of a distribution user, A request signal requesting that a first gift be given to the attachment site is received from the viewing user's terminal device via a communication line, and the attachment site of the avatar object is received based on the plurality of request signals. If it is determined that the combination of the plurality of first gifts given to the avatar object satisfies a predetermined condition, a second gift having characteristics common to the first gift is attached to the attachment site of the avatar object. "generating the video that includes an animation of the gift being attached."

The computer program according to the twenty-second aspect is “generated by being executed by one or more processors based on an operation of a viewing user who views a video including an animation of an avatar object of a distribution user, A request signal requesting that a first gift be given to the attachment site is received from the viewing user's terminal device via a communication line, and the attachment site of the avatar object is received based on the plurality of request signals. If it is determined that the combination of the plurality of first gifts given to the avatar object satisfies a predetermined condition, a second gift having characteristics common to the first gift is attached to the attachment site of the avatar object. The processor functions to transmit information related to the gift to the terminal device or server device of the viewing user that generates the video including the animation in which the second gift is attached.

A computer program according to a twenty-third aspect is that in the twenty-first aspect or the twenty-second aspect, "the processor is a central processing unit (CPU), a microprocessor, or a graphics processing unit (GPU)." .

A computer program according to a twenty-fourth aspect is one in which "the communication line includes the Internet" in any one of the twenty-first to twenty-third aspects.

The computer program according to the twenty-fifth aspect is “generated by being executed by one or more processors based on the operation of a viewing user who views a video including an animation of an avatar object of a distribution user, and A request signal requesting that a first gift be given to the wearing part is transmitted via a communication line, and a plurality of gifts are given to the wearing part of the avatar object based on the plurality of request signals. If it is determined that the combination of the first gifts satisfies a predetermined condition, an animation is performed in which a second gift having characteristics common to the first gift is attached to the attachment site of the avatar object. "The processor is operated to receive information regarding the second gift from the distribution user's terminal device or server device in order to generate the video containing the second gift."

The computer program according to the twenty-sixth aspect is “generated by being executed by one or more processors based on the operation of a viewing user who views a video including an animation of an avatar object of a distribution user, A request signal requesting that a first gift be given to the wearing part is transmitted via a communication line, and a plurality of gifts are given to the wearing part of the avatar object based on the plurality of request signals. If it is determined that the combination of the first gifts satisfies a predetermined condition, an animation is performed in which a second gift having characteristics common to the first gift is attached to the attachment site of the avatar object. The processor functions to receive the moving image containing the content from the distribution user's terminal device or server device.

A computer program according to a twenty-seventh aspect is that in the twenty-fifth aspect or the twenty-sixth aspect, "the processor is a central processing unit (CPU), a microprocessor, or a graphics processing unit (GPU)." .

A computer program according to a twenty-eighth aspect is one in which "the communication line includes the Internet" in any of the twenty-fifth to twenty-seventh aspects.

7. Fields to which the technology disclosed in the present application is applicable The technology disclosed in the present application can be applied, for example, to the following fields.
(1) Application services that deliver live videos featuring avatar objects (2) Application services that allow communication using text and avatar objects (chat applications, messengers, email applications, etc.)

1 Video distribution system 10 (10A to 10D) Terminal device 20 Server device 20A Gift API server device 20B Information processing server device 20C Distribution server device 20D Comment API server device 100 Video generation section 220 Gift processing section 230 Database section 240 Judgment section 250 Comment processing section
400 Avatar object 401, 402 Video 410, 420 First gift 420Z Second gift 500 Table

Claims (30)

A video processing method performed by one or more processors executing computer-readable instructions, the method comprising:
The terminal device of the viewing user generates a request signal that is generated based on the operation of the viewing user who views a video including the animation of the distribution user's avatar object, and requests that the avatar object be given a first gift. a receiving step of receiving data from the user via a communication line;
If it is determined that the combination of the plurality of first gifts given to the avatar object based on the plurality of request signals satisfies a predetermined condition, the number of first gifts is greater than the total number of the first gifts. a generation step of synthesizing at least one second gift with a small number of second gifts, and generating the moving image including an animation in which one of the at least one second gift is attached to the avatar object;
Video processing methods including. The video processing method according to claim 1, wherein the at least one second gift is a plurality of second gifts. 3. The video processing method according to claim 2, wherein the one second gift is determined by the distribution user's selection from among the plurality of second gifts. the first gift and the second gift have common characteristics;
The common feature is that the attachment parts to the avatar objects are common.
The moving image processing method according to any one of claims 1 to 3. the first gift and the second gift have common characteristics;
The common feature is the same color, the same size, or the same type of article;
The moving image processing method according to any one of claims 1 to 3. The video processing method according to any one of claims 1 to 5, wherein the combination is a combination of a plurality of the first gifts given to a plurality of attachment parts of the avatar object. 7. The video processing method according to claim 1, wherein the predetermined condition is set to change at predetermined time intervals. When it is determined that the combination satisfies the predetermined condition, the plurality of types of the second gift are combined, and the second gift is determined by the distribution user's selection from among the plurality of types of the second gift. The video processing method according to any one of claims 1 to 7, wherein the video including the animation in which one of the second gifts is attached is generated in the generation step. The video processing method according to any one of claims 1 to 8, wherein the animation is generated based on the action of the distribution user. If the combination of the plurality of first gifts does not satisfy the predetermined condition, the method further includes the step of transmitting comment information indicating a necessary condition for the combination to satisfy the predetermined condition to the terminal devices of the plurality of viewing users. The moving image processing method according to any one of claims 1 to 9, comprising: In response to the request signal received from a terminal device of a certain viewing user, the first gift is given to the avatar object, and/or the first gift is given to the avatar object. The video processing method according to any one of claims 1 to 10, further comprising the step of transmitting comment information indicating that the device is worn to terminal devices of a plurality of viewing users. The determination as to whether the combination satisfies the predetermined condition is made by storing the type of the first gift given to the avatar object and the number of gifts given for each type of first gift in association with each other. The video processing method according to any one of claims 1 to 11, wherein the video processing method is performed based on a reference table. If the combination satisfies the predetermined condition, setting a duration for the second gift to last in the video, and deleting the second gift from the video when the duration elapses. The moving image processing method according to any one of claims 1 to 12, further comprising an erasing step. Any one of claims 1 to 13, wherein the receiving step and the generating step are performed by a server device connected to the viewing user's terminal device and/or the distribution user's terminal device via a communication line. The video processing method described in item 1. The video processing method according to any one of claims 1 to 13, wherein the generation step is executed by a terminal device of the distribution user. It is determined by a server device connected to the viewing user's terminal device and/or the distribution user's terminal device via a communication line, or by the distribution user's terminal device that the combination satisfies the predetermined condition. The moving image processing method according to claim 15. The video processing method according to any one of claims 1 to 16, wherein the processor is a central processing unit (CPU), a microprocessor, or a graphics processing unit (GPU). The video processing method according to any one of claims 1 to 17, wherein the communication line includes the Internet. comprising one or more processors,
The processor is
The terminal device of the viewing user generates a request signal that is generated based on the operation of the viewing user who views a video including the animation of the distribution user's avatar object, and requests that the avatar object be given a first gift. received via a communication line from
If it is determined that the combination of the plurality of first gifts given to the avatar object based on the plurality of request signals satisfies a predetermined condition, the number of first gifts is greater than the total number of the first gifts. The server device generates the moving image including an animation in which at least one second gift with a small number of second gifts is combined and one second gift among the at least one second gift is attached to the avatar object. comprising one or more processors,
The processor is
The terminal device of the viewing user generates a request signal that is generated based on the operation of the viewing user who views a video including the animation of the distribution user's avatar object, and requests that the avatar object be given a first gift. received via a communication line from
If it is determined that the combination of the plurality of first gifts given to the avatar object based on the plurality of request signals satisfies a predetermined condition, the number of first gifts is greater than the total number of the first gifts. The second gift is synthesized with information related to at least one second gift with a small number of items , and one second gift of the at least one second gift includes an animation attached to the avatar object. A server device that transmits the video to the viewing user's terminal device that generates the video or the distribution user's terminal device. The server device according to claim 19 or 20, wherein the processor is a central processing unit (CPU), a microprocessor, or a graphics processing unit (GPU). The server device according to any one of claims 19 to 21, wherein the communication line includes the Internet. By being executed by one or more processors,
The terminal device of the viewing user generates a request signal that is generated based on the operation of the viewing user who views a video including the animation of the distribution user's avatar object, and requests that the avatar object be given a first gift. received via a communication line from
If it is determined that the combination of the plurality of first gifts given to the avatar object based on the plurality of request signals satisfies a predetermined condition, the number of first gifts is greater than the total number of the first gifts. generating the video including an animation in which at least one second gift of the at least one second gift is attached to the avatar object;
A computer program that causes the processor to function as described above. By being executed by one or more processors,
The terminal device of the viewing user generates a request signal that is generated based on the operation of the viewing user who views a video including the animation of the distribution user's avatar object, and requests that the avatar object be given a first gift. received via a communication line from
If it is determined that the combination of the plurality of first gifts given to the avatar object based on the plurality of request signals satisfies a predetermined condition, the number of first gifts is greater than the total number of the first gifts. The second gift is synthesized with information related to at least one second gift with a small number of items , and one second gift of the at least one second gift includes an animation attached to the avatar object. transmitting the video to the terminal device or server device of the viewing user that generates the video;
A computer program that causes the processor to function as described above. 25. A computer program product according to claim 23 or claim 24, wherein the processor is a central processing unit (CPU), a microprocessor or a graphics processing unit (GPU). 26. A computer program product according to any one of claims 23 to 25, wherein the communication line includes the Internet. By being executed by one or more processors,
A request signal is generated based on the operation of a viewing user who views a video including an animation of the distribution user's avatar object, and transmits a request signal requesting that the avatar object be given a first gift via a communication line. death,
If it is determined that the combination of the plurality of first gifts given to the avatar object based on the plurality of request signals satisfies a predetermined condition, the number of first gifts is greater than the total number of the first gifts. of the at least one second gift is combined to generate the video including an animation in which the at least one second gift of the at least one second gift is attached to the avatar object. 2. Receive information related to the gift from the distribution user's terminal device or server device,
A computer program that causes the processor to function as described above. By being executed by one or more processors,
A request signal is generated based on the operation of a viewing user who views a video including an animation of the distribution user's avatar object, and transmits a request signal requesting that the avatar object be given a first gift via a communication line. death,
If it is determined that the combination of the plurality of first gifts given to the avatar object based on the plurality of request signals satisfies a predetermined condition, the number of first gifts is greater than the total number of the first gifts. At least one second gift with a small number of images is synthesized, and one of the at least one second gift is attached to the avatar object and delivers the video including the animation attached to the avatar object. receive from the device,
A computer program that causes the processor to function as described above. 29. A computer program product according to claim 27 or claim 28, wherein the processor is a central processing unit (CPU), a microprocessor or a graphics processing unit (GPU). 30. A computer program product according to any one of claims 27 to 29, wherein the communication line includes the Internet.

Images