JP2024063117A - Computer program, server device, terminal device, and method - Google Patents

Abstract

[Problem] To provide a computer program, a server device, a terminal device, and a method capable of expressing an avatar more appropriately. [Solution] In a system including one or more server devices connected to a communication network and one or more terminal devices connected to the communication network, the server device has an acquisition unit that acquires information related to one or more of a predetermined action by a broadcaster, the broadcaster's facial expression, the broadcaster's complexion, the broadcaster's voice, or a predetermined situation occurring to the broadcaster, and a condition application unit that determines whether the acquired information satisfies a predetermined condition, and when it is determined that the predetermined condition is satisfied, generates information for changing the facial expression or pose of an avatar related to the broadcaster that has been generated based on the facial expression or pose related to the broadcaster to a predetermined facial expression or pose corresponding to the predetermined condition, and displays the avatar to which the predetermined facial expression or pose has been applied on the terminal device. [Selected Figure] Figure 3

Description

The technology disclosed in this application relates to a computer program, a server device, a terminal device, and a method.

In recent years, technology using avatars played by broadcasters has been developed. In this type of technology, viewers can view the avatars played by broadcasters via a broadcasting service.

In relation to services using avatars, a service called "Custom Cast" is known as a service that uses a technology to control the facial expressions and movements of an avatar object based on the movements of a performer, etc. (Non-Patent Document 1). In this service, a performer pre-assigns one of a large number of prepared facial expressions and movements to each of a number of flick directions on a smartphone screen, and by flicking the smartphone screen in the direction corresponding to the desired facial expression or movement, the performer can make the avatar object displayed in the video express that facial expression or movement.

"Custom Cast", [online], Custom Cast Inc., [searched December 10, 2019], Internet (URL: https://customcast.jp/)

However, in the technology disclosed in Non-Patent Document 1, the performer must flick the smartphone screen while speaking, which may make it difficult for the performer to perform the flick operation.

The above non-patent document 1 is incorporated herein in its entirety by reference.

Some embodiments disclosed in the present application provide a method for creating a more suitable representation of an avatar.
A computer program, a server device, a terminal device, and a method are provided.

A computer program in one embodiment, when executed by one or more processors, causes the one or more processors to function as follows: detect information relating to one or more of a predetermined action by a broadcaster, the broadcaster's facial expression, the broadcaster's complexion, the broadcaster's voice, or a predetermined situation occurring to the broadcaster; determine whether the detected information satisfies a predetermined condition; and, if it is determined that the predetermined condition is satisfied, generate information for changing the facial expression or pose of an avatar relating to the broadcaster that was generated based on the facial expression or pose of the broadcaster to a predetermined facial expression or pose that corresponds to the predetermined condition; and display the avatar with the predetermined facial expression or pose applied.

A computer program according to one embodiment is executed by one or more processors to detect information relating to one or more of a predetermined action by a distributor, a facial expression of the distributor, a complexion of the distributor, the voice of the distributor, or a predetermined situation occurring to the distributor, determine whether the detected information satisfies a predetermined condition, and if it is determined that the predetermined condition is satisfied, generate information for changing a facial expression or pose of an avatar relating to the distributor that has been generated based on a facial expression or pose relating to the distributor to a predetermined facial expression or pose corresponding to the predetermined condition, and
The one or more processors are operable to transmit.

In addition, a server device in one embodiment is equipped with one or more processors, and the processor executes computer-readable instructions to detect information related to one or more of a predetermined action by a broadcaster, a facial expression of the broadcaster, a complexion of the broadcaster, the voice of the broadcaster, or a predetermined situation occurring to the broadcaster, determines whether the detected information satisfies a predetermined condition, and if it is determined that the predetermined condition is satisfied, generates information for changing the facial expression or pose of an avatar related to the broadcaster that was generated based on the facial expression or pose of the broadcaster to a predetermined facial expression or pose that corresponds to the predetermined condition, and transmits the information for changing the predetermined facial expression or pose.

In addition, a method according to one embodiment is a method executed by one or more processors executing computer-readable instructions, which detects information relating to one or more of a predetermined action by a broadcaster, a facial expression of the broadcaster, a complexion of the broadcaster, the voice of the broadcaster, or a predetermined situation occurring to the broadcaster, determines whether the detected information satisfies a predetermined condition, and if it is determined that the predetermined condition is satisfied, generates information for changing the facial expression or pose of an avatar relating to the broadcaster that was generated based on the facial expression or pose of the broadcaster to a predetermined facial expression or pose that corresponds to the predetermined condition, and displays the avatar with the predetermined facial expression or pose applied.

In addition, a method according to one embodiment is a method executed by one or more processors executing computer-readable instructions, which detects information relating to one or more of a predetermined action by a broadcaster, a facial expression of the broadcaster, a complexion of the broadcaster, the voice of the broadcaster, or a predetermined situation occurring to the broadcaster, determines whether the detected information satisfies a predetermined condition, and if it is determined that the predetermined condition is satisfied, generates information for changing the facial expression or pose of an avatar relating to the broadcaster that was generated based on the facial expression or pose of the broadcaster to a predetermined facial expression or pose that corresponds to the predetermined condition, and transmits the information for changing the predetermined facial expression or pose.

In addition, a viewer terminal in one embodiment is equipped with one or more processors, and the processor executes computer-readable instructions to detect information related to one or more of a predetermined action by a distributor, a facial expression of the distributor, a complexion of the distributor, the voice of the distributor, or a predetermined situation occurring to the distributor, determines whether the detected information satisfies a predetermined condition, and if it is determined that the predetermined condition is satisfied, generates information to change the facial expression or pose of an avatar related to the distributor that was generated based on the facial expression or pose of the distributor to a predetermined facial expression or pose that corresponds to the predetermined condition, and displays the avatar with the predetermined facial expression or pose applied.

In addition, a broadcaster terminal in one embodiment is equipped with one or more processors, and the processor executes computer-readable instructions to detect information related to one or more of a predetermined action by the broadcaster, the broadcaster's facial expression, the broadcaster's complexion, the broadcaster's voice, or a predetermined situation occurring to the broadcaster, determine whether the detected information satisfies a predetermined condition, and if it is determined that the predetermined condition is satisfied, generate information for changing the facial expression or pose of an avatar related to the broadcaster that was generated based on the facial expression or pose of the broadcaster to a predetermined facial expression or pose that corresponds to the predetermined condition, and transmit the information for changing the predetermined facial expression or pose.

FIG. 1 is a block diagram showing an example of a configuration of a system according to an embodiment. FIG. 2 is a block diagram illustrating an example of a hardware configuration of the server device 20 (terminal device 30) illustrated in FIG. FIG. 3 is a block diagram illustrating an example of the functions of a system according to an embodiment. FIG. 4 is an example of data held by the system according to an embodiment. FIG. 5 is an example of data held by a system according to an embodiment. FIG. 6 is an example of data held by the system according to an embodiment. FIG. 7 is an example of a flow of processing performed by a system according to an embodiment. FIG. 8 is an example of a flow of processing performed by a system according to an embodiment. FIG. 9 is an example of a flow of processing performed by a system according to an embodiment.

Various embodiments of the present invention will be described below with reference to the accompanying drawings. Please note that components depicted in one drawing may be omitted in another drawing for convenience of explanation. Furthermore, please note that the accompanying drawings disclose an embodiment of the present invention, but are not necessarily drawn to scale. Also,
In this document, the term application refers to software, which may be called a program, and may be instructions for a computer that are combined to produce a result.

1. Example of a System FIG . 1 is a block diagram showing an example of a system configuration according to an embodiment. As shown in FIG. 1, the system 1 may include one or more server devices 20 connected to a communication network 10, and one or more terminal devices 30 connected to the communication network 10. Note that FIG. 1 shows three server devices 20A to 20C as examples of the server devices 20, and one or more terminal devices 30.
As an example of the system, three terminal devices 30A to 30C are illustrated, but one or more other server devices 20 may be connected to the communication network 10 as the server device 20, and one or more other terminal devices 30 may be connected to the communication network 10 as the terminal device 30. In addition, in the present application documents, the term "system" is used to refer to both the server and the terminal device, or to only the server or only the terminal device. In other words, the system may be in any of the following forms: only the server, only the terminal device, or both the server and the terminal device.

The system may be an information processing device on a cloud. The system may be a virtual information processing device and may be logically configured as one information processing device. The owner and the administrator of the system may be different persons.

The communication network 10 may be a mobile phone network, a wireless LAN, a landline telephone network, the Internet, an intranet, an Ethernet (registered trademark), and/or a combination thereof, and
The communication network 10 may communicate various data disclosed in the present application, but is not limited to these.

By executing a specific installed application, the server device 20 may be able to perform operations such as detecting information relating to one or more of a specified action by the broadcaster, the broadcaster's facial expression, the broadcaster's complexion, the broadcaster's voice, or a specified situation occurring to the broadcaster, determining whether the detected information satisfies specified conditions, and if it determines that the specified conditions are satisfied, generating information to change the facial expression or pose of the avatar relating to the broadcaster that was generated based on the facial expression or pose relating to the broadcaster to a specified facial expression or pose that corresponds to the specified conditions.

Alternatively, the terminal device 30 may be able to execute an installed web browser or a specific application to detect information relating to one or more of a predetermined action by the broadcaster, the broadcaster's facial expression, the broadcaster's complexion, the broadcaster's voice, or a predetermined situation occurring to the broadcaster, determine whether the detected information satisfies predetermined conditions, and if it determines that the predetermined conditions are satisfied, generate information to change the facial expression or pose of the avatar relating to the broadcaster that was generated based on the facial expression or pose relating to the broadcaster to a predetermined facial expression or pose that corresponds to the predetermined conditions.

The terminal device 30 is any terminal device capable of performing such operations, and may be, but is not limited to, a smartphone, a tablet, a mobile phone (feature phone), and/or a personal computer.

2. Hardware Configuration of Each Device Next, an example of the hardware configuration of the server device 20 will be described.

2-1. Hardware Configuration of Server Device 20 An example of the hardware configuration of the server device 20 will be described with reference to FIG.
2 is a block diagram showing an example of a hardware configuration of the server device 20 (terminal device 30) shown in FIG. 2 (note that in FIG. 2, the reference numerals in parentheses indicate the hardware configuration of each terminal device 30 as described later).
0.)

2, the server device 20 may mainly include a computing device 21, a main memory device 22, and an input/output interface device 23. The server device 20 may further include an input device 24 and an auxiliary output device 26. These devices may be connected to each other by a data bus and/or a control bus.

The arithmetic unit 21 performs calculations using instructions and data stored in the main memory 22, and stores the results of the calculations in the main memory 22.
The server device 20 can control an input device 24, an auxiliary storage device 25, an output device 26, etc. via an input/output interface device 23. The server device 20 may include one or more computing devices 21. The computing device 21 may include one or more of a central processing unit (CPU), a microprocessor,
and/or may include a graphics processing unit (GPU).

The main memory 22 has a storage function, and is connected to the input device 24, the auxiliary memory 25, and the communication network 10.
The main memory 22 stores commands and data received from the server device 20, etc. via the input/output interface device 23, as well as the results of the calculations performed by the calculation device 21.
This may include, but is not limited to, AM (random access memory), ROM (read only memory) and/or flash memory.

The auxiliary storage device 25 is a storage device. It may store instructions and data (computer programs) that configure the specific application, the web browser, etc., and is used by the computing device 21.
Under the control of the I/O interface device 23, these instructions and data (computer programs) may be loaded into the main memory device 22 via the I/O interface device 23. The auxiliary memory device 25 may be a magnetic disk device and/or an optical disk device, a file server, etc.
Not limited to these.

The input device 24 is a device for inputting data from the outside, and may be a touch panel, a button, a keyboard, a mouse, and/or a sensor, etc.

The output device 26 may include, but is not limited to, a display device, a touch panel, and/or a printer device, etc. Also, the input device 24 and the output device 26 may be integrated together.

In such a hardware configuration, the calculation device 21 sequentially loads instructions and data (computer program) constituting a specific application stored in the auxiliary storage device 25 into the main storage device 22, and calculates the loaded instructions and data, thereby controlling the output device 26 via the input/output interface device 23, and/or transmitting and receiving various information to and from other devices (e.g., a server device 20 and other terminal devices 30, etc.) via the input/output interface device 23 and the communication network 10.

By having such a configuration and executing a specific application installed, the server device 20 may detect information related to one or more of a predetermined action by a distributor, a facial expression of the distributor, a complexion of the distributor, a voice of the distributor, or a predetermined situation occurring to the distributor, as defined below, determine whether the detected information satisfies a predetermined condition, and if it is determined that the predetermined condition is satisfied, generate information for changing the facial expression or pose of the avatar related to the distributor, which was generated based on the facial expression or pose of the distributor, to a predetermined facial expression or pose corresponding to the predetermined condition (including various operations described in detail later). In addition, such operations may be performed by a user giving an instruction to a system of an example of the invention disclosed in the present application documents using the input device 24 or the input device 34 related to the terminal device 30 described later. In addition, when the program is executed on the calculation device 21, it may be displayed by the output device 26 of the server device 20 as a system used by the user, or may be configured to be displayed on the output device 36 of the terminal device 30 as a system used by the user.

2-2. Hardware Configuration of Terminal Device 30 An example of the hardware configuration of the terminal device 30 will be described with reference to Fig. 2. The hardware configuration of each terminal device 30 may be the same as that of each server device 20 described above. Therefore, the reference numerals for the components of each terminal device 30 are shown in parentheses in Fig. 2.

2, each terminal device 30 can mainly include a calculation device 31, a main memory device 32, an input/output interface device 33, an input device 34, an auxiliary memory device 35, and an output device 36. These devices are connected to each other by a data bus and/or a control bus.

The arithmetic device 31, the main memory device 32, the input/output interface device 33, the input device 34, the auxiliary memory device 35, and the output device 36 may be substantially the same as or different from the arithmetic device 21, the main memory device 22, the input/output interface device 23, the input device 24, the auxiliary memory device 25, and the output device 26 included in each of the above-mentioned server devices 20. The arithmetic device and the memory devices may have different capacities and capabilities. The input device 34 and the output device 36 in particular will be described in detail below.

In such a hardware configuration, the calculation device 31 sequentially loads instructions and data (computer programs) constituting a specific application stored in the auxiliary storage device 35 into the main storage device 32, and by calculating the loaded instructions and data, it is possible to control the output device 36 via the input/output interface device 33, and also to send and receive various information between the output device 36 and other devices (e.g., each server device 20, etc.) via the input/output interface device 33 and the communication network 10.

The terminal device 30 may be equipped with such a configuration, and may detect information related to one or more of a predetermined action by a distributor, the facial expression of the distributor, the complexion of the distributor, the voice of the distributor, or a predetermined situation occurring to the distributor, determine whether the detected information satisfies a predetermined condition, and if it is determined that the predetermined condition is satisfied, generate information for changing the facial expression or pose of the avatar related to the distributor, which was generated based on the facial expression or pose of the distributor, to a predetermined facial expression or pose corresponding to the predetermined condition (including various operations described in detail later). Also, by executing the installed web browser, it is possible to receive and display a web page from the server device 20 and perform the same operation. Also, such operations may be performed by a user giving an instruction to the system of the example of the invention disclosed in the present application document using the input device 34. Also, when the program is executed on the computing device 31, it may be configured to be displayed on the output device 36 of the terminal device 30 as a system used by the user.

In the present application, there may be two types of terminal devices, a "distributor terminal" and a "viewer terminal". These may be names given to the roles of the terminals, and a terminal device may be a viewer terminal, a distributor terminal, or a viewer terminal and a distributor terminal at the same time. Also, the distributor terminal and the viewer terminal may be different. The distributor terminal may be a terminal used by a distributor who appears as a subject such as an avatar in the distributed content. On the other hand, the viewer terminal may be a terminal used mainly by a viewer who views the distributed content. In addition, the viewer may also have a certain influence on the distributed content by a gifting act of providing a gift to a subject such as an avatar, such that a gift appears in the distributed content. Similarly, the viewer may have a certain influence on the distributed content in the form of a voice, a comment, or the like. The distributed content may be a video, a video, or an animation, and it is sufficient that a subject such as an avatar played by the distributor appears in some of these.
The above description is merely an example of application of one embodiment of the present invention, and each embodiment of the present invention may be applied to a virtual space in which a subject without such a premise appears. In the present application, animation may be virtual reality or augmented reality.

As described above, the viewer terminal may be configured to allow voice input, comment input, gifting, and the like. For example, the viewer terminal may have, as the input device 34, a keyboard, a touch panel, or the like that can input comments and acquire comments as text information. The viewer terminal may also have a function of generating certain comments and gifts by the computing device 34. These acquired comments and gifts may be similarly transmitted to the communication network 10 via the input/output interface device 23. These comments and gifts may also be used to generate items to be distributed.

The comment according to the present invention may be a comment automatically generated by the system, in addition to a comment inputted by a viewer terminal. Such a comment may be transmitted to the viewer terminal and displayed on the viewer terminal.

The terminal may also include a microphone capable of acquiring sound as the input device 34. Human voice may be acquired by the microphone as the input device 34 and treated as sound, or may be acquired by the microphone as the input device 34 and then converted into text by voice recognition by the calculation device 31 and treated as a comment. The viewer terminal may generate sound by the calculation device 31. The acquired sound or comment may be transmitted to the communication network 10 via the input/output interface device 23 in the same manner.

On the other hand, the broadcaster terminal may include a motion sensor capable of detecting human movement as the input device 34. The motion sensor may be attached to the broadcaster's body and have a function of detecting the movement of each part of the body. The parts of the body include the head, face, upper arm, forearm, hand, thigh, lower leg,
Various locations such as the feet, trunk, etc. may be mentioned, but are not limited to some or all of these, and the sensor may be attached to a position capable of detecting body movements. Such a sensor may track the facial expression and pose of the broadcaster and generate the information. The motion sensor may include an acceleration sensor, a gyro sensor, a geomagnetic sensor, etc. The acquired information may be transmitted to the communication network 10 via the input/output interface device 23 in the same manner.

The distributor terminal may also include a camera capable of capturing video as the input device 34. The camera may capture the head, face, eyes, part of the body, the whole body, etc., of the distributor.
When the head or face is imaged, it is possible to detect the direction of the face, when the face is imaged, it is possible to detect facial expressions, when the eyes are imaged, it is possible to detect the line of sight, and when a part of the body or the whole body is imaged, it is possible to detect the movement of the body together with or instead of the above-mentioned motion sensor. Information obtained about the direction of the face, the line of sight, the facial expressions, and the movement of the body based on these image information is similarly transmitted via the input/output interface device 23 to:
It may be transmitted to the communication network 10 .

Furthermore, the output device 36 in the viewer terminal and the distributor terminal may be a display device or a sound generating device. The display device may be any of a variety of devices capable of displaying information, including, but not limited to, a liquid crystal display, a plasma display, an organic EL display, a micro LED display, and a head mounted display. The sound generating device may be a speaker, a headphone, an earphone, etc. The information output by the output device 36 may be information generated by the arithmetic device 31,
The information may be received from the communication network 10. The distributor terminal may be a device in a studio equipped with a dedicated sensor for acquiring information about the performer. In this case, there is an advantage that information about the performer can be acquired more accurately. The distributor may also include the performer who plays the avatar.

3. Functions of Each Device Next, an example of functions of each of the server device 20 and the terminal device 30 will be described with reference to Fig. 3. Fig. 3 is a block diagram showing an example of functions of the system shown in Fig. 1.

3.1 Acquisition Unit The acquisition unit has a function of acquiring information that can be used to determine whether or not a predetermined condition, which will be described later, is applicable.

The information acquired by the acquisition unit may include information related to one or more of a predetermined action by the distributor, the facial expression of the distributor, the complexion of the distributor, the voice of the distributor, or a predetermined situation occurring to the distributor (sometimes referred to as a predetermined action, etc. in the present application documents). The information acquired by the acquisition unit may be information detected by an information processing device to which the system according to the present invention is connected, or information based on the detected information. The detected information may be information according to the detector that detects the information, and the detected information may be acquired by the acquisition unit as is, or information corresponding to the detected specific measurement information according to a predetermined threshold or range may be acquired by the acquisition unit. The detected information may be information on the movement of the motion data, information on the amount of alcohol from a breathalyzer, temperature information from a thermal sensor, etc. Then, information corresponding to these specific measurement information may be information on a specific facial expression or a specific movement from the motion data, information on the amount of alcohol consumed relative to the amount of alcohol, etc., which may be processed into information according to a predetermined condition according to the present invention. The condition application unit processes the information acquired by the acquisition unit as follows:
The specific contents will be described below. Note that Fig. 4 shows an example of part of the information acquired by these acquisition units.

Here, the detection method may be various methods. For example, the detected information may include information detected by image processing using an image captured by one or more imaging devices that capture the broadcaster. The information acquired by such image processing may detect, for example, a blend shape based on motion data. In addition, sound may be detected by a microphone that captures sound in a room where the broadcaster is present, or by detecting the operation of a sound generating device operated by the broadcaster. In addition, the movement of the hand or arm may be detected using the movement of a sensor attached to the broadcaster's hand or arm, or the movement of the throat may be detected using a sensor wrapped around the throat to detect eating and drinking actions. In addition, the broadcaster's drinking action may be detected by an alcohol detector. The system according to the present application may be connected to these detectors and detect information about the broadcaster's actions, etc. In addition, the broadcaster may include an actor who can operate an avatar.

Figure 5 shows examples of detection methods and examples of detected information.

The information processing device from which the acquisition unit acquires information may be the same as or different from the information processing device having the function of the acquisition unit. The term "acquire" may be a higher-level concept of receiving information from another information processing device, reading information from a storage device, receiving information from another program via a storage device, or detecting information by an information processing device.

The condition application unit has a function of determining whether the detected information satisfies a predetermined condition, and when it is determined that the predetermined condition is satisfied, generating information for changing the facial expression or pose of the avatar related to the distributor, which has been generated based on the facial expression or pose related to the distributor, to a predetermined facial expression or pose corresponding to the predetermined condition.

Here, generating information for changing the facial expression or pose of the avatar of the distributor, which has been generated based on the facial expression or pose of the distributor, to a predetermined facial expression or pose corresponding to the predetermined condition may mean generating only information for changing the facial expression of the avatar of the distributor, which has been generated based on the facial expression of the distributor, to a predetermined facial expression corresponding to the predetermined condition, when the predetermined condition is satisfied, or generating only information for changing the pose of the avatar of the distributor, which has been generated based on the pose of the distributor, to a predetermined pose corresponding to the predetermined condition, or generating information for changing the facial expression and pose of the avatar of the distributor, which have been generated based on the facial expression and pose of the distributor, to a predetermined facial expression and pose corresponding to the predetermined condition. In the present application, the predetermined facial expression and/or pose specified in response to the predetermined condition may be referred to as a predetermined facial expression, etc., and the facial expression and/or pose may be referred to as a facial expression, etc.

In addition, the predetermined facial expression may be an expression displayed using predetermined parameters of a facial expression, or a parameter of a facial expression based on the broadcaster's facial expression corrected based on a predetermined rule, or an expression displayed by setting a predetermined shape or movement, or a pose displayed using predetermined pose parameters, or a parameter of a pose based on the broadcaster's pose corrected based on a predetermined rule, or a pose displayed by setting a predetermined shape or movement. Here, the predetermined parameters of a facial expression may indicate a plurality of parameters that affect the movement of each part of the face, and one facial expression may be displayed by setting each of the plurality of parameters to a predetermined value. Such one or more parameters may be parameters that affect the movement of each part of the face, and each parameter may be for a specific part of the face, or may be for the entire face (or a part of the face) but the shape of the entire face (or a part of the face) may differ for each parameter. The latter is a technique in which a plurality of objects with the same area but different shapes are prepared, and one facial expression is expressed by combining a part or a plurality of them. An example of the latter is, for example, a blend shape,
In this technology, a plurality of different overall facial shapes are prepared in advance, and a single facial expression is generated by synthesizing the plurality of shapes. In this technology, a special facial expression may be made by preparing a predetermined shape in advance in addition to normal facial elements. For example, if a shape showing vertical lines on the face is prepared in advance, such a vertical line on the face can be produced, and if an element for producing a cross-eyed expression is prepared in advance, such a cross-eyed expression can be produced. In this case, since the parameters are set in advance, the facial expression is reproduced regardless of the broadcaster's facial expression, etc., and there is an advantage that an easy-to-understand facial expression can always be reproduced when appropriate parameters indicating the facial expression are selected. In this case, the predetermined facial expression may be set differently depending on the avatar. That is, the predetermined facial expression applied when the same predetermined condition is satisfied for avatar A and avatar B may be different for avatar A and avatar B. In this case, it may be set for each avatar, or it may be set according to the type of avatar.

On the other hand, if the predetermined facial expression is a parameter of a facial expression based on the facial expression of the distributor, the predetermined facial expression may be a facial expression based on the facial expression of the distributor. For example, assume that the distributor has a surprised expression at some gift. In this case, the facial expression of the avatar is set to a facial expression corresponding to surprise by one or more parameters in response to the detection of the facial expression of the distributor, but the value of the multiple parameters related to surprise is increased or decreased by a predetermined value as a predetermined facial expression corresponding to a predetermined condition, so that the surprised facial expression is emphasized. While the above-mentioned technology is absolute in that one or more parameters are determined as a predetermined facial expression, the present technology may use a parameter relative to information obtained from the current facial expression of the distributor. Therefore, even when the same avatar played by the same distributor is applied with the same predetermined conditions, the parameters of the facial expression based on the facial expression of the distributor at that time may be the same or different, so that the facial expression according to such different parameters may become the predetermined facial expression. Therefore, it is possible to provide more variation in facial expressions.

Furthermore, the facial expression displayed by setting a predetermined shape or movement may be one in which a predetermined shape, such as a mask, is itself the specified facial expression.

Although the above describes predetermined facial expressions, the corresponding techniques may be used for predetermined poses as well.

Furthermore, the predetermined facial expression, etc. may be a facial expression, etc. associated with a predetermined action, etc. to be performed. In this case, there is an advantage that the viewer can easily understand the reason why the predetermined facial expression, etc. is expressed by associating it with the predetermined action, etc.

Furthermore, the predetermined facial expression, etc. may act on parameters for all parts constituting the facial expression, etc., or may act on some of the parameters. For example, the predetermined facial expression, etc. may act only on parameters for the mouth, and parameters for parts other than the mouth may be parameters linked to the facial expression of the distributor and based on information corresponding to the movement of the distributor's facial expression from the distributor terminal. The predetermined facial expression, etc. may also include the appearance of the avatar. For example, the predetermined facial expression, etc. may include a performance in which the avatar is charred. Note that, examples of parameters may include parameters indicating the degree of opening of the eyes and mouth (for example, numerical values indicating 0% to 100%), parameters indicating the left and right positions and up and down positions of both ends of the mouth, and parameters indicating the up and down positions of the eyebrows.

The predetermined pose may be a variety of poses. The pose may be only a part of the avatar's body or the entire avatar's body. For example, a pose of only the head as a head tilt pose, a pose of only the hands including the effect of the hands being enlarged or red and swollen,
It can be a pose of just the upper body, such as a fist pump, or a pose of the whole body, such as holding a gun or jumping.

The specified condition may use information relating to one or more of a specified action of an avatar performed by the broadcaster, a facial expression of the avatar, a complexion of the avatar, a voice of the avatar, or a specified situation occurring regarding the avatar.

The predetermined action of the broadcaster may be a predetermined action that is perceived by the broadcaster with the five senses. The predetermined action of the broadcaster may include an action of looking at a predetermined object. For example, when a broadcaster is surprised to see a predetermined object such as another broadcaster or a predetermined object in a place such as a stage or a room where the broadcaster is present, a surprised expression may be set as the predetermined expression.

In addition, the predetermined object is not limited to an object that brings about a feeling of surprise to the distributor, but may be an object that brings about other feelings or sensations, for example, an object that brings about a feeling of fear, an object that brings about a feeling of like such as steak, an object that brings about a feeling of dislike such as natto, or the like, which may be an object that causes a certain feeling or sensation when a person sees the object. In addition, as for the predetermined facial expression when seeing such an object, a predetermined facial expression showing a face that a person generally makes when seeing such an object may be set. The above-mentioned objects that bring about a feeling of fear, an object that brings about a feeling of like, and an object that brings about a feeling of dislike may correspond to a facial expression of horror, a smile, a facial expression of disgust, and the like.

Furthermore, when the predetermined target is another broadcaster, the action of looking at the predetermined target may include looking at the gaze of the one broadcaster and the eyes of the other broadcaster. In this case, when the other broadcaster also looks at the eyes of the one broadcaster, the gaze of the one broadcaster and the other broadcaster will meet. In such a case, the avatar may be set to a predetermined facial expression, etc. In addition, when the gazes of the broadcasters meet, a predetermined facial expression, etc. may be set only for one avatar, or a predetermined facial expression, etc. may be set for both avatars. In the latter case, for example, an expression in which the gazes of both avatars meet and the faces of both avatars turn red or an expression of excitement may be considered. In addition, in the latter case where a predetermined facial expression, etc. is set for both avatars, the set facial expression, etc. may be the same predetermined facial expression, etc., or may be different predetermined facial expressions, etc. The reason for such different predetermined facial expressions, etc. may be a predetermined facial expression, etc. according to the type to which the avatar belongs, a predetermined facial expression, etc. according to the preferences or characteristics of the avatar, or a predetermined facial expression, etc. selected probabilistically.

In this case, the broadcaster is captured by an imaging device with an imaging field of view that includes the broadcaster and its surroundings, and when the distance between the broadcaster and other broadcasters in the captured image becomes equal to or less than a predetermined distance, or when the broadcaster looks at other broadcasters, a predetermined facial expression or the like may be set. When the latter broadcaster looks at other broadcasters, the broadcaster's line of sight is changed to Ra as an image processing in the image.
The direction of the streamer's line of sight may be calculated by using a command such as ycast, and if the line of sight intersects with another streamer, it may be determined that the streamer is looking at the other streamer.
Similarly, when the distance between the broadcaster and an object becomes less than a predetermined distance, or when the broadcaster sees such an object, a predetermined facial expression or the like may be set. In addition, the predetermined action of the broadcaster may include an action of touching a predetermined object. The predetermined object may be determined in advance, and such a touching action may be
As described above, the broadcaster is imaged using an imaging device whose imaging field of view includes the broadcaster and its surroundings, and when the distance between the broadcaster and a specified object in the captured image becomes less than a specified distance, or when the distance between the broadcaster and a specified object becomes zero, a specified facial expression, etc. can be set.

In this regard, depending on the position of the imaging device that captures the broadcaster, the timing when the broadcaster comes into contact with the specified object and the timing when the broadcaster's outline and the outline of the specified object come into contact in the image may be different. In particular, in the latter case, even if the broadcaster himself does not come into contact with the specified object, the outline of the broadcaster and the outline of the specified object may overlap in the image. Therefore, in addition to or instead of the determination using image processing, a sensor worn by the broadcaster may be used. For example, the broadcaster may wear a glove with a sensor, and the sensor in the glove may detect the contact between the glove and the specified object. In addition, a sensor attached to the body of the broadcaster may detect a situation in which the broadcaster comes into contact with another broadcaster or comes into contact with another object. In addition, the sensor attached to the body of the broadcaster may be a marker-type tracker. In addition, by attaching one or more trackers to the object, the relationship between the broadcaster and the object, such as contact, may be determined based on the relationship between the tracker attached to the object and the tracker attached to the broadcaster's body.

The predetermined action of the distributor may include a handshake action. The handshake may be determined by detecting finger movements using a glove with a sensor worn by the distributor, and when the finger movements of the handshake are detected, the handshake may be determined to be occurring.

The predetermined action of the distributor may include an eating and drinking action of a predetermined target.
A sensor wrapped around the streamer's throat may be used to detect throat movement, and thus it may be possible to detect that the streamer is eating or drinking. A predetermined action or the like when the streamer eats or drinks may be an action of the streamer's avatar moving its mouth with food in it. For example, it may be an expression of munching. Such an expression while eating or drinking may correspond to the movement of the mouth, and such mouth movement may correspond to the above-mentioned throat movement. Furthermore, in addition to or instead of the above-mentioned technology, a predetermined action or the like may be set based on information in an image captured of the streamer. For example, it may be determined that the streamer is eating a predetermined thing based on information in the image, and an expression showing the corresponding way of eating the predetermined thing may be used. For example, when the streamer eats ramen,
When eating curry, meat, fish, vegetables, etc., the corresponding food items (ramen, curry, meat, fish, vegetables, etc.) may be displayed as items or animations in the virtual space, while the eating manner corresponding to the food may be a predetermined facial expression, etc. For example, when eating meat, the eating manner may be biting into meat on the bone from the side, and for curry, the eating manner may be eating with a spoon, etc. Also, an item may be generated to display in the virtual space that corresponds to the predetermined object. In the case of a facial expression for eating food, the corresponding food item described above may be generated and displayed.

In the above description, food or drink items corresponding to eating and drinking actions are described as items displayed as effects in the virtual space, but the newly displayed items are not limited to these. For example, an item corresponding to an object perceived by the broadcaster in the real space may be displayed as an item to be performed in the virtual space. For example, when the sound of a musical instrument is heard, the musical instrument may be displayed as an item. Also, when a tub or a ball comes into contact with the broadcaster, the corresponding tub or ball item may be displayed on the screen as an effect.

Furthermore, the items displayed as such effects may be displayed in response to the timing when the avatar related to the distributor is set to a predetermined facial expression, etc. Furthermore, when the predetermined facial expression, etc. set for the avatar related to the distributor reverts to the facial expression, etc. of the avatar corresponding to the original distributor, the items displayed as such effects may disappear from the screen. The items displayed as such effects may be displayed in response to the time when the avatar related to the distributor is set to a predetermined facial expression, etc.

Furthermore, the position of the items displayed as such a presentation may be determined and moved in relation to a specific action of the broadcaster. For example, in a specific action of the broadcaster, the position, movement, and size of the corresponding item may differ depending on the strength of the sound, the speed of the action, the distance of the action, and the like. More specifically, for example, if the sound is strong, the item may be closer, and if the sound is weak, the item may be farther away. Also, for example, if the action of eating is fast, the food item may be reduced quickly. Also,
When a ball is thrown, a ball item corresponding to the ball may be displayed in the virtual space, and the speed and size of the ball in the virtual space may be displayed in correspondence with the speed and size of the flying ball in the real space. In this case, the speed of the ball in the real space may be made to correspond to the speed of the ball in the virtual space, and the size of the ball in the real space may be made to correspond to the size of the ball in the virtual space, or the speed and the ball may be made to correspond alternately.

In addition, when the predetermined behavior of the distributor is drinking alcohol, the predetermined facial expression may be, for example, a facial expression of the distributor under the influence of alcohol, such as by turning the face of the distributor red. Drinking may be determined by an alcohol breathalyzer. In addition, the facial color of the avatar may be changed according to the amount of alcohol detected by the alcohol breathalyzer. For example, when the amount of alcohol is detected to be within a first predetermined range, the facial color of the avatar may be set to pink, when the amount of alcohol is detected to be within a second predetermined range that is greater than the first predetermined range, the facial color of the avatar may be set to red, and when the amount of alcohol is detected to be within a third predetermined range that is greater than the second predetermined range, the facial color of the avatar may be set to dark red.

The predetermined behavior of the distributor may include smelling or a predetermined smelling gesture. The predetermined smelling gesture may include a behavior related to the nose. For example, it may include a gesture that indicates that a general person can smell, such as a so-called sniffing or twitching of the nose. The predetermined smelling gesture may also include a gesture that indicates that a general person can smell, such as a behavior of trying to cover the nostrils, a behavior of stopping breathing, or a behavior of moving air in front of the nose with the hands. The predetermined smelling gesture may also be only a movement of the nose and face, only a movement of the hands, or both. Such behaviors include:
It may be a facial expression, etc.

The broadcaster's smelling or smelling gesture may be based on motion data, and nose information may be obtained based on blend shapes. Information may also be obtained using an optical sensor or image processing.

In addition, as a predetermined facial expression of the avatar corresponding to a predetermined action of the distributor, the avatar related to the distributor may have a facial expression of smelling or smelling in response to a smelling or smelling action by the distributor.
Similarly, the avatar's smelling or smelling expression as a predetermined expression may be a predetermined smelling or smelling motion that an ordinary person feels is acting like they are smelling or smelling. Such a smelling or smelling expression by an avatar may be the same type of expression as the smelling or smelling expression by the distributor, or may be a different type of expression. In addition, if the expressions are the same type, the smelling or smelling expression by the avatar may be made clear to be a smelling or smelling expression by making the range of movement of the smelling or smelling expression by the distributor larger, or by making the speed of the movement clearer.

The predetermined action of the distributor may include a predetermined gesture for listening to/listening to a sound. The predetermined gesture for listening to/listening to a sound may be an ear movement only, a head movement only, a hand movement only, or some or all of these movements. An example of an ear movement is a movement of moving the ears. An example of a head movement is a movement of moving the ears.
An example of the motion is to turn the ear to a certain direction. The certain direction may be the direction in which the broadcaster feels that the sound can be heard. The hand motion may include a motion of bringing the hand close to a certain range of the ear. The hand motion may include a motion of turning the palm toward the ear, and may be in a form that collects only sounds coming from a specific direction. The motion may be a motion of only one hand, or may be a motion of both hands. When the palm is placed in a form that collects only sounds coming from a specific direction for one ear, the body or head may be turned so that the ear in the specific direction approaches. The broadcaster's motion may include a gesture that an ordinary person recognizes as a motion of listening to/attempting to listen to a sound, as described above. Such a motion may be a facial expression, etc. In response to such a facial expression, etc. of the broadcaster, the predetermined facial expression, etc. for the avatar related to the broadcaster may be a facial expression, etc. that an ordinary person recognizes as a facial expression, etc., of listening to/attempting to listen to a sound, as described above. The facial expression, etc. of the avatar listening to/attempting to listen to the sound may be the same as or different from the facial expression, etc. of the broadcaster listening to/attempting to listen to the sound. In addition, if the facial expressions are the same, the facial expression, etc. of the avatar may be different from the facial expression, etc. of the broadcaster listening to/attempting to listen to the sound.
By increasing the range of movement and clearly varying the speed of movement, it is possible to make it clear that the facial expression is one of listening to a sound/attempting to hear.

Furthermore, the predetermined situation occurring to the distributor may include the occurrence of a predetermined sound.
The predetermined sound may be a human voice, a natural sound, a sound produced by a musical instrument, an artificial sound, or a dramatic sound, and there is no limit to the type of sound. The human voice may be various voices emitted by humans, such as a normal human voice, an unusual human voice, a strange human voice, a human scream, etc. The natural sound may be various sounds such as the sound of wind, the sound of water, the sound of a fire, the sound of a typhoon, the sound of a flood, etc. The musical instrument sound may be various sounds such as a violin, a piano, a flute, etc. The artificial sound may be various sounds such as a ping-pong sound effect indicating a correct answer to a question, or a buzzing sound effect indicating an incorrect answer. The dramatic sound may be, for example, a slurping sound that is commonly used when a ghost appears and allows ordinary people to recognize the appearance of a ghost.

The predetermined sound may be detected by various methods. For example, a sound that can be heard by the distributor may be detected by a microphone installed in the room or studio where the distributor is present, or information from a sound generating device that can play a sound to the distributor, such as a sound generated by an instruction from an operator in the studio or an instruction input by the distributor himself, may be detected.

In response to the generation of such a predetermined sound, the predetermined facial expression, etc. of the avatar related to the distributor may be a predetermined facial expression, etc. corresponding to such sound. For example, when a strange voice or a scream is detected, the avatar may be set to an expression of a strange voice or a scream as the predetermined facial expression, etc. Specifically, the facial expression may be set to an expression that can be recognized as an ordinary person screaming or making a strange voice, such as opening the mouth up and down and moving the eyes and eyebrows up and down. Also, when a sound effect indicating the correct answer, such as a ping-pong sound, is detected, the predetermined facial expression, etc. of the avatar may be an expression of an idea. In this case,
As a further effect in the virtual space, an exclamation mark may be displayed around the avatar's head within a predetermined area to indicate that the avatar has had an idea. In response to a slurping sound that would allow an average person to recognize the appearance of a ghost, the avatar may make a frightened expression as a predetermined facial expression, etc. When the sound of a musical instrument is detected, the avatar may make a singing expression as a predetermined facial expression, etc. In this case, the avatar's mouth may move in time with the sound of the musical instrument,
For example, the mouth of the avatar may move in accordance with changes in the sound of the instrument. Furthermore, a predetermined facial expression, etc. of the avatar related to the distributor may be set according to the volume. For example, when a sound with a volume louder than a predetermined volume is detected, a pose with the ears covered may be set as the predetermined facial expression, etc. of the avatar related to the distributor. Furthermore, when a sound with a volume quieter than the predetermined volume is detected, a pose with the hand placed over the ear may be set as the predetermined facial expression, etc. of the avatar related to the distributor, with the distance between the ear and the hand shorter than the predetermined distance, so that the hand is close enough to touch the ear. Furthermore,
A predetermined facial expression, etc. of the avatar of the distributor may be set according to the sound quality of the sound. For example, when a hoarse voice set as the predetermined sound quality is detected, a stern expression may be set as the predetermined facial expression, etc. of the avatar of the distributor. Also, when a falsetto set as the predetermined sound quality is detected, an intoxicated expression may be set as the predetermined facial expression, etc. of the avatar of the distributor.

In addition, the predetermined situation occurring to the distributor may include a predetermined object coming into contact with the distributor. The predetermined object may be various objects. For example, it may be a ball, a bat, a basin, or the like. However, when an object actually comes into contact with the distributor, for safety reasons, it may not be a real object, but may be an imitation of such a real object. When a predetermined object comes into contact, a corresponding facial expression may be displayed as a predetermined facial expression of the avatar related to the distributor. For example, the contact may result in a surprised facial expression or a painful facial expression. In this case, as a representation in the virtual space, a representation generally indicating pain, such as a star mark, may be displayed around a predetermined range from the head of the avatar to indicate that the avatar is in pain.

The face color of the distributor may include a change in face color. The change in face color may be determined based on image processing of the distributor, as described above. The change in face color may be measured using the temperature of the distributor's face using a thermal sensor. The change in face color may be used in combination with other information to set a predetermined facial expression, etc. For example, the degree of drinking may be detected using the change in face color and information on detected alcohol. More specifically, a predetermined facial expression, etc. related to the avatar may be set in which the face color of the distributor changes from a normal skin color to a color such as pink or red, and the degree of the detected amount of alcohol. Such a predetermined facial expression, etc. of the avatar may be an expression of intoxication due to alcohol, as described above. The expression of intoxication due to alcohol may include a staggering movement of the avatar's legs and body. A scared facial expression may be set as a predetermined facial expression, etc. related to the avatar, using a change in face color from a normal skin color to a pale or white color and a predetermined item such as a ghost character.

The voice of the distributor may include a change in the quality of the voice of the distributor beyond a predetermined range, or a change in the volume of the voice of the distributor beyond a predetermined amount. The quality of the voice and the volume of the voice may be determined using information detected from a microphone. When there is a certain change in the voice of the distributor, a predetermined facial expression, etc. corresponding to such change may be set. Such information may also be detected in combination with other information. For example, the predetermined facial expression, etc. of the avatar may be set using the quality and/or volume of the voice of the distributor and information based on an image obtained by an imaging device that images the distributor. More specifically, for example, when the volume of the voice of the distributor is detected to be higher than a predetermined volume and it is determined that the distributor's hands and mouth are within a predetermined distance such that the distributor places his/her hands on his/her mouth, the predetermined facial expression, etc. of the avatar related to the distributor may be made to be red in the face and have the eyes closed. In this case, a megaphone may further be displayed, and/or a megaphone may be displayed.
Alternatively, an illustration like thunder may be displayed to indicate the volume of the voice. Here, the size of the illustration like thunder may correspond to the volume of the voice. Also, for example, when the voice of the distributor is detected to be lower than a predetermined tone quality and the distance between the eyebrows of the distributor is determined to be shorter than a predetermined length, a mark indicating anger may be attached to the face of the avatar of the distributor as a predetermined facial expression, etc. of the avatar of the distributor. Here, the distance between the eyebrows being shorter than a predetermined length may be a state in which the eyebrows are wrinkled. Also, when it is detected that the distributor has uttered a specific keyword such as "transform" and it is determined that the distributor has taken a predetermined pose, a predetermined transformation animation may be displayed as a predetermined facial expression, etc. of the avatar of the distributor. For example, when the distributor moves one hand in a circular motion in front of the body and simultaneously utters a voice such as "transform", for example, a Kamen Rider transformation animation may be activated and displayed as a predetermined facial expression, etc. of the avatar of the distributor.

The detected information may include information detected as a result of input by a third party.
For example, when broadcasting in a studio, information input by a third party other than the director may be detected. For example, the sound of a ping pong that is heard when the director answers a question correctly may be information detected by detecting input information by a third party who inputs the ping pong sound, in addition to or instead of detecting the sound itself. In addition, as for information other than sound, for example, a third party may input information indicating contact when a director comes into contact with another person or object, including another director, or a third party may input information corresponding to a director's eating or drinking action, such as eating or drinking food, or a third party may input information about the generation of a smell or odor. In this way, information that a broadcaster can recognize with the five senses may be information detected as input by the broadcaster himself or a third party.

In addition, the predetermined condition may be satisfied in a plurality of cases. For example, when one broadcaster sees another broadcaster and at the same time a third broadcaster contacts the first broadcaster, etc. In such a case, when the predetermined facial expressions, etc. corresponding to the conditions of seeing the other broadcaster and being within a predetermined distance range from the third broadcaster are different, processing may be defined when the plurality of predetermined conditions are satisfied. For example, when a first predetermined condition is satisfied and a second predetermined condition is satisfied within a predetermined time, a predetermined facial expression or pose corresponding to the case where the predetermined condition with a higher priority is satisfied among the first predetermined condition or the second predetermined condition may be applied. Here, the priority may be stored in association with each predetermined condition, or may be set as a rule when a plurality of predetermined conditions are satisfied. For example, when there are predetermined conditions such as A1, A2, A3, A4, etc., it is sufficient to be able to select a predetermined condition with priority after comparing each of the predetermined conditions. Such a function using the priority has the advantage of being able to process appropriately even when a plurality of predetermined conditions are satisfied, especially when a plurality of predetermined conditions are satisfied almost simultaneously. Moreover, the priority may be expressed by a probability. For example, the probability may be expressed as follows:
In the case where the determination is made by a pseudorandom function or the like, and the first and second predetermined conditions are satisfied, if the value of the pseudorandom function is equal to or less than a predetermined value, a predetermined facial expression or the like corresponding to the first predetermined condition is applied, and if the value exceeds the predetermined value, a predetermined facial expression or the like corresponding to the second predetermined condition is applied. In this case, the variety of presentations increases, which may further attract the interest of viewers.

In addition, since the judgment regarding the multiple conditions that can be satisfied is made after waiting for the passage of a predetermined time, if the predetermined time is set to be a certain time or more, there is a possibility that a time lag will occur between the satisfaction of the predetermined condition and the presentation of the predetermined facial expression, etc. Therefore, without setting such a priority, when the multiple conditions described above are satisfied, a predetermined facial expression, etc. corresponding to the predetermined condition that was satisfied first in time may be set. In this case, "first in time" may mean that the processing unit judges that the predetermined condition has been satisfied first, even if the predetermined conditions are satisfied substantially at almost the same time as a result of a network or information processing, etc. In this case, there is an advantage that the predetermined facial expression, etc. corresponding to the predetermined condition that was satisfied first can be set without waiting for the passage of a predetermined time. In addition,
In such a case, a predetermined facial expression, etc. corresponding to the later-satisfied predetermined condition may or may not be set. If a predetermined facial expression, etc. corresponding to the earlier-satisfied predetermined condition is overwritten, but there is an advantage that the predetermined facial expression, etc. corresponding to the later-satisfied predetermined condition can also be presented to the viewer. On the other hand, if a predetermined facial expression, etc. is not set, there is an advantage that the viewer can fully enjoy the earlier predetermined facial expression, etc. In this way, the first and second predetermined conditions can be overwritten.
When both of the above predetermined conditions are satisfied, a predetermined facial expression, etc. may be set according to the timing when the first predetermined condition is satisfied and the timing when the second predetermined condition is satisfied. For example, when the time between the first timing when the first predetermined condition is satisfied and the second timing when the second predetermined condition is satisfied is shorter than a predetermined time, only the first predetermined facial expression, etc. corresponding to the first predetermined condition may be produced, and the second predetermined facial expression, etc. corresponding to the second predetermined condition may not be produced. In this case, with regard to the production of the first predetermined facial expression, etc., the viewer may
On the other hand, if the time between the first timing and the second timing is longer than a predetermined time, a predetermined facial expression or the like corresponding to the second predetermined condition may be set at the timing when the second predetermined condition is satisfied.
In this case, there is an advantage that the viewer can enjoy both the first predetermined facial expression and the second predetermined facial expression.

The above-mentioned predetermined time for determining the time between the first timing and the second timing may be set according to a set of predetermined conditions that are satisfied. For example, it is assumed that a first predetermined facial expression or the like is produced when a first predetermined condition is satisfied, a second predetermined facial expression or the like is produced when a second predetermined condition is satisfied, and a third predetermined facial expression or the like is produced when a third predetermined condition is satisfied. Also, if the first predetermined condition is satisfied at the first timing,
Assume that the second predetermined condition is satisfied at the second timing, and the third predetermined condition is satisfied at the third timing. At this time, if the time between the first timing and the second timing is shorter than the twelfth predetermined time, the second predetermined facial expression, etc. is not produced, and if it is equal to or longer than the twelfth predetermined time, the second predetermined facial expression, etc. is produced. Also, if the time between the first timing and the third timing is shorter than the thirteenth predetermined time, the third predetermined facial expression, etc. is not produced, and if it is equal to or longer than the thirteenth predetermined time, the third predetermined facial expression, etc. is produced. At this time, the twelfth predetermined time and the thirteenth predetermined time may be the same or different. If they are the same, it means that the production time of the first predetermined facial expression, etc. remains the same whether the second predetermined condition is satisfied or the third predetermined condition is satisfied after the first predetermined condition is satisfied. On the other hand, if they are different, the presentation time of the first predetermined facial expression, etc. will change when the second predetermined condition is satisfied and when the third predetermined condition is satisfied after the first predetermined condition is satisfied. In this case, the variety of presentations will increase, which may attract more viewers' interest.

In the above, priority is associated with a specified condition, or examples of rules regarding the specified condition being satisfied are described. However, priority may also be associated with an object, event, situation, etc. within the specified condition.

An object, event, or situation related to a specified condition (sometimes referred to as an object, etc. in this application)
The priority of the above is determined based on the size of the object, the location of the object, the timing of the object's appearance,
The image may be generated using the above.

For example, regarding the displayed size of an object, if a second object is larger than a first object, a higher priority may be set for the second object than for the first object. In this case, the larger object attracts more attention from the viewer, so giving priority to the larger object is considered to be in line with the viewer's wishes.
On the other hand, the opposite may be true. In this case, the advantage is that it creates an element of surprise.

In addition to the size of the object, the price may be used if the object has a price, or the consumption points may be used if the object has consumption points. For example, if the price of the second object is higher than the price of the first object, a higher priority may be set for the second object than the first object. In this case, the higher the price, the more attention the viewer will pay, so it is considered that giving priority to the object is in line with the viewer's intention. On the other hand, the opposite may be set. In the opposite setting, there is an advantage in that it creates a sense of surprise. Similarly, for example, if the consumption points of the second object are higher than the consumption points of the first object, the second object may be set to a higher priority than the first object. In this case, the higher the price, the more attention the viewer will pay, so it is considered that giving priority to the object is in line with the viewer's intention. On the other hand, the opposite may be set. In the opposite setting, there is an advantage in that it creates a sense of surprise.

The location of an object in the virtual space may be, for example, an object that appears in the center of the virtual space.
A higher priority may be set for objects that appear at the left or right edge of the virtual space. When the priority of an object that appears in a conspicuous place in the virtual space is set high, the viewer can include the object that appears in the conspicuous place in his or her field of vision, which has the advantage that the viewer can easily understand the meaning of a certain facial expression.

In addition, the priority of the object may be static information that is set in advance for the object, or may be dynamic information that is set only when the object is provided. The latter may be information that is set according to the position where the object appears. For example, the priority may be set according to the relationship between the position where the object appears in the virtual space and the position of the avatar. In this case, the distance itself may play the role of the priority, or the distance may be used as one piece of information for setting the priority. For example, in a case where the distance between the avatar and the position related to the first object that appeared is a first distance, and the distance between the avatar and the position related to the second object that appeared is a second distance, if the first distance is longer than the second distance, the priority of the first object may be set higher than the second object. In addition, if the first distance between the center of the screen and the position related to the first object is shorter than the second distance between the center of the screen and the position related to the second object, the priority of the first object may be set higher than the second object since the first object appears closer to the center of the screen. When the priority of the object or the like is dynamically set, there is an advantage that a gift to which a predetermined expression or the like is set can be selected more appropriately and dynamically.

In addition, when a predetermined condition is satisfied, the predetermined facial expression or the like to be applied may be the type to which the avatar to which the predetermined facial expression or the like is applied belongs. For example, the predetermined condition is
A type to which the avatar belongs, which is one of a plurality of types to which the avatar belongs, may be used. This may be such that, when certain information is acquired, a predetermined facial expression, etc. is obtained in a first avatar type, but a predetermined facial expression, etc. is not obtained in a second avatar type. For example, a certain type of avatar may be set to have a timid personality, and when the distance from other broadcasters is within a first distance range, a surprised facial expression may be easily produced. Also, a certain type of avatar may be set to have a cheerful personality, and the condition for a facial expression including a smile as a predetermined facial expression, etc. may be set to be looser than other avatars. In this case, the avatar may be provided with a personality characteristic such as timid, cheerful, bullish, or competitive, and the satisfaction of a predetermined condition may be determined using the personality characteristic. The personality may be set according to the type to which the avatar belongs, or may be set for each avatar separately from the type to which the avatar belongs. Also, such an avatar type may be a personality set when the avatar is set, or may be determined from the parts or clothes equipped on the avatar at the time when the predetermined condition is determined.

The predetermined condition may include a probability. That is, the predetermined condition may include a pseudorandom number function, etc., and a predetermined facial expression, etc. may be generated when it is determined that the function value satisfies the predetermined condition. When such a probability is included, the predetermined facial expression, etc. is not always generated, so that the viewer's interest can be continuously attracted. Also, for example, when the probability value is lower than a predetermined value and a rare facial expression is generated, the viewer's interest can be attracted in that the viewer can see a valuable facial expression.

The condition application unit may set the facial expression, etc., specified by the satisfaction of a predetermined condition for the distributor, as the facial expression, etc., for the avatar played by the distributor. More specifically, the information indicating the facial expression, etc., specified by the satisfaction of a predetermined condition may be an ID indicating the specified facial expression, etc., or one or more parameters of the facial expression indicating the specified facial expression, etc., and the specified facial expression, etc. may be
The avatar's facial expression, etc. may be set using information indicating the specified facial expression, etc., so that the facial expression, etc. of the avatar can be set to such an avatar's facial expression. After such ID and parameter information is specified or generated by the condition application unit, such information may be transmitted to a server or terminal for generating video or animation, and may be used for a specified period of time. Here, the condition application unit may store information indicating the specified facial expression, etc., in association with each specified condition, and may be configured to be able to specify information indicating the corresponding specified facial expression, etc., when each specified condition is satisfied. Note that,
After a predetermined facial expression or the like that is applied when a predetermined condition is satisfied is set for the avatar for a predetermined period of time, the avatar may return to its original behavior. That is, since the avatar is performed by the distributor, after a predetermined period of time, the avatar's facial expression and pose may be based on the information on the distributor's facial expression and pose. Here, based on the information on the distributor's facial expression and pose may mean that the avatar's behavior is determined by information on the distributor's facial expression and pose detected by an optical sensor or an inertial sensor.

When a first information processing device on which the condition application unit is executed is the same as a second information processing device which performs processing including setting information indicating the facial expressions, etc. of the avatar, the condition application unit may process the information indicating the facial expressions, etc. of the avatar using the facial expressions identified by the condition application unit upon satisfaction of specified conditions.

In addition, when a first information processing device on which the condition application unit is executed is different from a second information processing device which performs processing including setting information indicating the facial expressions, etc. of the avatar, the first information processing device may transmit information indicating the facial expressions, etc. identified by the condition application unit by satisfying a specified condition to the second information processing device, and the second information processing device may process the information indicating the facial expressions, etc. of the avatar using the information indicating the identified facial expressions, etc. to set the information indicating the facial expressions, etc. of the avatar.

Here, when the first information processing device is a server and the second information processing device is a server, information indicating the specified facial expression, etc. may be transmitted from the server of the first information processing device to the server of the second information processing device. Also, when the first information processing device is a server and the second information processing device is a viewer terminal, information indicating the specified facial expression, etc. may be transmitted from the server of the first information processing device to the viewer terminal of the second information processing device. Also, when the first information processing device is a server and the second information processing device is a distributor terminal, information indicating the specified facial expression, etc. may be transmitted from the server of the first information processing device to the distributor terminal of the second information processing device.

Similarly, when the first information processing device is a viewer terminal and the second information processing device is a server, information indicating the specified facial expression, etc. may be transmitted from the viewer terminal of the first information processing device to the server of the second information processing device. Also, when the first information processing device is a viewer terminal and the second information processing device is a viewer terminal, information indicating the specified facial expression, etc. may be transmitted from the viewer terminal of the first information processing device to the viewer terminal of the second information processing device.
Furthermore, when the first information processing device is a viewer terminal and the second information processing device is a broadcaster terminal, information indicating such identified facial expressions, etc. may be transmitted from the viewer terminal of the first information processing device to the broadcaster terminal of the second information processing device.

Similarly, when the first information processing device is a broadcaster terminal and the second information processing device is a server, information indicating the specified facial expression, etc. may be transmitted from the broadcaster terminal of the first information processing device to the server of the second information processing device. Also, when the first information processing device is a broadcaster terminal and the second information processing device is a broadcaster terminal, information indicating the specified facial expression, etc. may be transmitted from the broadcaster terminal of the first information processing device to the broadcaster terminal of the second information processing device.
Furthermore, when the first information processing device is a broadcaster terminal and the second information processing device is a viewer terminal, information indicating such identified facial expressions, etc. may be transmitted from the broadcaster terminal of the first information processing device to the viewer terminal of the second information processing device.

Although the above describes an aspect in which the condition application section is executed in a single information processing device, the condition application section may be distributed among a plurality of information processing devices for processing.

Furthermore, the condition application unit may set information indicating the predetermined facial expression, etc., using the timing at which the predetermined facial expression, etc. is set. The timing may be, for example, as described above, the timing when the gaze of one broadcaster and the avatar of another broadcaster meets the gaze of the avatar or character, the timing when the avatar comes into contact with an object, etc. In this case, information including the information indicating the predetermined facial expression, etc. may be transmitted to an information processing device on which the information is displayed and displayed. The timing difference in this case may include a time lag in communication or processing.

On the other hand, the timing at which the information indicating the predetermined facial expression or the like is set after the predetermined condition is satisfied may be set separately. For example,
For example, the avatar may assume a predetermined facial expression two seconds after a situation that surprises the avatar occurs, and the avatar may assume a predetermined facial expression two seconds after the situation.

6 is an example showing the relationship between the predetermined conditions and the predetermined facial expressions, etc. By using the acquired information as the predetermined conditions, the corresponding predetermined facial expressions, etc. are identified.

As described above, when a certain condition is satisfied, the function of the condition application unit allows a corresponding predetermined facial expression, etc. to be realized, which has the advantage of allowing the facial expressions and poses of the avatar in the virtual space to be varied. In addition, there is an advantage that a facial expression, etc. can be set for the avatar in the virtual space in response to an event that actually occurred to the performer in the real world. In addition, information for setting the predetermined facial expression or pose may be generated without obtaining information for operating the avatar from the distributor terminal. In this case, there is an advantage that the performer can change the facial expression, etc. of the avatar without the burden of inputting changes in the avatar's facial expression, etc., because it corresponds to the situation in the real world, and without any information from the performer.

4. Embodiments
4-1. Embodiment 1
The first embodiment is an example in which the condition application unit operates in the viewer terminal. In this case, the acquisition unit may operate in the viewer terminal. This example will be described with reference to FIG. 7. Since the condition application unit operates in the viewer terminal, it may be possible to change the expression of the avatar for each viewer terminal by including information based on the viewer terminal as a predetermined condition.

Step 1
The acquisition unit acquires information on a predetermined action, etc. (step 001). The information on the predetermined action, etc. may be acquired at a distributor terminal or at a server, and may be acquired by a viewer terminal on which a condition application unit operates, via a server or the like.

Step 2
Next, the condition application unit determines whether a predetermined condition is satisfied using at least the information on the predetermined operation, etc.
When it is determined that the predetermined condition is satisfied, the facial expression, etc. corresponding to the satisfied predetermined condition is identified (step 002). For this purpose, the condition application unit may have information indicating at least some of the predetermined conditions and the predetermined facial expression, etc. when such conditions are satisfied.

Step 3
The specified facial expression, etc. is set in the viewer terminal as the facial expression, etc. associated with the avatar used to satisfy the predetermined condition (step 003). This has the advantage that the viewer can watch the reaction of the avatar as the facial expression, etc. of the avatar changes in response to the predetermined action, etc.

Step 4
At each viewer terminal, after a predetermined facial expression of the avatar is displayed for a predetermined period of time, the facial expression of the avatar returns to the facial expression based on the facial expression of the distributor associated with the original avatar (step 004).

4-2. Embodiment 2
The second embodiment is an example in which the condition application unit operates in the server. In this case, the acquisition unit may operate in the server. Note that this example will be described with reference to FIG. 8 on the premise that animation is generated on a viewer terminal.

Step 1
The acquisition unit acquires information on a predetermined action or the like (step 001).

Step 2
Next, the condition application unit determines whether a predetermined condition is satisfied using at least the information on the predetermined operation, etc.
If it is determined that the predetermined condition is satisfied, the facial expression or the like corresponding to the satisfied predetermined condition is identified (step 002).

Step 3
The server transmits information indicating the identified facial expression, etc. to each viewer terminal viewing the distribution of the avatar (step 003).

Step 4
In each viewer terminal, the facial expression of the avatar is set using the information indicating the facial expression, etc. acquired from the server (step 004). Here, in each viewer terminal, an animation may be generated using the set facial expression, etc. of the avatar.

Step 5
In each viewer terminal, the facial expression of the avatar using the information indicating the facial expression obtained from the server is displayed for a predetermined period of time, and then the facial expression is set to the facial expression of the distributor related to the original avatar (step 005).
Such generated facial expressions, etc. may be used and displayed within each animation for a predetermined period of time, and thereafter, facial expressions, etc. obtained by tracking based on the facial expressions, etc. of the broadcaster associated with the avatar may be used as the avatar's facial expressions, etc.

4-3. Embodiment 3
The third embodiment is an example in which the condition application unit operates in the server. In this case, the acquisition unit may operate in the server. Note that this example will be described with reference to FIG. 9 on the premise that the server generates a video including the facial expression of the avatar.

Step 1
The acquisition unit acquires information on a predetermined action or the like (step 001).

Step 2
Next, the condition application unit determines whether a predetermined condition is satisfied using at least the information on the predetermined operation, etc.
If it is determined that the predetermined condition is satisfied, the facial expression or the like corresponding to the satisfied predetermined condition is identified (step 002).

Step 3
The server generates a video for the distribution of the avatar using information indicating the specified facial expression, etc. (step 003). Here, when the server generating the video for the distribution of the avatar and the server executing the condition application unit are the same server, the server executing the condition application unit may generate the video for the distribution of the avatar. On the other hand, when the server generating the video for the distribution of the avatar and the server executing the condition application unit are different servers, information indicating the specified facial expression, etc. may be transmitted from the server executing the condition application unit to the server generating the video, and the information indicating the specified facial expression, etc. may be used in the server generating the video. Note that when an animation is generated in the server, an animation for the distribution of the avatar may be generated together with or instead of the video, and in this case, the animation may be used together with or instead of the video hereafter.

Step 4
An image of an avatar generated using the specified facial expression based on information indicating the specified facial expression, etc. is transmitted to each viewer terminal (step 004).

Step 5
At each viewer terminal, an image using information indicating facial expressions, etc. obtained from the server is displayed for a predetermined period of time, and then an image of an avatar based on facial expressions, etc. based on the facial expressions, etc. of the distributor associated with the original avatar is displayed (step 005).

4-4. Various embodiments The computer program according to the first aspect "causes the one or more processors to function in the following manner: detect information related to one or more of a predetermined action by a distributor, a facial expression of the distributor, a complexion of the distributor, the voice of the distributor, or a predetermined situation occurring to the distributor; determine whether the detected information satisfies a predetermined condition; and, if it is determined that the predetermined condition is satisfied, generate information for changing the facial expression or pose of an avatar related to the distributor that was generated based on the facial expression or pose related to the distributor to a predetermined facial expression or pose corresponding to the predetermined condition; and display the avatar to which the predetermined facial expression or pose is applied."

The computer program according to a second aspect is the computer program according to the first aspect, wherein "the specified action of the distributor is a predefined action that is perceived by the distributor through the five senses."

The computer program according to a third aspect is the computer program according to the first or second aspect, wherein "the specified action of the distributor includes an action of looking at a specified target."

The computer program according to a fourth aspect is the computer program according to any one of the first to third aspects, wherein "the specified action of the distributor includes an action of contacting a specified target."

The computer program according to a fifth aspect is the computer program according to any one of the first to fourth aspects, wherein "the specified action of the distributor includes an eating and drinking action of a specified target."

The computer program according to a sixth aspect is one in any one of the first to fifth aspects that "generates an item that represents in a virtual space something that corresponds to the specified target."

The computer program according to a seventh aspect is the computer program according to any one of the first to sixth aspects, wherein "the specified action of the distributor includes smelling or a specified gesture of smelling."

The computer program according to the eighth aspect is one in any one of the first to seventh aspects, in which "the specified action of the distributor includes a specified gesture regarding listening/listening to sound."

The computer program according to a ninth aspect is any one of the first to eighth aspects above, wherein "the detected information includes information detected as a result of input by a third party."

The computer program according to a tenth aspect is any one of the first to ninth aspects above, wherein "the detected information includes information detected by image processing using images captured by one or more imaging devices that capture the broadcaster."

The computer program according to an eleventh aspect is the computer program according to any one of the first to tenth aspects, wherein "the specified situation occurring to the distributor includes the occurrence of a specified sound."

The computer program according to the 12th aspect is any one of the first to 11th aspects above, in which "the specified situation occurring to the distributor includes a specified object coming into contact with the distributor."

The computer program according to a thirteenth aspect is the computer program according to any one of the first to twelfth aspects, wherein "the distributor's facial color includes a change in facial color."

The computer program according to the fourteenth aspect is any one of the first to thirteenth aspects above, wherein "the voice of the distributor includes the sound quality of the voice of the distributor changing by more than a predetermined range, or the volume of the voice of the distributor changing by more than a predetermined amount."

The computer program according to the fifteenth aspect is, in any one of the first to fourteenth aspects above, "applying a predetermined facial expression or pose corresponding to when a first predetermined condition is satisfied and a second predetermined condition is satisfied within a predetermined time, and when the first predetermined condition or the second predetermined condition, whichever is set as the higher priority condition, is satisfied."

The computer program according to the 16th aspect is any one of the aspects 1 to 15 above, in which "information for achieving the specified facial expression or pose is generated without obtaining information for operating the avatar from the distributor terminal."

A computer program according to a seventeenth aspect is a computer program,
When executed by one or more processors, the program detects information related to one or more of a predetermined action by a distributor, a facial expression of the distributor, a complexion of the distributor, a voice of the distributor, or a predetermined situation occurring to the distributor, determines whether the detected information satisfies a predetermined condition, and when it is determined that the predetermined condition is satisfied, changes the facial expression or pose of an avatar related to the distributor that was generated based on the facial expression or pose related to the distributor,
"The one or more processors are caused to function to generate information for a predetermined facial expression or pose corresponding to the predetermined condition, and to transmit the information for the predetermined facial expression or pose."

The computer program according to an eighteenth aspect is the computer program according to the seventeenth aspect, wherein "the specified action of the distributor is a predefined action that is perceived by the distributor through the five senses."

The server device according to the 19th aspect is equipped with "one or more processors, which execute computer-readable instructions to detect information relating to one or more of a predetermined action by a broadcaster, the broadcaster's facial expression, the broadcaster's complexion, the broadcaster's voice, or a predetermined situation occurring to the broadcaster, determine whether the detected information satisfies predetermined conditions, and if it is determined that the predetermined conditions are satisfied, generate information for changing the facial expression or pose of an avatar relating to the broadcaster that has been generated based on the facial expression or pose of the broadcaster to a predetermined facial expression or pose that corresponds to the predetermined conditions, and transmit the information for changing the facial expression or pose."

The server device according to a twentieth aspect is the server device according to the nineteenth aspect, wherein "the specified action of the distributor is a predefined action that is perceived by the distributor through the five senses."

The method according to the twenty-first aspect is "a method executed by one or more processors that execute computer-readable instructions, which detects information relating to one or more of a predetermined action by a broadcaster, a facial expression of the broadcaster, a complexion of the broadcaster, the voice of the broadcaster, or a predetermined situation occurring to the broadcaster, determines whether the detected information satisfies a predetermined condition, and if it is determined that the predetermined condition is satisfied, generates information for changing the facial expression or pose of an avatar of the broadcaster that was generated based on the facial expression or pose of the broadcaster to a predetermined facial expression or pose that corresponds to the predetermined condition, and displays the avatar with the predetermined facial expression or pose applied."

The method according to the twenty-second aspect is the method according to the twenty-first aspect, wherein "the predetermined operation of the distributor is
It is a predetermined action that is perceived by the distributor through the five senses.

The method according to the 23rd aspect is "a method executed by one or more processors that execute computer-readable instructions, which detects information relating to one or more of a predetermined action by a broadcaster, a facial expression of the broadcaster, a complexion of the broadcaster, the voice of the broadcaster, or a predetermined situation occurring to the broadcaster, determines whether the detected information satisfies a predetermined condition, and if it is determined that the predetermined condition is satisfied, generates information for changing the facial expression or pose of an avatar of the broadcaster that was generated based on the facial expression or pose of the broadcaster to a predetermined facial expression or pose that corresponds to the predetermined condition, and transmits the information for changing the facial expression or pose."

The method according to the 24th aspect is the method according to the 23rd aspect, in which "the predetermined operation of the distributor is
It is a predetermined action that is perceived by the distributor through the five senses.

The viewer terminal device according to the twenty-fifth aspect is provided with one or more processors, and the processor executes computer-readable instructions to detect information relating to one or more of a predetermined action by a distributor, a facial expression of the distributor, a complexion of the distributor, a voice of the distributor, or a predetermined situation occurring to the distributor, and determines whether the detected information satisfies a predetermined condition, and when it is determined that the predetermined condition is satisfied,
"Information is generated to change the facial expression or pose of the avatar related to the distributor, which has been generated based on the facial expression or pose related to the distributor, to a specified facial expression or pose that corresponds to the specified conditions, and the avatar is displayed with the specified facial expression or pose applied."

A broadcaster terminal device according to a twenty-sixth aspect of the present invention is provided with one or more processors, and the processor executes computer-readable instructions to detect information relating to one or more of a predetermined action by a broadcaster, a facial expression of the broadcaster, a complexion of the broadcaster, a voice of the broadcaster, or a predetermined situation occurring to the broadcaster, and determines whether the detected information satisfies a predetermined condition, and when it is determined that the predetermined condition is satisfied,
"generating information for changing the facial expression or pose of the avatar related to the distributor, which has been generated based on the facial expression or pose related to the distributor, to a specified facial expression or pose that corresponds to the specified conditions, and transmitting the information for changing to the specified facial expression or pose."

The system and information processing described in the present application may be implemented by software, hardware, or a combination of these, and a part or all of the system and information processing may be implemented as computer programs and executed by various computers, and these computer programs may be stored in storage media. These programs may also be stored in non-transient or temporary storage media.

It goes without saying that what is described in this application document is not limited to what is described in this application document, but can be applied to various examples within the scope of various technical ideas having various technical advantages and configurations described in this application document.

REFERENCE SIGNS LIST 1 System 10 Communication network 20 (20A to 20C) Server device 30 (30A to 30C) Terminal device 21 (31) Arithmetic device 22 (32) Main memory device 23 (33) Input/output interface 24 (34) Input device 25 (35) Auxiliary memory device 26 (36) Output device 41 Acquisition unit 42 Condition application unit

Claims (10)

Computer,
A first means for acquiring a motion in a real space of an actor corresponding to an avatar moving in a virtual space;
a second means for determining whether the movement of the actor in real space satisfies any of one or more conditions;
A program that functions as a third means for: (1) making the avatar's movements in the virtual space correspond to the movements of the actor in real space acquired by the first means when none of the one or more conditions are met; and (2) making the avatar's movements in the virtual space correspond to the movements of the actor in real space acquired by the first means when a certain condition of the one or more conditions is met, and after a predetermined period of time, making the avatar's movements in the virtual space correspond to the movements of the actor in real space associated with the certain condition. The program according to claim 1, wherein the avatar's movement is the avatar's facial expression or pose. The program according to claim 1 or 2, wherein each of the one or more conditions is associated with an action in advance. the one or more conditions include at least a first condition and a second condition,
The computer-readable medium according to claim 1 , wherein the action associated with the first condition is different from the action associated with the second condition. The program according to any one of claims 1 to 4, wherein the movement of the actor in real space is detected by a motion sensor and acquired by the first means. The program according to any one of claims 1 to 5, wherein the movement associated with the certain condition is different from the movement of the actor in real space acquired by the first means. The program according to any one of claims 1 to 6, wherein one of the one or more conditions is that the distributor smells or makes a certain gesture that smells. one of the one or more conditions being that a predetermined action has been performed on a predetermined object existing in real space;
The program according to claim 1 , wherein the action associated with the condition is to generate an item corresponding to the predetermined target in a virtual space. A first means for acquiring a motion in a real space of an actor corresponding to an avatar moving in a virtual space;
A second means for determining whether the movement of the actor in the real space satisfies any of one or more conditions;
An information processing device comprising: (1) a third means for, when none of the one or more conditions are satisfied, making the avatar's movements in the virtual space correspond to the movements of the actor in the real space acquired by the first means; and (2) a third means for, when a certain condition of the one or more conditions is satisfied, making the avatar's movements in the virtual space correspond to the movements associated with the certain condition, and after a predetermined period of time, making them correspond to the movements of the actor in the real space. A step in which a first means acquires an action in a real space of an actor corresponding to an avatar that operates in a virtual space;
A step of a second means for determining whether or not the movement of the actor in the real space satisfies any of one or more conditions;
An information processing method comprising: a third means for (1) setting the avatar's movements in the virtual space to correspond to the actor's movements in real space acquired by the first means when none of the one or more conditions are satisfied; and (2) setting the avatar's movements in the virtual space to movements associated with the certain condition when a certain condition of the one or more conditions is satisfied, and after a predetermined period of time, setting the avatar's movements in the virtual space to correspond to the actor's movements in real space.