WO2024080135A1 - 表示制御装置、表示制御方法および表示制御プログラム - Google Patents

表示制御装置、表示制御方法および表示制御プログラム Download PDF

Info

Publication number
WO2024080135A1
WO2024080135A1 PCT/JP2023/034925 JP2023034925W WO2024080135A1 WO 2024080135 A1 WO2024080135 A1 WO 2024080135A1 JP 2023034925 W JP2023034925 W JP 2023034925W WO 2024080135 A1 WO2024080135 A1 WO 2024080135A1
Authority
WO
WIPO (PCT)
Prior art keywords
display control
avatar
control device
user
virtual space
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2023/034925
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
悟郎 高木
智裕 石井
河野　温子
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sony Group Corp
Original Assignee
Sony Group Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sony Group Corp filed Critical Sony Group Corp
Priority to CN202380070690.5A priority Critical patent/CN119998774A/zh
Priority to JP2024551389A priority patent/JPWO2024080135A1/ja
Publication of WO2024080135A1 publication Critical patent/WO2024080135A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/16—Sound input; Sound output
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating three-dimensional [3D] models or images for computer graphics

Definitions

  • the present disclosure relates to a display control device, a display control method, and a display control program for representing an avatar in virtual reality.
  • Metaverse Various technologies have been put into practical use that present users with a three-dimensional virtual space constructed within a computer or computer network, known as the Metaverse.
  • VR Virtual Reality
  • AR Augmented Reality
  • This disclosure therefore proposes a display control device, a display control method, and a display control program that can represent avatars in a virtual space in a variety of ways.
  • a display control device includes an acquisition unit that acquires user behavior in real space as input information, a determination unit that determines whether the input information includes an input corresponding to a specific action registered in advance, and a display control unit that reflects the user behavior corresponding to the input information in an avatar in a virtual space, and, if it is determined that the input information includes an input corresponding to the specific action, further reflects the specific action in an avatar in the virtual space and displays the avatar in the virtual space.
  • FIG. 2 is a diagram showing an overview of a display control process according to the embodiment.
  • FIG. 11 is a diagram illustrating an example of a determination process according to the embodiment.
  • FIG. 1 is a diagram illustrating an example of the configuration of a display control device according to an embodiment.
  • FIG. 4 is a diagram illustrating an example of a gesture storage unit according to the embodiment.
  • 5 is a flowchart showing a flow of a display control process according to the embodiment.
  • FIG. 11 is a diagram (1) for explaining a first variation of the display control process according to the embodiment.
  • FIG. 11 is a diagram (2) for explaining a first variation of the display control process according to the embodiment.
  • FIG. 11 is a diagram (1) for explaining a second variation of the display control process according to the embodiment.
  • FIG. 11 is a diagram illustrating an example of a determination process according to the embodiment.
  • FIG. 1 is a diagram illustrating an example of the configuration of a display control device according to an embodiment.
  • FIG. 4 is
  • FIG. 11 is a diagram (2) for explaining a second variation of the display control process according to the embodiment.
  • FIG. 11 is a diagram (3) for explaining a second variation of the display control process according to the embodiment.
  • FIG. 11 is a diagram for explaining a third variation of the display control process according to the embodiment;
  • FIG. 11 is a diagram (2) for explaining a third variation of the display control process according to the embodiment.
  • FIG. 11 is a diagram (3) for explaining a third variation of the display control process according to the embodiment.
  • FIG. 4 is a diagram for explaining a third variation of the display control process according to the embodiment.
  • FIG. 5 is a diagram for explaining a third variation of the display control process according to the embodiment;
  • FIG. 6 is a diagram for explaining a third variation of the display control process according to the embodiment.
  • FIG. 11 is a diagram for explaining a fourth variation of the display control process according to the embodiment;
  • FIG. 13 is a diagram (2) for explaining a fourth variation of the display control process according to the embodiment.
  • FIG. 11 is a diagram (3) for explaining a fourth variation of the display control process according to the embodiment.
  • FIG. 11 is a diagram for explaining a fifth variation of the display control process according to the embodiment;
  • FIG. 13 is a diagram (2) for explaining a fifth variation of the display control process according to the embodiment.
  • FIG. 11 is a diagram (3) for explaining a fifth variation of the display control process according to the embodiment.
  • FIG. 4 is a diagram for explaining a fifth variation of the display control process according to the embodiment.
  • FIG. 5 is a diagram (5) for explaining a fifth variation of the display control process according to the embodiment.
  • FIG. 2 is a hardware configuration diagram illustrating an example of a computer that realizes the functions of the display control device.
  • Embodiment 1-1 Overview of display control processing according to embodiment 1-2. Configuration of display control device according to embodiment 1-3. Processing procedure according to embodiment 1-4. Variations of processing according to embodiment 1-4-1. Expression of excitement between avatars 1-4-2. Expression of avatar emotions 1-4-3. Visualization of conversation 1-4-4. Proximity effect between avatars 1-4-5. Expression related to display and interaction of avatars 1-5. Modifications 2. Other embodiments 3. Effects of a display control device according to the present disclosure 4. Hardware configuration
  • FIG. 1 is a diagram showing an overview of the display control process according to the embodiment.
  • the display control process according to the embodiment is executed by a display control device 100, not shown in FIG. 1.
  • the display control device 100 is an information processing terminal for implementing VR and AR technology. That is, in the embodiment, the display control device 100 displays a three-dimensional virtual space (hereinafter simply referred to as "virtual space") constructed in a computer or computer network, such as a metaverse, on a display unit (display, etc.).
  • the display control device 100 also displays an avatar, which is a character that resembles a user, in the virtual space.
  • the display control device 100 is implemented by any information processing device, such as a PC (Personal Computer), a tablet terminal, or a smartphone.
  • the display control device 100 may be, for example, a head mounted display (HMD) or AR glasses.
  • HMD head mounted display
  • the display control device 100 captures an image of a user in real space with an image sensor (such as a camera) and reflects the user's gestures and facial expressions in an avatar in the virtual space. This allows the user to experience the sensation of having an alter-ego of himself or herself present in the virtual space.
  • an image sensor such as a camera
  • users can communicate via the avatars in the same way as in the real space. For example, users can deepen their interactions with other users by greeting them and conversing (voice chat or text chat) via their avatars.
  • virtual spaces can be expressed in a variety of ways, they can be constructed not only as realistic spaces based on real space, but also as spaces with cartoon-like expressions that users can relate to. In this case, it is also desirable for the provider of the virtual space to not only be able to make the avatar reproduce realistic movements based on reality, but also to be able to provide a variety of emotional expressions and diverse movements that users can relate to.
  • the display control device 100 realizes the above processing by the configuration shown below.
  • the display control device 100 acquires user behavior in real space as input information, and determines whether the input information includes an input corresponding to a specific action registered in advance.
  • the display control device 100 then reflects the user behavior corresponding to the input information in an avatar in a virtual space, and if it is determined that the input information includes an input corresponding to a specific action, it further reflects the specific action in the avatar in the virtual space.
  • the display control device 100 registers in advance comical movements including cartoon expressions, such as a thumbs-up, a signature pose of each avatar, and a high-five, as specific movements (hereinafter referred to as "gestures") to be performed by the avatar.
  • the display control device 100 receives an input from the user of an action for activating the gesture, the display control device 100 reflects the gesture, rather than the user's movement itself, in the avatar.
  • the display control device 100 may reflect the user's gesture itself in the avatar. In this way, the display control device 100 reflects both gestures associated with the user's conscious movements and the user's unconscious movements in the avatar.
  • the display control device 100 can display an avatar in a virtual space that not only imitates the user's movements, but also has expressive and comical movements. Through such expressions, the display control device 100 can build a virtual space that makes it easier for users to communicate with each other and to feel familiar with each other.
  • the display control device 100 displays an avatar 10A that exists in a virtual space on an example display screen 20.
  • the display control device 100 uses a camera to constantly capture images of the user corresponding to the avatar 10A (i.e., the user whose movements and expressions are reflected in the avatar 10A).
  • the display control device 100 acquires input information input by the user based on image data obtained from the camera.
  • input information is information acquired from various sensors including the camera, and is a general term for information used for displaying avatars, etc.
  • the display control device 100 acquires the user's facial expression 21 as input information using an image-based facial expression extraction technique.
  • the display control device 100 also acquires the user's skeleton 22 using an image-based skeleton estimation technique.
  • the display control device 100 also acquires gesture information 23 indicating the movement of the user's arms and hands using hand tracking technique.
  • the display control device 100 may acquire not only image data, but also voice or text information input by the user as input information.
  • the display control device 100 then combines this information to accept input information from the user to be reflected in the avatar 10A.
  • the display control device 100 acquires the avatar 10A nodding, shaking its head to indicate refusal, tilting its head in distress, etc. as unconscious actions of the user that are not specific actions.
  • the display control device 100 may also acquire various emotional expressions by the user, such as a laughing expression or expression, or a surprised expression or expression, as unconscious actions. In this disclosure, these are collectively referred to as "unconscious actions.”
  • the example on the left side of Figure 1 shows an example in which the unconscious action of the user raising his hand is reflected in the avatar 10A and displayed in the virtual space.
  • the display control device 100 can also determine, based on the acquired image data, whether or not the user is performing a specific action to trigger a gesture. That is, the display control device 100 determines whether or not the input information includes an input corresponding to a specific action that has been registered in advance. For example, as shown in the right part of FIG. 1, the display control device 100 displays an avatar 10B that exists in a virtual space in a screen display example 25. At this time, the display control device 100 uses a camera to constantly capture images of the user corresponding to the avatar 10B (i.e., the user whose movements and expressions are reflected in the avatar 10B).
  • avatar 10A and avatar 10B are drawn separately to make the comparison easier to understand, but in reality, unconscious actions and conscious actions are expressed by the same avatar.
  • avatar 10 when there is no need to distinguish between avatar 10A, avatar 10B, etc., they will be collectively referred to simply as "avatar 10.”
  • the display control device 100 accepts a request to transition to an input mode for determining the specific action. For example, when the user desires that the avatar 10B make a distinctive gesture based on a specific action by the user, the user requests the display control device 100 to transition to an input mode for consciously making the gesture.
  • the display control device 100 acquires the user's behavior, such as the user's facial expression 26, the user's skeletal structure 27, and gesture information 28, based on the captured image. The display control device 100 then combines the acquired information to determine whether or not the characteristics of the user's behavior match the characteristics for activating a pre-registered gesture. Note that various known technologies may be used for acquiring the user's behavior and determining the gesture.
  • the display control device 100 determines that the behavior input by the user is a movement that has enough characteristics to trigger a gesture, it reflects the gesture in the avatar 10B. For example, as shown in the right part of FIG. 1, the avatar 10B makes a pose with its index finger and thumb. At this time, the display control device 100 may perform a comical expression that includes a dramatic effect such as displaying radial lines around the avatar 10B along with the pose. This expression allows the user to distinguish whether the movement of the avatar 10B is a specific gesture registered in advance or the user's movement itself. In other words, such specific gestures are consciously made to be performed by the avatar 10B by the user. For this reason, in this disclosure, these are collectively referred to as "conscious movements.”
  • the display control device 100 can reflect both these unconscious and conscious actions in the avatar 10 and express them as reactions of the avatar 10. This allows the display control device 100 to display the avatar 10 in the virtual space in a variety of expressions, including not only a direct reflection of the user's movements, but also cartoon-like expressions, etc.
  • the user can switch between the input modes of unconscious actions and conscious actions, and reflect input information from unconscious actions or conscious actions in the avatar 10. Such switching may be performed, for example, based on an explicit designation by the user (manual operation such as pressing a button, voice input, etc.).
  • the display control device 100 can set in advance a spatial range within which the user will perform conscious actions, and switch the input mode based on such a setting. This point will be explained using FIG. 2.
  • FIG. 2 is a diagram showing an example of the determination process according to the embodiment.
  • FIG. 2 shows an example in which a user inputs his/her own behavior by being imaged by the camera 141 of the display control device 100.
  • the user makes hand signs 30 in a pre-set space 31 (e.g., a space within a predetermined distance from the camera 141).
  • the display control device 100 determines whether the user's behavior was performed in the space 31, for example, using image analysis or a depth sensor. If the user's behavior was performed in the space 31, the display control device 100 considers the behavior to be one that ignites a conscious action and determines that a gesture has been ignited. On the other hand, the display control device 100 reflects the user's behavior observed in space 32, which is a range outside the space 31, in the avatar 10 as an unconscious action.
  • the display control device 100 when the display control device 100 is located at a predetermined distance from the camera 141, it does not determine that the input from the user is an action for igniting a gesture, but only determines whether the user's actions performed in the space 31 are actions for igniting a gesture. This allows the user to reflect unconscious and conscious actions in the avatar 10 at will, without having to explicitly specify the switching of input modes.
  • the switching of input modes is not limited to the example shown in FIG. 2.
  • voice chat i.e., voice input
  • text chat i.e., manual character input
  • the display control device 100 may automatically switch to an input mode for extracting conscious actions (gesture detection).
  • the display control device 100 may perform motion blending that reflects both movements while switching modes.
  • the display control device 100 may turn off the input mode itself while the user is performing a UI (User Interface) operation, so that movements of different intentions are not reflected in the avatar 10.
  • the display control device 100 may activate the gesture input mode only within a predetermined time after performing an arbitrary key operation.
  • the display control device 100 may prevent erroneous gesture input by requesting the user to input a predetermined word when performing a gesture.
  • the display control device 100 may activate the gesture input mode when a mute button that turns off voice input from the user is pressed, and may set the display control device 100 to the unconscious action input mode in other cases.
  • the display control device 100 may simultaneously turn off the voice input and transition to a privacy mode (wherein input from the user is not accepted). In this way, the display control device 100 may use different input modes in response to various aspects.
  • the display control process according to the embodiment can reflect unconscious and conscious actions in the avatar 10 as desired, allowing the avatar 10 to be displayed in a variety of ways.
  • Fig. 3 is a diagram showing an example of the configuration of the display control device 100 according to an embodiment.
  • the display control device 100 is one of the components of a display control system 1 according to the embodiment shown in Fig. 3.
  • the display control system 1 includes the display control device 100 and an analysis server 50.
  • the analysis server 50 performs various analysis processes described below and provides the analysis results to the display control device 100.
  • the analysis server 50 performs information processing with a relatively high processing load, such as text-based sentiment analysis and machine learning processing for image recognition, which are not performed by the display control device 100.
  • the analysis server 50 is a cloud server managed by a business operator that operates a service related to virtual space.
  • the display control device 100 has a communication unit 110, a memory unit 120, a control unit 130, a sensor unit 140, and a display unit 150.
  • the display control device 100 may also have an input unit (such as a keyboard or a touch panel) that accepts various operations from a user who operates the display control device 100.
  • the communication unit 110 is realized, for example, by a NIC (Network Interface Card) or a network interface controller.
  • the communication unit 110 is connected to the network N by wired or wireless means, and transmits and receives information to and from the analysis server 50, etc., via the network N.
  • the network N is realized, for example, by a wireless communication standard or method such as Bluetooth (registered trademark), the Internet, Wi-Fi (registered trademark), UWB (Ultra Wide Band), or LPWA (Low Power Wide Area).
  • the storage unit 120 is realized, for example, by a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk.
  • a semiconductor memory element such as a random access memory (RAM) or a flash memory
  • a storage device such as a hard disk or an optical disk.
  • the storage unit 120 stores various information related to the display control process according to the embodiment.
  • the storage unit 120 includes a gesture storage unit 121.
  • FIG. 4 is a diagram showing an example of the gesture storage unit 121 according to an embodiment. As shown in FIG. 4, the gesture storage unit 121 has items such as "gesture ID,” "gesture,” and "determination item.”
  • Gesture ID is identification information for identifying a gesture.
  • Gesture indicates the specific content of the gesture.
  • Determination item indicates the item for extracting a feature amount used when determining whether or not the user's behavior will trigger the gesture.
  • the information on each gesture may include motion data (such as animation of the skeleton and facial expressions, dramatic effects, etc.) for moving the 3D model of the avatar 10 in accordance with the gesture.
  • the example shown in Figure 4 indicates that the gesture with gesture ID "A01" corresponds to a "thumbs up,” and whether or not to fire that gesture is determined by the user's "hand position” and "arm movement.”
  • the sensor unit 140 is a sensor that detects various environmental information.
  • the sensor unit 140 includes a camera 141, which is an imaging device that is composed of an outward-facing camera that captures images of the outside of the display control device 100, an inward-facing camera that captures images of the user, etc.
  • the camera 141 recognizes a subject (e.g., a user located in real space) located in front of the display control device 100.
  • the camera 141 acquires an image of the user who is the subject, and can calculate the distance from the display control device 100 (in other words, the camera 141) to the subject based on the parallax between images captured by the stereo camera and an analysis of the captured image data.
  • the sensor unit 140 may detect the distance in real space using a depth sensor capable of detecting the distance to any subject, such as a user.
  • the sensor unit 140 also includes a microphone 142 that can measure the user's voice and surrounding sounds. For example, when engaging in voice chat, the user can input voice by using the microphone 142 provided in the display control device 100.
  • the sensor unit 140 may have a function of detecting various information related to the user's motion, such as the orientation, inclination, motion, and moving speed of the user's body, in addition to the camera 141 and microphone 142. Specifically, the sensor unit 140 detects information related to the user's motion, such as information related to the user's head and posture, the motion of the user's head and body (acceleration and angular velocity), the direction of the field of view, and the speed of the viewpoint movement.
  • information related to the user's motion such as information related to the user's head and posture, the motion of the user's head and body (acceleration and angular velocity), the direction of the field of view, and the speed of the viewpoint movement.
  • the sensor unit 140 functions as various motion sensors such as a three-axis acceleration sensor, a gyro sensor, and a speed sensor, and detects information related to the user's motion. More specifically, the sensor unit 140 detects at least one change in the position and posture of the user's head by detecting the components of the yaw direction, pitch direction, and roll direction as the motion of the user's head.
  • the sensor unit 140 does not necessarily have to be provided in the display control device 100, and may be, for example, an external sensor connected to the display control device 100 by wire or wirelessly.
  • the camera 141 does not necessarily have to be provided inside the display control device 100, and may be an imaging device installed outside the display control device 100.
  • the display unit 150 displays various information output from the control unit 130.
  • the display unit 150 is a display that outputs video to the user.
  • the display unit 150 may also include an audio output unit (such as a speaker) that outputs audio.
  • the control unit 130 is realized, for example, by a CPU (Central Processing Unit), MPU (Micro Processing Unit), GPU, etc., executing a program stored inside the display control device 100 (for example, a display control program related to the present disclosure) using a RAM or the like as a working area.
  • the control unit 130 is also a controller, and may be realized, for example, by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array).
  • control unit 130 has an acquisition unit 131, a determination unit 132, and a display control unit 133.
  • the acquisition unit 131 acquires various types of information. For example, the acquisition unit 131 acquires input information to be reflected in the avatar 10 by sensing the user using the sensor unit 140. Specifically, the acquisition unit 131 acquires the user's behavior in real space as input information.
  • the acquisition unit 131 may also acquire various operations and setting requests from the user. For example, the acquisition unit 131 accepts designation from the user as to whether or not to reflect a specific action in the avatar 10. In other words, the acquisition unit 131 accepts designation from the user as to whether to set an input mode for inputting an unconscious action or an input mode for inputting a conscious action (gesture).
  • the acquisition unit 131 may acquire surrounding information, etc., using the sensor unit 140, etc. Specifically, as shown in FIG. 2, the acquisition unit 131 acquires the spatial position from the camera 141 when the user's behavior is captured. This allows the determination unit 132, which will be described later, to determine at what distance from the camera 141 the hand sign performed by the user is captured, and therefore the mode can be automatically switched to the gesture mode.
  • the acquisition unit 131 can acquire information not only about the user currently captured by the camera 141, but also about multiple users connected to the virtual space. In other words, the acquisition unit 131 can acquire the behavior of multiple users as input information. By using such input information from multiple users, the determination unit 132 can determine the level of interest in a conversation between multiple users, as described below.
  • the acquisition unit 131 may also receive various requests sent from the user via the UI. For example, the acquisition unit 131 receives requests to change the appearance of the avatar, and various operations for services related to the virtual space, such as purchasing, selecting, or exchanging an object (such as an item in the virtual space) linked to the avatar.
  • the determination unit 132 performs various determinations for controlling the display of the avatar 10. For example, the determination unit 132 determines whether or not the input information from the user includes an input corresponding to a specific action that has been registered in advance.
  • the input corresponding to a specific action that has been registered in advance is, in other words, a registered behavior that the user uses to trigger a gesture.
  • the determination unit 132 determines whether or not the input information includes an input corresponding to a specific action. In other words, the determination unit 132 determines whether or not the behavior performed by the user in front of the camera 141 is an intentional action that triggers a gesture.
  • the determination unit 132 does not determine whether or not the input information includes an input corresponding to a specific action if the position from the camera 141 is farther than a predetermined distance, but determines whether or not the input information includes an input corresponding to a specific action if the position from the camera 141 is within the predetermined distance.
  • the determination unit 132 performs gesture determination for behaviors such as hand signs made by the user in a position relatively close to the camera 141, and does not perform gesture determination for behaviors such as hand signs made by the user outside the set spatial range, determining them to be unconscious actions. This allows the determination unit 132 to freely switch input modes to express the avatar 10.
  • the display control unit 133 controls the display unit 150 to display information output from the control unit 130.
  • the display control unit 133 outputs the virtual space image rendered as video content to the display.
  • the output destination device is not limited to the built-in display, but may be other information devices such as a smartphone, or a video output device such as a television.
  • the display control unit 133 based on input information acquired from the user, such as an image of the user, a voice uttered by the user, or text entered by the user, the display control unit 133 reflects the user's behavior corresponding to the input information in the avatar 10 in the virtual space. Furthermore, when the display control unit 133 determines that the input information includes an input corresponding to a specific action, it reflects a gesture, which is a specific action, in the avatar 10 in the virtual space and displays the avatar 10 in the virtual space.
  • the display control unit 133 when the display control unit 133 is in a mode for inputting unconscious actions, it reflects the user's skeletal data, etc., acquired based on an image of the user, in the avatar 10.
  • the display control unit 133 when the display control unit 133 is in a gesture mode, if the determination unit 132 determines that the user's behavior will trigger a gesture, it reflects a gesture, including a specific animation or special effects, that has been registered in advance, in the avatar 10. That is, gestures made by the avatar 10 are expressed by animations that involve special movements or special effects, such as a thumbs-up, high-five, or clapping.
  • the display control unit 133 may control the sound output by the avatar (pre-registered laughter) in addition to the gestures.
  • Fig. 5 is a flowchart showing the flow of a display control process according to the embodiment.
  • the display control device 100 uses the sensor unit 140 to acquire an image of the user (step S11). As described above, the display control device 100 may also acquire the user's voice, input text, and the like along with the image.
  • the display control device 100 converts the user's image into skeletal data based on known posture estimation techniques, etc. (step S12).
  • the display control device 100 determines whether the current mode is the gesture mode (step S13). If the current mode is the gesture mode (step S13; Yes), the display control device 100 determines the feature point positions and movements in the skeletal data (step S14).
  • the display control device 100 determines whether or not the feature point position and movement correspond to a registered gesture (step S15). If they correspond to a registered gesture (step S15; Yes), the display control device 100 reads out the motion data of the gesture from the storage unit 120 (step S16). The display control device 100 then reflects the read out motion data in the avatar 10 (step S18).
  • step S13 If the mode is not the gesture mode (step S13; No), the display control device 100 does not perform gesture determination, and converts the skeletal data obtained from the user into skeletal data for the avatar 10 (step S17). Then, the display control device 100 reflects the skeletal data in the avatar 10 (step S18).
  • step S15 if the feature point positions and movements do not correspond to a gesture (step S15; No), the display control device 100 also converts the skeletal data obtained from the user into skeletal data for the avatar 10 (step S17). Then, the display control device 100 reflects the skeletal data in the avatar 10 (step S18).
  • the display control device 100 displays the avatar 10 in the virtual space based on the information reflected in the avatar 10 (step S19).
  • the display control device 100 repeats this display control process for each predetermined frame acquired by the sensor unit 140.
  • the display control device 100 displays various visual effects according to the excitement between multiple avatars.
  • the excitement between avatars means the excitement between users.
  • the display control device 100 can express in a way that other users can understand that active communication is taking place between multiple users in the virtual space.
  • the acquisition unit 131 acquires the behavior of multiple users as input information. Then, the determination unit 132 determines the level of excitement of the interaction between the multiple users in the virtual space based on the behavior of the multiple users. Furthermore, the display control unit 133 displays a presentation effect in the vicinity of the multiple users according to the level of excitement of the interaction.
  • the determination unit 132 determines the level of excitement in an exchange based on the number of users participating in the exchange, the frequency of conversations between users, and the degree of overlap in the voices of multiple users.
  • the display control unit 133 may also determine the expression or frequency of occurrence of the dramatic effect based on the level of excitement in the interaction, the body orientation of the avatar 10, and the content of the conversation between multiple users.
  • FIG. 6 is a diagram (1) for explaining the first variation of the display control process according to the embodiment.
  • the example shown in FIG. 6 shows that multiple avatars, avatar 10A, avatar 10B, and avatar 10C, are actively interacting by exchanging greetings and conversations.
  • the display control device 100 determines the level of excitement in the interaction between the multiple users, and displays an arc-shaped effect 200 and radial effect 201 around the avatars based on the determined level of excitement. In this way, the display control device 100 can perform a display that allows multiple avatars who are actively interacting to be distinguished simply by looking at the situation from a distance, even if they are not actually participating in the conversation.
  • the level of excitement between avatars may be derived from various parameters. For example, if the excitement between avatars is due to voice communication such as voice chat, parameters that may be used include the volume, speed, pitch, emotion, frequency of overlap, amount of filler, and the presence or absence of specific language expressions of the users' voices. In other words, the level of excitement in communication in a virtual space is not necessarily measured only by the volume of the voices or the amount of conversation.
  • the display control device 100 can use, for example, "overlap (simultaneous speech by multiple people)" as a parameter with high importance (weighted) for determining excitement.
  • the display control device 100 may also analyze the emotion of each voice by having the analysis server 50 or the like analyze the voices, and may calculate a high excitement level when a more emotional voice is included. Specifically, the display control device 100 may calculate a high excitement level for an exchange that includes shouts expressing admiration or strong emotion, such as "Yay (Japanese)" or “Wow! Mah! (English).” Alternatively, the display control device 100 may calculate a high excitement level based on the frequency of occurrence of greetings or words (positive words based on emotion analysis) that strongly imply an intention of celebration or joy, such as "Cheers!.
  • the display control device 100 can also change the presentation (visual) effects displayed depending on the level of excitement.
  • the display control device 100 may change the size or frequency of the dramatic effect represented by the arc depending on the level of excitement. Specifically, when the level of excitement is relatively high, the display control device 100 may display a large arc so as to cover all of the multiple users participating in the conversation. This allows the display control device 100 to accurately convey the excitement of the multiple users even to a user who is viewing the multiple users from a distance.
  • the display control device 100 may also display a large effect even if conversation is not necessarily active. For example, if multiple users are paying attention to some kind of event in the virtual space, conversation will be reduced, but it is determined that the users are actively interacting with each other. In this case, the display control device 100 may determine the level of excitement based on, for example, the intersection of the avatars' gazes or interactions other than conversation (such as the closeness of the users to each other). Then, depending on the level of excitement, the display control device 100 displays the arc-shaped or radial effect larger or brighter than usual. In this way, the display control device 100 can show other users that multiple users are actively interacting with each other, even without conversation.
  • FIG. 7 is a diagram (2) for explaining a first variation of the display control process according to the embodiment.
  • FIG. 7 shows a composite waveform 205 obtained from a conversation between multiple users.
  • range 206 shows a waveform in which the voices of multiple users are measured to be high, indicating that the conversation is taking place at a relatively loud volume.
  • Range 207 shows a waveform in which multiple voices from multiple users are synthesized, indicating that a lively exchange of conversation is taking place.
  • the display control device 100 determines the level of excitement among multiple users based on such waveform analysis. For example, the display control device 100 determines the level of excitement based on the characteristics of volume changes over a specified period of time. Alternatively, the display control device 100 may estimate the number of participants in a conversation based on waveform analysis, and determine the level of excitement to be higher the greater the estimated number of participants.
  • the display control device 100 may also change the representation of the dramatic effect based on the waveform.
  • the display control device 100 may change the color or density (frequency of occurrence) of an arc-shaped or radial display, the speed at which the arc-shaped or radial light spreads, etc., in accordance with the amplitude of the waveform.
  • the display control device 100 may not necessarily display an arc-shaped effect, but may display a speech bubble with a series of acute angles according to the voice, conversation content, or emotion. This allows the display control device 100 to express, for example, that multiple users are arguing with each other. In this case, the display control device 100 may display speech bubbles or other dramatic effects using a three-dimensional display (application of a rim shader) that is valid regardless of the viewing angle.
  • the display control device 100 may also change the speed at which the special effects are generated depending on the distance from the observer to the multiple users. Specifically, the display control device 100 may display the special effects slower the farther the distance is, and may display the special effects faster the closer the distance is. This allows the observing user to know at a glance how far away the actively interacting users are.
  • the determination unit 132 determines whether or not the input information from the user includes an input corresponding to a specific action linked to a specific emotion, such as laughing. If it is determined that the input information includes a specific action linked to a specific emotion, the display control unit 133 reflects the specific action linked to the specific emotion in the avatar 10.
  • the determination unit 132 may compare the input information with voiceprint information that is registered in advance by the user and is associated with a specific emotion, to determine whether the input information includes an input corresponding to a specific action associated with the specific emotion.
  • the display control unit 133 may also display an avatar 10 that reflects a specific action linked to a specific emotion, and may also display a dramatic effect linked to the specific emotion near the avatar 10.
  • FIG. 8 is a diagram (1) for explaining the second variation of the display control process according to the embodiment.
  • the avatar 10 is making a gesture associated with "laughing," which is an example of a specific emotion.
  • the display control device 100 may display a cartoon-like effect 210 associated with laughter around the avatar 10.
  • the display control device 100 can display an avatar 10 that is rich in emotional expression by having the avatar 10 perform a laughing gesture in response to the user laughing.
  • FIG. 9 is a diagram (2) for explaining a second variation of the display control process according to the embodiment.
  • FIG. 9 shows an example of an audio waveform 215 obtained from a user.
  • the measurement value obtained when the environmental sound (background noise) is measured while the user is engaged in voice chat or the like is set as measurement result 216.
  • the audio waveform obtained when the user is engaged in voice chat or the like is set as measurement result 217.
  • the peak of measurement result 217 is indicated by the line of result 218.
  • the difference between result 218 and measurement result 216 is indicated by difference 219.
  • the display control device 100 uses the microphone 142 to measure the environmental sound (background noise) when the user is engaged in voice chat, etc. Then, the display control device 100 measures the user's voice, estimates the peak volume as laughter, and measures the difference 219 with the environmental sound. In this way, the display control device 100 registers the peak value of the voice and the difference value with the environmental sound for each user.
  • the environmental sound background noise
  • the display control device 100 determines that the user has laughed. According to this method, the display control device 100 can determine whether or not the user has laughed based on the sound volume.
  • FIG. 10 is a diagram (3) for explaining a second variation of the display control process according to the embodiment.
  • FIG. 10 illustrates an example of a voice waveform 220 acquired from a user.
  • range 221 is the waveform when the user's laughter is measured.
  • the display control device 100 (or the analysis server 50) accumulates the waveforms when laughter is measured for each user, and learns the characteristics of the accumulated waveforms using a known voice learning method (e.g., deep learning that extracts characteristic patterns). This allows the display control device 100 to generate a learned voiceprint pattern that indicates the "laughing" state for each user.
  • a known voice learning method e.g., deep learning that extracts characteristic patterns
  • the display control device 100 can determine whether the voice contains laughter by comparing it with the learned voiceprint pattern. This allows the display control device 100 to determine with high accuracy whether the user has laughed.
  • human laughter is diverse, and the emotions contained in laughter include not only joy but also various other emotions. For this reason, in a machine learning model that determines laughter through learning, the display control device 100 may generate a model that categorizes emotions, etc., rather than a binary determination of whether or not someone is laughing.
  • the display control device 100 acquires a waveform indicating laughter based on the user's explicit designation. Specifically, the display control device 100 acquires the user's voice when the user presses an arbitrary button to cause the display control device 100 to acquire (record) laughter. At this time, the user laughs at will. This allows the display control device 100 to acquire a voiceprint pattern indicating the user's laughter (i.e., training data labeled "laughter"). Alternatively, the display control device 100 may acquire the user's voiceprint pattern so as to accumulate only the user's laughter using a general-purpose machine learning model for voice recognition that determines laughter. In general, the characteristics of laughter vary from person to person, so it is desirable for the display control device 100 to acquire and learn a voiceprint pattern for each individual user.
  • the display control device 100 may acquire voice based on various emotions of the user, not limited to laughter. Then, the display control device 100 can cause the avatar 10 to express the user's unique emotional expression by reflecting the expression based on the acquired voice in the avatar 10.
  • the display control device 100 may acquire the filler characteristics of each user and reflect the characteristics of the user who utters the filler in the avatar 10. Fillers such as “umm” and catchphrases such as "that's nice” uttered by a user can be said to indicate the characteristics of that user. For this reason, the display control device 100 learns the fillers and catchphrases of each user, and when a filler or catchphrase is observed, reflects it in the avatar 10, thereby reproducing the characteristics of that user in a virtual space. In this case, the display control device 100 can better reflect the characteristics of the user in the avatar 10 by emphasizing and cartoonish expressions, such as by expressing the sounds uttered as fillers or catchphrases in 3D using speech bubbles.
  • the display control unit 133 may display words uttered by the avatar 10 based on the input information, and may also display a history of words that transitions upward from the avatar.
  • the display control unit 133 may display the words uttered by the avatar 10 as icons that pictorially represent the meanings contained in the input information.
  • the determination unit 132 may also determine the distance between the position in the virtual space where the display of the words is observed (i.e., the position where the observer avatar is located) and the position of the avatar 10 that utters the words. In this case, the display control unit 133 displays the word history or icons as a dramatic effect that does not include the meaning of the words and icons, based on the determined distance. Note that, as will be described later, an example of a dramatic effect that does not include meaning is when no text or the like indicating the content of a conversation is displayed, and a history of lively conversation is simply displayed as a smoke signal or balloon.
  • the determination unit 132 may also determine the positional relationship between the position in the virtual space where the display of the words is observed and the position of the avatar 10 that speaks the words. In this case, the display control unit 133 may determine the angle and three-dimensional display of the display of the words observed in the virtual space based on the determined positional relationship.
  • the display control device 100 displays an icon or a conversation that includes visual effects, rather than a typical conversation displayed as text, etc. This allows the display control device 100 to display, in an easily observable manner, an active conversation taking place between avatars, etc.
  • FIG. 11 is a diagram (1) for explaining the third variation of the display control process according to the embodiment.
  • FIG. 11 shows that avatar 10A is having a conversation with the observer avatar, saying, "There's a movie I want to see!" Since avatar 10A is having the conversation in the vicinity of the observer, the conversation is displayed as text in speech bubble 230.
  • icon 231 the content of a conversation that avatar 10B is having, which is located farther away than avatar 10A, is shown by icon 231.
  • the display control device 100 performs a content and emotion analysis of the conversation of avatar 10B using the analysis server 50, and displays the analysis results as icon 231.
  • icon 231 indicates that avatar 10B had a conversation about going to the movies, then getting in a car, and then going out for dinner.
  • the display control device 100 displays the conversation content as icons (images, etc.) rather than displaying the text as is. This allows the observer to grasp the general content of the conversation and how lively it is, without having to decipher the detailed text information of distant conversations.
  • the display control device 100 can display conversations between avatars in an animation in which past history scrolls upward, as in icon 231. This allows the display control device 100 to communicate the flow of the conversation to other users in a visually easy-to-understand manner, and also allows users who are far away to understand how lively the conversation is.
  • FIG. 12 is a diagram (2) for explaining a third variation of the display control process according to the embodiment.
  • Figure 12 shows multiple avatars 235 having a conversation at a great distance from the observer.
  • the conversation between the multiple avatars 235 is displayed without displaying any semantic content such as text or icons, and only the conversation history 236 in the form of speech bubbles or circles (such displays are sometimes called "text balloons").
  • the conversation history 236 is animated and displayed upwards in the order of the most recent conversations. Therefore, in the example of FIG. 12, the conversations between multiple avatars 235 are expressed like smoke signals in the virtual space. This allows even a distant observer to know where in the virtual space an active conversation is taking place.
  • the display control device 100 can also change the expression method in various ways depending on the distance between the observer and the avatar 10. This will be explained using FIG. 13.
  • FIG. 13 is a diagram (3) for explaining a third variation of the display control process according to the embodiment.
  • Figure 13 shows how a conversation taking place near the observer (240), a conversation taking place at a medium distance from the observer (241), and a conversation taking place very far from the observer (242) are each expressed differently.
  • conversation 240 the content of the avatars' conversation is displayed in text, and the details of the conversation are displayed.
  • conversation 240 may also display image information shared between users, not just text, such as images sent by users during text chat. This allows an observer to get a rough idea of what the communication is about, without having to read the text.
  • conversation 241 the text display of the avatars' conversation is omitted, and icons and the like are inserted between the conversation so that the observer can grasp the general content of the conversation.
  • conversation 241 when two avatars are conversing, the observer can grasp the conversation content to the extent of which avatar is speaking.
  • older history in conversation 241 may be displayed in such a way that the lines of the avatars merge with each other as they move upward.
  • the display control device 100 may display the history more transparently the earlier it is (i.e., the conversation history further up), like a smoke signal in real space.
  • the display control device 100 may also display the display showing each line of dialogue smaller than conversations 240 and 241. In other words, the display control device 100 can change the display area, such as a speech bubble for displaying dialogue, depending on the distance.
  • the display control device 100 can provide the user with a UI that does not appear cluttered even when lively communication is taking place in the virtual space.
  • the display control device 100 can also change the expression method in various ways depending on the positional relationship between the observer and the avatar 10. This will be explained using FIG. 14.
  • FIG. 14 is a diagram (4) for explaining a third variation of the display control process according to the embodiment.
  • the left diagram in Figure 14 shows a conversation 250 taking place near the observer.
  • avatar 10A and avatar 10B are having a conversation.
  • conversation 250 includes a conversation history 251 of avatar 10A and a conversation history 252 of avatar 10B.
  • FIG. 14 shows an overhead view 255 of the conversation 250 as seen from above.
  • avatar 10A and avatar 10B are not facing each other as seen by an observer 256, but are standing at a slight angle.
  • the display control device 100 displays the conversation history 251 at an angle based on the positional relationship between the observer 256 and the avatar 10A (here, the angle between the line of sight of the observer 256 and the direction of the avatar 10A). Similarly, the display control device 100 displays the conversation history 252 at an angle based on the positional relationship between the observer 256 and the avatar 10B.
  • This representation allows the observer 256 to recognize the positional relationship with avatar 10A and avatar 10B simply by looking at the conversation history.
  • the display control device 100 may also perform display including dramatic effects not only on the angle but also on the content of the conversation itself, such as text. This point will be explained using FIG. 15.
  • FIG. 15 is a diagram (5) for explaining a third variation of the display control process according to the embodiment.
  • avatar 10A and avatar 10B are having a conversation, and text 260 indicating the content of the conversation, production effect 261, and text 262 are displayed.
  • the text 260 may include highlighting, such as being displayed in bold compared to other text.
  • the display control device 100 automatically highlights when the user utters a pre-registered word that strongly indicates an emotion, such as "yabai (crazy)."
  • the effect 261 is a character icon including a three-dimensional display.
  • the effect 261 is automatically displayed when the user utters a pre-registered word that strongly indicates an emotion, such as "yabai (crazy)."
  • the text 262 is also text that includes a three-dimensional display.
  • the text 262 is displayed based on a predetermined criterion, such as when the user utters a pre-registered word that strongly indicates an emotion, such as "surprise,” or when the user's voice volume is higher than normal.
  • These special text displays and special effects may be displayed based on, for example, sentiment analysis of user conversations.
  • the display control device 100 cooperates with the analysis server 50 to determine whether the content entered in the text chat is positive or negative using sentiment analysis.
  • the display control device 100 then automatically generates appropriate visual effects for the speech bubbles in which the text is displayed. Specifically, if the conversation is positive, the display control device 100 may display speech bubbles or text in bright colors or with animations of bouncing letters, or the like. Alternatively, if the conversation is negative, the display control device 100 may display speech bubbles in cool colors or with animations of crying text, or the like.
  • These speech bubbles and text may also be displayed in a history display (known as a word cloud, for example) in which the size of the text varies depending on the frequency of the words that appeared in the conversation.
  • a history display known as a word cloud, for example
  • FIG. 16 is a diagram (6) for explaining a third variation of the display control process according to the embodiment.
  • FIG. 16 shows examples of text 270 and text 275 with three-dimensional display.
  • Text 270 is configured so that the characters appear to be floating from the speech bubble. Therefore, in display 271 when text 270 is viewed from the side, the characters appear to be floating from the speech bubble.
  • the text 275 is also configured in such a way that the characters appear to be contained within a speech bubble. For this reason, when the text 275 is viewed from the side, the display 276 shows the characters as if they were contained within a cloud-like speech bubble. For this reason, to the observer, the characters appear to be displayed as two-dimensional plates, and the speech bubble appears to be displayed in a three-dimensional cloud or balloon-like shape.
  • the 3D display shown in Figure 16 etc. can also be displayed backwards. In this case, the images and text in the 3D display are displayed with the left and right reversed.
  • the display control device 100 can express text in conversations in a variety of ways, providing visual entertainment to the user.
  • the determination unit 132 may determine whether the first avatar is included within a range recognized by the second avatar. Then, when it is determined that the first avatar is included within a range recognized by the second avatar, the display control unit 133 may adjust the movement path of the first avatar so that the first avatar approaches the second avatar.
  • the determination unit 132 may also determine whether or not the first avatar has approached the second avatar in the virtual space. When it is determined that the first avatar has approached the second avatar, the display control unit 133 reflects a specific action corresponding to the approach in at least one of the first avatar and the second avatar.
  • the display control device 100 encourages interaction between users by using special displays and actions so that avatars can recognize each other's presence and communicate with each other in close proximity.
  • FIG. 17 is a diagram (1) for explaining the fourth variation of the display control process according to the embodiment.
  • the first diagram in FIG. 17 shows avatar 10D approaching a location where avatar 10A, avatar 10B, and avatar 10C are having a conversation.
  • avatar 10A recognizes that avatar 10D has entered a range where proximity is determined, either through initially set viewpoint recognition or by other means.
  • avatar 10A When avatar 10A recognizes the proximity of avatar 10D as shown in FIG. 17 (FIG. 2), it initiates a specific action. For example, avatar 10A performs a gesture of greeting by raising its hand. At this time, display control device 100 may automatically display a friendly message or the like to encourage avatar 10A to join the conversation with avatar 10D.
  • avatar 10D When avatar 10D responds to the greeting and indicates its intention to join the conversation with avatar 10A, etc., avatar 10D joins the conversation, as shown in FIG. 3 in FIG. 17.
  • the display control device 100 performs corrections so that the gaze of the avatar 10A and the avatar 10D meets, and causes the avatar 10A and the avatar 10D to perform a greeting gesture. In this way, the display control device 100 can allow the avatars to actively interact with each other.
  • the display control device 100 performs control processing based on several conditions. For example, the display control device 100 determines that one or more avatars are already in a conversation (chat) state. The display control device 100 also determines that another avatar has approached the area where the conversation is taking place. When these conditions are met, the display control device 100 corrects the display so that the avatars' viewpoints are more easily aligned, or causes them to perform gestures.
  • FIG. 18 is a diagram (2) for explaining a fourth variation of the display control process according to the embodiment.
  • the display control device 100 recognizes a proximity area 280 of avatar 10A and a proximity area 281 with avatar 10D.
  • the display control device 100 also recognizes a distance 282 between avatar 10A and avatar 10D.
  • the display control device 100 corrects the positions of avatar 10A and avatar 10D so that their lines of sight meet, for example by making a correction so that their bodies face each other.
  • the display control device 100 recognizes that the proximity area 280 of the avatar 10A and the proximity area 281 of the avatar 10D have come into contact. In this case, the display control device 100 causes the avatar 10A and the avatar 10D to make gestures to greet each other.
  • display control device 100 starts a conversation such as a voice chat between the two. At this time, display control device 100 newly recognizes conversation area 283 in which the conversation between avatar 10A and avatar 10D takes place.
  • the display control device 100 may correct the positions of their avatars so that they are not close to each other.
  • the display control device 100 may also perform control to prioritize voice chat when avatars are physically close to each other. This allows the display control device 100 to create an environment in which a newly joining avatar can easily call out to the unspecified number of avatars already participating in the conversation.
  • the display control device 100 may also prioritize text chat when the user has muted voice input.
  • FIG. 19 is a diagram (3) for explaining a fourth variation of the display control process according to the embodiment.
  • FIG. 19 shows a situation in which some kind of event is taking place in a virtual space.
  • the event involves a specific avatar appearing on stage 290.
  • the display control device 100 sets conversation-prohibited areas 292 and conversation-prohibited areas 293 in the passageways leading to stage 290 or in locations away from stage 290, so that the avatar does not remain in the passageways leading to stage 290 or in locations away from stage 290.
  • the display control device 100 sets conversation areas 294 around stage 290, etc., to actively encourage conversation around stage 290 and around monument 291.
  • the display control device 100 does not make any corrections to allow avatars to start a conversation in the conversation prohibited area 292 or the conversation prohibited area 293.
  • the display control device 100 makes corrections to allow avatars to start a conversation in the conversation area 294. This allows the display control device 100 to actively guide conversations between avatars in areas in the virtual space where conversation is encouraged.
  • the display control device 100 may correct the movement path of the avatar 10D so that the avatar 10D can easily participate in the conversation. For example, the display control device 100 may correct the walking path or walking speed so that the path along which the avatar 10D is walking automatically approaches an area close to the avatar 10A or an area where the avatars 10A and the like are having a conversation. The display control device 100 may also correct the body orientation so that the avatar 10D can easily make eye contact with the avatar 10A while walking.
  • the acquisition unit 131 acquires a request from the user to change the appearance of the avatar 10.
  • the display control unit 133 then reflects the change based on the request in the avatar 10, changing the appearance of the avatar 10 in the virtual space along with a dramatic effect.
  • the acquisition unit 131 may also acquire from the user a request to exchange any one of a plurality of objects linked to the first avatar with a second avatar. In this case, when the request is acquired, the display control unit 133 displays a list of the plurality of objects linked to the first avatar near the first avatar.
  • the acquisition unit 131 may also acquire an instruction from the user to select one of the objects displayed in a list to be exchanged for the second avatar.
  • the display control unit 133 acquires the instruction, it displays one of the objects so that it floats from the list display, and moves the floating object in the virtual space so that it overlaps with the second avatar in accordance with the operation received from the user.
  • the display control device 100 performs display control such as changing the appearance of the avatar 10 and actively encouraging the exchange of items between avatars 10. This allows the display control device 100 to provide visual entertainment for the user and actively encourage interaction between users.
  • FIG. 20 is a diagram (1) for explaining the fifth variation of the display control process according to the embodiment.
  • a UI displays candidate options 300 for changing the appearance of avatar 10A.
  • the second diagram in FIG. 20 shows the display control device 100 changing the appearance of the avatar 10A.
  • the display control device 100 may display a dramatic effect 302 in which the avatar 10A appears to be covered in smoke.
  • FIG. 3 in FIG. 20 shows the state in which the display control device 100 has changed the appearance of the avatar 10A.
  • the display control device 100 displays the changed appearance of the avatar 10A in the virtual space. In this way, the user can arbitrarily change the appearance of the avatar 10A.
  • FIG. 21 is a diagram (2) for explaining a fifth variation of the display control process according to the embodiment.
  • FIG. 21 shows the UI display when avatar 10A and an observer exchange items.
  • Avatar 10A presents exchange candidate items 310 to the observer.
  • the observer also presents exchange candidate items 311.
  • exchange button 312 When the observer selects exchange button 312, the items are exchanged.
  • the display control device 100 may perform internal processing so that such ownership is also exchanged when the items are handed over.
  • FIG. 22 is a diagram (3) for explaining a fifth variation of the display control process according to the embodiment.
  • FIG. 22 shows a state in which avatar 10A and avatar 10B are about to exchange item 320.
  • the user corresponding to avatar 10A selects item 320 on the screen display using a pointing device such as a mouse.
  • the display control device 100 enlarges the display around the item 320.
  • the user can move the item 320 closer to the avatar 10B by, for example, moving their arm in real space.
  • the display control device 100 displays a bone display 321 of the avatar 10A that corresponds to the user's arm, and moves the item 320 displayed near the arm of the avatar 10A toward the avatar 10B.
  • the display control device 100 determines that the item 320 has been transferred. The display control device 100 then displays the item 320 near the bone display 322 and transfers ownership of the item 320 to the avatar 10B.
  • the display control device 100 can complete the item exchange in a way that makes it appear as if avatars are actually handing over items to each other, rather than displaying a UI that indicates the item exchange.
  • FIG. 23 is a diagram (4) for explaining a fifth variation of the display control process according to the embodiment.
  • the avatar 10A has a head 330, a face 332, a body 334, and feet 338 set as parts of the appearance that can be changed.
  • the user can display a list of items that can be changed in each part.
  • the display control device 100 displays an item candidate 331 that corresponds to the head 330.
  • the display control device 100 displays an item candidate 333 that corresponds to the face 332, an item candidate 335 that corresponds to the body 334, and an item candidate 337 that corresponds to the feet 338.
  • the display control device 100 can also display an item candidate 339 for changing the overall appearance of the avatar 10A.
  • the display control device 100 displays a pop-up display of target item candidates for each part of the avatar 10A where a change in appearance is requested. Then, the display control device 100 selects an item for changing the appearance of the avatar 10A in accordance with the user's operation.
  • the display control device 100 may, for example, superimpose items present in a lower hierarchical level on a selected item.
  • the display control device 100 may further display multiple candidate items included in the glasses category. In this case, the user can select one item from the candidate items 333, for example, in a single stroke, and then select an item in the lower hierarchical level.
  • the display control device 100 may also set an appropriate shortcut depending on the type of device used by the user when selecting an item. For example, when selecting an item, the display control device 100 may accept input by key operation on a keyboard or the like, in addition to input by a pointing device such as a mouse or by the user's gestures.
  • the display control device 100 may also control the delivery of items using a UI display. This will be described with reference to FIG. 24.
  • FIG. 24 is a diagram (5) for explaining a fifth variation of the display control process according to the embodiment.
  • FIG. 24 shows a state in which the user selects item 341 from among item candidates 340 held by avatar 10A.
  • the user selects item 341 by pressing and holding a finger or a pointing device on the screen.
  • the display control device 100 transitions item 341 to a display in which it appears to be floating above item candidates 340 (floating display).
  • the user drags the floating item 341 to the avatar 10B.
  • the display control device 100 determines that the item 341 has been handed over to the avatar 10B.
  • the display control device 100 can perform processes such as item exchange between users using various UIs and screen displays.
  • the things exchanged between avatars in the virtual space are described as items, but the form that the items take can be arbitrarily set by the service side providing the virtual space.
  • the items may be data such as NFT (Non-Fungible Token).
  • each device in Fig. 3 conceptually shows a function in the display control system 1, and may take various forms depending on the embodiment.
  • the display control device 100 performs processing in cooperation with the analysis server 50.
  • the display control device 100 may also perform the processing performed by the analysis server 50 on its own device.
  • the display control device 100 may also be composed of multiple devices.
  • the display control device 100 may be divided into a first information processing device that acquires the user's behavior as an image and processes the acquired image, and a second information processing device that processes input of the user's voice, text, etc.
  • the first information processing device acquires the user's behavior as an image, performs posture estimation based on the image, and performs hand-raising determination, head direction determination, hand sign determination, etc.
  • the second information processing device may also perform laughter determination based on voice and emotion analysis based on text.
  • the display control device 100 may also be a device in which the display unit and the information processing unit are configured separately.
  • the information processing unit of the display control device 100 may be any information processing device such as a server or a PC.
  • each component of each device shown in the figure is a functional concept, and does not necessarily have to be physically configured as shown in the figure.
  • the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc.
  • the display control device according to the present disclosure includes an acquisition unit (the acquisition unit 131 in the embodiment), a determination unit (the determination unit 132 in the embodiment), and a display control unit (the display control unit 133 in the embodiment).
  • the acquisition unit acquires a user's behavior in the real space as input information.
  • the determination unit determines whether or not the input information includes an input corresponding to a specific action registered in advance.
  • the display control unit reflects the user's behavior corresponding to the input information in an avatar in a virtual space, and when it is determined that the input information includes an input corresponding to a specific action, the display control unit further reflects the specific action in the avatar in the virtual space and displays the avatar in the virtual space.
  • the display control device displays an avatar by combining unconscious actions that accompany the user's natural movements with conscious actions to activate gestures.
  • This allows the display control device to display an avatar in a virtual space not only in a way that directly reflects the user's movements, but also in a variety of expressions, including cartoon-like expressions, etc.
  • the acquisition unit also receives a designation from the user as to whether or not a specific action should be reflected in the avatar. If there is a designation from the user, the determination unit determines whether or not the input information includes an input corresponding to the specific action.
  • the acquisition unit also acquires the position from the imaging device when the user's behavior is captured. If the position from the imaging device is farther than a predetermined distance, the determination unit does not determine whether the input information includes an input corresponding to a specific action, and if the position from the imaging device is within the predetermined distance, the determination unit determines whether the input information includes an input corresponding to the specific action.
  • the display control device reflects the conscious action in the avatar when the user specifies that a conscious action should be performed or when the user makes a gesture in a specific area. This allows the display control device to display the avatar in accordance with the user's intention.
  • the acquisition unit also acquires the behavior of the multiple users as input information.
  • the determination unit determines the level of excitement of the interaction between the multiple users in the virtual space based on the behavior of the multiple users.
  • the display control unit displays a presentation effect in the vicinity of the multiple users according to the level of excitement of the interaction.
  • the determination unit also determines the level of excitement in the exchange based on the number of users participating in the exchange, the frequency of conversation between users, and the degree of overlap in the voices of multiple users.
  • the display control unit also determines the expression or frequency of occurrence of the dramatic effect based on the level of excitement in the interaction, the orientation of the avatar's body, and the content of the conversation between multiple users.
  • the display control device can visually display the excitement of a conversation between avatars, effectively showing even users who are not participating in the conversation how the avatars are actively interacting with each other.
  • the determination unit also determines whether or not the input information includes an input corresponding to a specific action linked to a specific emotion. If it is determined that the input information includes a specific action linked to a specific emotion, the display control unit reflects the specific action linked to the specific emotion in the avatar and displays the avatar in the virtual space.
  • the determination unit also compares the voiceprint information registered in advance by the user, which is associated with a specific emotion, with the input information to determine whether the input information includes an input corresponding to a specific action associated with the specific emotion.
  • the display control unit also displays an avatar that reflects a specific action linked to a specific emotion, and displays a dramatic effect linked to the specific emotion near the avatar.
  • the display control device can reflect gestures associated with specific emotions, such as laughter, in an avatar. This allows the display control device to express a wide variety of emotions in virtual space, just as it does in real space.
  • the display control unit also displays the words spoken by the avatar based on the input information, and displays the history of those words in an upward transition starting from the avatar.
  • the display control unit also displays the words uttered by the avatar as icons that pictorially represent the meaning contained in the input information.
  • the determination unit also determines the distance between the position in the virtual space where the display of the words is observed and the position of the avatar uttering the words. Based on the determined distance, the display control unit displays the word history or an icon as a dramatic effect that does not include the meaning of the words and icons.
  • the determination unit also determines the positional relationship between the position in the virtual space where the display of the words is observed and the position of the avatar uttering the words.
  • the display control unit determines the angle and three-dimensional display of the display of the words observed in the virtual space based on the determined positional relationship.
  • the display control device can present the conversation history in various ways, without cluttering the screen display, and allows the user to see at a glance where active communication is taking place.
  • the determination unit also determines whether the first avatar is included within a range recognized by the second avatar in the virtual space. If it is determined that the first avatar is included within a range recognized by the second avatar, the display control unit adjusts the movement path of the first avatar so that the first avatar approaches the second avatar.
  • the determination unit also determines whether or not the first avatar has approached the second avatar in the virtual space.
  • the display control unit displays the first avatar and the second avatar while reflecting a specific action corresponding to the approach of at least one of the first avatar and the second avatar.
  • the display control device can correct the behavior of avatars, thereby encouraging the avatars to actively interact with each other. This allows the display control device to increase opportunities for conversations between users in the virtual space, and stimulate interaction in the virtual space.
  • the acquisition unit also acquires a request from the user to change the appearance of the avatar.
  • the display control unit reflects the changes based on the request in the avatar, and changes the appearance of the avatar in the virtual space along with the dramatic effects.
  • the acquisition unit also acquires a request from the user to exchange any one of the multiple objects linked to the first avatar for the second avatar.
  • the display control unit displays a list of the multiple objects linked to the first avatar near the first avatar.
  • the acquisition unit also acquires an instruction from the user to select one of the objects displayed in the list to be exchanged for the second avatar.
  • the display control unit acquires the instruction, it displays one of the objects in question so that it floats from the list display, and moves the floating object in the virtual space so that it overlaps with the second avatar in accordance with the operation received from the user.
  • the display control device provides a UI with excellent operability, thereby reducing stress for users when interacting with each other, such as exchanging items. This allows the display control device to support active interaction between users.
  • Fig. 25 is a hardware configuration diagram showing an example of a computer 1000 that realizes the functions of the display control device 100.
  • the computer 1000 has a CPU 1100, a RAM 1200, a ROM (Read Only Memory) 1300, a HDD (Hard Disk Drive) 1400, a communication interface 1500, and an input/output interface 1600.
  • Each unit of the computer 1000 is connected by a bus 1050.
  • the CPU 1100 operates based on the programs stored in the ROM 1300 or the HDD 1400 and controls each component. For example, the CPU 1100 loads the programs stored in the ROM 1300 or the HDD 1400 into the RAM 1200 and executes processes corresponding to the various programs.
  • the ROM 1300 stores boot programs such as the Basic Input Output System (BIOS) that is executed by the CPU 1100 when the computer 1000 starts up, as well as programs that depend on the hardware of the computer 1000.
  • BIOS Basic Input Output System
  • HDD 1400 is a computer-readable recording medium that non-temporarily records programs executed by CPU 1100 and data used by such programs.
  • HDD 1400 is a recording medium that records a display control program related to the present disclosure, which is an example of program data 1450.
  • the communication interface 1500 is an interface for connecting the computer 1000 to an external network 1550 (e.g., the Internet).
  • the CPU 1100 receives data from other devices and transmits data generated by the CPU 1100 to other devices via the communication interface 1500.
  • the input/output interface 1600 is an interface for connecting the input/output device 1650 and the computer 1000.
  • the CPU 1100 receives data from an input device such as a keyboard or a mouse via the input/output interface 1600.
  • the CPU 1100 also transmits data to an output device such as a display, edger, or printer via the input/output interface 1600.
  • the input/output interface 1600 may also function as a media interface that reads programs and the like recorded on a specific recording medium. Examples of media include optical recording media such as DVDs (Digital Versatile Discs) and PDs (Phase change rewritable Disks), magneto-optical recording media such as MOs (Magneto-Optical Disks), tape media, magnetic recording media, and semiconductor memories.
  • the CPU 1100 of the computer 1000 executes a display control program loaded onto the RAM 1200 to realize the functions of the control unit 130, etc.
  • the display control program according to the present disclosure and data in the storage unit 120 are stored in the HDD 1400.
  • the CPU 1100 reads and executes the program data 1450 from the HDD 1400, but as another example, the CPU 1100 may obtain these programs from other devices via the external network 1550.
  • the present technology can also be configured as follows.
  • an acquisition unit that acquires user behavior in a real space as input information; a determination unit that determines whether or not the input information includes an input corresponding to a specific action that has been registered in advance; a display control unit that reflects a behavior of the user corresponding to the input information in an avatar in a virtual space, and, when it is determined that the input information includes an input corresponding to the specific action, further reflects the specific action in an avatar in the virtual space and displays the avatar in the virtual space;
  • a display control device comprising: (2) The acquisition unit is accepting a designation from the user as to whether or not the specific action is to be reflected in the avatar; The determination unit is determining whether or not an input corresponding to the specific action is included in the input information when the specific action is designated by the user; The display control device according to (1).
  • the acquisition unit is Acquire a position from an imaging device when the behavior of the user is imaged;
  • the determination unit is When the position from the imaging device is farther than a predetermined distance, it is not determined whether or not the input corresponding to the specific action is included in the input information, and when the position from the imaging device is within a predetermined distance, it is determined whether or not the input corresponding to the specific action is included in the input information.
  • the display control device according to (1) or (2).
  • the acquisition unit is Obtaining the behavior of multiple users as input information,
  • the determination unit is determining a degree of excitement of an interaction between the plurality of users in the virtual space based on the behavior of the plurality of users;
  • the display control unit is displaying a performance effect in the vicinity of the plurality of users according to the level of excitement of the exchange;
  • the display control device according to any one of (1) to (3).
  • the determination unit is determining a level of excitement in the exchange based on the number of users participating in the exchange, the frequency of conversations between the users, and the degree of overlap of voices between the multiple users;
  • the display control device according to (4).
  • the display control unit is determining an expression or occurrence frequency of the performance effect based on the level of excitement of the exchange, the body orientation of the avatar, and the content of the conversation between the plurality of users;
  • the display control device according to (5) above.
  • the determination unit is determining whether an input corresponding to the specific action associated with a specific emotion is included in the input information;
  • the display control unit is when it is determined that the input information includes a specific action associated with the specific emotion, reflecting the specific action associated with the specific emotion in the avatar and displaying the avatar in the virtual space;
  • a display control device according to any one of (1) to (6).
  • the determination unit is comparing the input information with voiceprint information registered in advance by the user, the voiceprint information being associated with the specific emotion, to determine whether or not the input information includes an input corresponding to a specific action associated with the specific emotion;
  • the display control device according to (7) above.
  • the display control unit is displaying the avatar reflecting a specific action associated with the specific emotion, and displaying a performance effect associated with the specific emotion in the vicinity of the avatar;
  • the display control unit is displaying words uttered by the avatar based on the input information, and displaying a history of the words in an upward transition manner starting from the avatar;
  • a display control device according to any one of (1) to (9).
  • the display control unit is displaying the words uttered by the avatar as icons pictorially representing the meanings contained in the input information;
  • the display control device according to (10).
  • the determination unit is determining a distance between a position in a virtual space where the display of the words is observed and a position of an avatar uttering the words;
  • the display control unit is displaying the history of the words or the icons as a dramatic effect not including the meaning of the words and the icons based on the determined distance;
  • the display control device according to (11).
  • the determination unit is determining a positional relationship between a position in a virtual space where the display of the words is observed and a position of an avatar uttering the words;
  • the display control unit is determining an angle and a three-dimensional representation of the representation of the words observed in a virtual space based on the determined positional relationship;
  • the display control device according to (11) or (12).
  • the determination unit is determining whether a first avatar is included within a range recognized by a second avatar in the virtual space;
  • the display control unit is When it is determined that the first avatar is included in a range recognized by the second avatar, a movement path of the first avatar is adjusted so that the first avatar approaches the second avatar;
  • the display control device according to any one of (1) to (13).
  • the determination unit is determining whether the first avatar has come close to the second avatar in the virtual space;
  • the display control unit is when it is determined that the first avatar is in proximity to the second avatar, displaying the first avatar and the second avatar by reflecting a specific action corresponding to the proximity in at least one of the first avatar and the second avatar;
  • the display control device according to (14).
  • the acquisition unit is receiving a request from the user to change an appearance of the avatar;
  • the display control unit is reflecting the change based on the request in the avatar, thereby changing the appearance of the avatar in the virtual space together with a dramatic effect;
  • the display control device according to any one of (1) to (15).
  • the acquisition unit is receiving from the user a request to exchange any one of a plurality of objects associated with a first avatar with a second avatar;
  • the display control unit is When the request is received, a list of a plurality of objects associated with the first avatar is displayed in the vicinity of the first avatar.
  • the display control device according to any one of (1) to (16).
  • the acquisition unit is obtaining, from the user, an instruction to select one of the plurality of objects displayed in the list to be exchanged for the second avatar;
  • the display control unit is When the instruction is acquired, the display device displays any one of the objects in a floating manner from the list display, and moves the floating object in a virtual space so as to overlap the second avatar in accordance with an operation received from the user.
  • the display control device according to (17).
  • the computer The user's behavior in the real world is acquired as input information, determining whether the input information includes an input corresponding to a specific action registered in advance; reflecting the behavior of the user corresponding to the input information in an avatar in a virtual space, and when it is determined that the input information includes an input corresponding to the specific action, further reflecting the specific action in an avatar in the virtual space and displaying the avatar in the virtual space;
  • a display control method comprising: (20) Computer, an acquisition unit that acquires user behavior in a real space as input information; a determination unit that determines whether or not the input information includes an input corresponding to a specific action that has been registered in advance; a display control unit that reflects a behavior of the user corresponding to the input information in an avatar in a virtual space, and, when it is determined that the input information includes an input corresponding to the specific action, further reflects the specific action in an avatar in the virtual space and displays the avatar in the virtual space;
  • a display control program for causing the display control device to function as a display control device having the display
  • REFERENCE SIGNS LIST 10 Avatar 50 Analysis server 100
  • Display control device 110 Communication unit 120
  • Control unit 131 Acquisition unit 132 Determination unit 133
  • Sensor unit 141 Camera 142
  • Microphone 150 Display unit

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Software Systems (AREA)
  • Computer Hardware Design (AREA)
  • Computer Graphics (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • General Health & Medical Sciences (AREA)
  • User Interface Of Digital Computer (AREA)
PCT/JP2023/034925 2022-10-13 2023-09-26 表示制御装置、表示制御方法および表示制御プログラム Ceased WO2024080135A1 (ja)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202380070690.5A CN119998774A (zh) 2022-10-13 2023-09-26 显示控制装置、显示控制方法和显示控制程序
JP2024551389A JPWO2024080135A1 (https=) 2022-10-13 2023-09-26

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2022-165071 2022-10-13
JP2022165071 2022-10-13

Publications (1)

Publication Number Publication Date
WO2024080135A1 true WO2024080135A1 (ja) 2024-04-18

Family

ID=90669086

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2023/034925 Ceased WO2024080135A1 (ja) 2022-10-13 2023-09-26 表示制御装置、表示制御方法および表示制御プログラム

Country Status (3)

Country Link
JP (1) JPWO2024080135A1 (https=)
CN (1) CN119998774A (https=)
WO (1) WO2024080135A1 (https=)

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US12121820B2 (en) 2021-10-14 2024-10-22 Galiant Arts, LLC System for validating play of game applications via game NFTs and methods for use therewith
US12138546B2 (en) 2021-10-14 2024-11-12 Galiant Arts, LLC Derivative game NFTS with expiration restrictions
US12172076B2 (en) 2021-10-14 2024-12-24 Galiant Arts, LLC Facilitating generation of player token NFTs and methods for use therewith
US12175838B2 (en) 2021-10-14 2024-12-24 Galiant Arts, LLC Game platform using player token NFTs and methods for use therewith
US12250305B2 (en) 2021-10-14 2025-03-11 Galiant Arts, LLC NFT platform using player token NFTS for multiplayer game support and methods for use therewith
US12290755B2 (en) 2021-10-14 2025-05-06 Galiant Arts, LLC Facilitating generation of unique game characters and methods for use therewith
US12303794B2 (en) 2021-10-14 2025-05-20 Galiant Arts, LLC Generating and updating player token NFTs and methods for use therewith
US12311269B2 (en) 2021-10-14 2025-05-27 Galiant Arts, LLC Generating and updating non-player character NFTS and methods for use therewith
US12318699B2 (en) 2021-10-14 2025-06-03 Galiant Arts, LLC Generating player token NFTS via a blockchain-based distributed computer network based on player hierarchy
US12489637B2 (en) 2021-10-14 2025-12-02 Galiant Arts, LLC NFT-based authentication system for tagged objects and methods for use therewith
US12541755B2 (en) 2021-10-14 2026-02-03 Galiant Arts, LLC Client device for use with game token NFTs and methods for use therewith

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2016071558A (ja) * 2014-09-29 2016-05-09 シャープ株式会社 表示制御装置、制御方法、制御プログラム、および記録媒体
WO2018020766A1 (ja) * 2016-07-28 2018-02-01 ソニー株式会社 情報処理装置、情報処理方法、およびプログラム
JP2018092523A (ja) * 2016-12-07 2018-06-14 株式会社コロプラ 仮想空間を介して通信するための方法、当該方法をコンピュータに実行させるためのプログラム、および当該プログラムを実行するための情報処理装置
JP2019106220A (ja) * 2019-03-27 2019-06-27 株式会社コロプラ ヘッドマウントデバイスを介して仮想空間を提供するためにコンピュータで実行されるプログラムおよび方法ならびに情報処理装置
JP2020162882A (ja) * 2019-03-29 2020-10-08 株式会社バンダイナムコエンターテインメント サーバシステムおよびプレイデータコミュニティシステム
JP2021077255A (ja) * 2019-11-13 2021-05-20 富士フイルム株式会社 画像処理装置、画像処理方法及び画像処理システム
WO2022209450A1 (ja) * 2021-03-30 2022-10-06 ソニーグループ株式会社 情報処理装置、情報処理システムおよび情報処理方法

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2016071558A (ja) * 2014-09-29 2016-05-09 シャープ株式会社 表示制御装置、制御方法、制御プログラム、および記録媒体
WO2018020766A1 (ja) * 2016-07-28 2018-02-01 ソニー株式会社 情報処理装置、情報処理方法、およびプログラム
JP2018092523A (ja) * 2016-12-07 2018-06-14 株式会社コロプラ 仮想空間を介して通信するための方法、当該方法をコンピュータに実行させるためのプログラム、および当該プログラムを実行するための情報処理装置
JP2019106220A (ja) * 2019-03-27 2019-06-27 株式会社コロプラ ヘッドマウントデバイスを介して仮想空間を提供するためにコンピュータで実行されるプログラムおよび方法ならびに情報処理装置
JP2020162882A (ja) * 2019-03-29 2020-10-08 株式会社バンダイナムコエンターテインメント サーバシステムおよびプレイデータコミュニティシステム
JP2021077255A (ja) * 2019-11-13 2021-05-20 富士フイルム株式会社 画像処理装置、画像処理方法及び画像処理システム
WO2022209450A1 (ja) * 2021-03-30 2022-10-06 ソニーグループ株式会社 情報処理装置、情報処理システムおよび情報処理方法

Cited By (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US12121820B2 (en) 2021-10-14 2024-10-22 Galiant Arts, LLC System for validating play of game applications via game NFTs and methods for use therewith
US12138546B2 (en) 2021-10-14 2024-11-12 Galiant Arts, LLC Derivative game NFTS with expiration restrictions
US12138544B2 (en) 2021-10-14 2024-11-12 Galiant Arts, LLC Facilitating play of games via game NFTS
US12172076B2 (en) 2021-10-14 2024-12-24 Galiant Arts, LLC Facilitating generation of player token NFTs and methods for use therewith
US12175838B2 (en) 2021-10-14 2024-12-24 Galiant Arts, LLC Game platform using player token NFTs and methods for use therewith
US12226703B2 (en) 2021-10-14 2025-02-18 Galiant Arts, LLC Client device for validating play of game applications via game NFTS and methods for use therewith
US12239912B2 (en) 2021-10-14 2025-03-04 Galiant Arts, LLC NFT wallet for validating play of game applications via game NFTS and methods for use therewith
US12250305B2 (en) 2021-10-14 2025-03-11 Galiant Arts, LLC NFT platform using player token NFTS for multiplayer game support and methods for use therewith
US12257502B2 (en) 2021-10-14 2025-03-25 Galiant Arts, LLC Facilitating generation of player token NFTs and methods for use therewith
US12290755B2 (en) 2021-10-14 2025-05-06 Galiant Arts, LLC Facilitating generation of unique game characters and methods for use therewith
US12303794B2 (en) 2021-10-14 2025-05-20 Galiant Arts, LLC Generating and updating player token NFTs and methods for use therewith
US12311271B2 (en) 2021-10-14 2025-05-27 Galiant Arts, LLC System and method for validating game application versions via a game NFT
US12311269B2 (en) 2021-10-14 2025-05-27 Galiant Arts, LLC Generating and updating non-player character NFTS and methods for use therewith
US12311270B2 (en) 2021-10-14 2025-05-27 Galiant Arts, LLC Client device for validating play of game applications via game NFTs and methods for use therewith
US12318699B2 (en) 2021-10-14 2025-06-03 Galiant Arts, LLC Generating player token NFTS via a blockchain-based distributed computer network based on player hierarchy
US12394279B2 (en) 2021-10-14 2025-08-19 Galiant Arts, LLC Client device with a player token NFT corresponding to a game character
US12403401B2 (en) 2021-10-14 2025-09-02 Galiant Arts, LLC Game platform for updating game NFTs based on authenticated game play
US12403404B2 (en) 2021-10-14 2025-09-02 Galiant Arts, LLC Method for generating NFTs in response to in-app issuance requests
US12489637B2 (en) 2021-10-14 2025-12-02 Galiant Arts, LLC NFT-based authentication system for tagged objects and methods for use therewith
US12541755B2 (en) 2021-10-14 2026-02-03 Galiant Arts, LLC Client device for use with game token NFTs and methods for use therewith
US12592124B2 (en) 2021-10-14 2026-03-31 Galiant Arts, LLC Client device with updated player token NFTs and methods for use therewith

Also Published As

Publication number Publication date
CN119998774A (zh) 2025-05-13
JPWO2024080135A1 (https=) 2024-04-18

Similar Documents

Publication Publication Date Title
WO2024080135A1 (ja) 表示制御装置、表示制御方法および表示制御プログラム
US11398067B2 (en) Virtual reality presentation of body postures of avatars
US20220165013A1 (en) Artificial Reality Communications
US11100694B2 (en) Virtual reality presentation of eye movement and eye contact
KR102758381B1 (ko) 3차원(3d) 환경에 대한 통합된 입/출력
US20250111605A1 (en) Systems and methods of annotating in a three-dimensional environment
JP4395687B2 (ja) 情報処理装置
EP3646149A1 (en) Providing living avatars within virtual meetings
CN114787759A (zh) 交流支持程序、交流支持方法、交流支持系统、终端装置以及非语言表达程序
JP7278307B2 (ja) コンピュータプログラム、サーバ装置、端末装置及び表示方法
KR20230082374A (ko) 아바타를 적응적으로 생성하기 위한 전자 장치 및 그 동작 방법
US20260064236A1 (en) User interfaces and techniques for managing content
US20260072954A1 (en) User interfaces and techniques for interactions
US20260056603A1 (en) User interfaces and techniques for moving a computer system
US20260072639A1 (en) User interfaces for updating an indication of an activity
KR20260055473A (ko) 상호작용들을 위한 사용자 인터페이스들 및 기법들
KR20260055463A (ko) 콘텐츠를 관리하기 위한 사용자 인터페이스들 및 기법들
KR20260057173A (ko) 객체가 디스플레이되는 방식을 변경하기 위한 사용자 인터페이스들 및 기법들
WO2025072385A1 (en) User interfaces and techniques for changing how an object is displayed
WO2025072328A1 (en) User interfaces and techniques for performing an operation based on learned characteristics
CN121957353A (zh) 用于移动计算机系统的用户界面和技术
CN121957355A (zh) 用于移动计算机系统的用户界面和技术
CN121957354A (zh) 用于移动计算机系统的用户界面和技术
KR20260026588A (ko) 콘텐츠를 제시하기 위한 사용자 인터페이스들 및 기법들

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23877141

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2024551389

Country of ref document: JP

WWE Wipo information: entry into national phase

Ref document number: 202380070690.5

Country of ref document: CN

WWP Wipo information: published in national office

Ref document number: 202380070690.5

Country of ref document: CN

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 23877141

Country of ref document: EP

Kind code of ref document: A1