WO2023017584A1 - 仮想空間体感システム、及び、複合空間体感システム - Google Patents

仮想空間体感システム、及び、複合空間体感システム Download PDF

Info

Publication number
WO2023017584A1
WO2023017584A1 PCT/JP2021/029641 JP2021029641W WO2023017584A1 WO 2023017584 A1 WO2023017584 A1 WO 2023017584A1 JP 2021029641 W JP2021029641 W JP 2021029641W WO 2023017584 A1 WO2023017584 A1 WO 2023017584A1
Authority
WO
WIPO (PCT)
Prior art keywords
user
avatar
environment
space
virtual space
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2021/029641
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
良哉 尾小山
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Abal Inc
Original Assignee
Abal Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Abal Inc filed Critical Abal Inc
Priority to JP2022529032A priority Critical patent/JP7138392B1/ja
Priority to PCT/JP2021/029641 priority patent/WO2023017584A1/ja
Priority to JP2022084262A priority patent/JP2023026312A/ja
Publication of WO2023017584A1 publication Critical patent/WO2023017584A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G06F3/0346—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G06F3/04815—Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating three-dimensional [3D] models or images for computer graphics

Definitions

  • the present invention provides a virtual space sensory system for recognizing that the user exists in a virtual space generated by a server or the like, and a complex space for recognizing a complex space in which virtual objects exist in the real space. It relates to a spatial sensation system.
  • a virtual space is generated by a server or the like, and a user recognizes an image of the virtual space through a head-mounted display (hereinafter sometimes referred to as "HMD"), and the user himself/herself exists in the virtual space.
  • HMD head-mounted display
  • This type of virtual space experience system recognizes the user's motion in the real space (for example, movement of coordinates, change of posture including orientation, etc.) by a motion capture device or the like, and responds to the user according to the recognized motion.
  • a motion capture device or the like There is an avatar that operates in a virtual space (see Patent Document 1, for example).
  • an avatar corresponding to one user is made to exist in a first virtual space (for example, a virtual space of the sea) while maintaining a state in which the users can communicate with each other, and the user experiences the first virtual space.
  • a first virtual space for example, a virtual space of the sea
  • a second virtual space for example, a virtual space of a mountain
  • a complex space has been developed in which a virtual object exists in the real space and a user existing in the real space can influence the virtual object (for example, move the object).
  • a complex spatial experience system has been proposed that generates a complex space and allows the user to experience the complex space.
  • the virtual objects that exist in the complex space do not necessarily have to be common to all users, and may be different for each user. . That is, even when using the same complex space sensation system, each user may be configured to experience a complex space with different existing virtual objects.
  • An object is to provide a system and a complex spatial sensation system.
  • the virtual space experience system of the first invention is a virtual space generation unit that generates a virtual space corresponding to a real space in which a plurality of users including a first user and a second user exist; an avatar generation unit that generates a plurality of avatars including a first avatar corresponding to the first user and a second avatar corresponding to the second user in the same virtual space; a user action recognition unit that recognizes actions of the first user and the second user in the physical space; Avatar action of controlling the action of the first avatar in the virtual space based on the action of the first user, and controlling the action of the second avatar in the virtual space based on the action of the second user.
  • a control unit an avatar environment determination unit that determines environments of the first avatar and the second avatar in the virtual space; a user environment determination unit that determines an environment to be recognized by the one user based on the environment of the one avatar and determines an environment to be recognized by the second user based on the environment of the second avatar; a trigger event recognition unit that recognizes a predetermined trigger event;
  • a virtual space experience system comprising an environment output device that allows the first user and the second user to recognize the environment of the virtual space, Before the predetermined trigger event is recognized, The avatar action control unit controls a first property that is a property that defines the action of the avatar corresponding to the user in response to the user's action, and a property that is different from the first property and is the property that defines the action of the user's action
  • One characteristic of a second characteristic which is a characteristic that defines the behavior of the avatar corresponding to the user, is given to the first avatar, and then the behavior of the first avatar is controlled, and the first characteristic and the first characteristic are controlled.
  • the avatar environment determining unit determines the environment of the first avatar based on the behavior of the first avatar to which the one property is assigned, and the behavior of the second avatar to which the other property is assigned. determining the environment of the second avatar based on After the predetermined trigger event is recognized, The avatar action control unit imparts one of the first property and the second property to the first avatar and the second avatar, and then the action of the first avatar and the action of the second avatar. to control the The avatar environment determining unit determines the environment of the first avatar based on the behavior of the first avatar to which the one property is assigned, and the behavior of the second avatar to which the one property is assigned. and determining the environment of the second avatar.
  • the avatar's “environment” refers to things that affect the avatar in the virtual space. For example, it is the state of an object existing in the virtual space, physical laws in the virtual space (speed of time flow, magnitude of gravity, etc.), and the like.
  • the “environment to be recognized” by the user refers to the environment of the virtual space that is recognized by the user through the five senses. For example, the user's environment is images, sounds, etc. of the virtual space that the user recognizes.
  • the motion of each avatar is controlled based on the motion of the corresponding user. Also, the environment of the virtual space to be recognized by each user is determined based on the environment of the corresponding avatar in the virtual space.
  • avatars have different characteristics (for example, size, function, etc.), even if each user performs the same action in the real space, the environment during or after the action of the corresponding avatar may differ. . As a result, the environment experienced by each user may also be different.
  • each avatar can be different (that is, when the environment experienced by each user can be different), for example, when recognizing an avatar of a user other than oneself, the user who has recognized the avatar is a user other than himself. There may be a desire to experience the environment, or a user who has recognized that desire to allow other users to experience his/her own environment.
  • one of the first characteristic and the second characteristic is imparted to the first avatar before the predetermined trigger event is recognized, and the first avatar moves. Based on, the environment of the first avatar is determined, the other of the first property and the second property is given to the second avatar, and based on the behavior of the second avatar, the second avatar Determine your environment.
  • one of the first characteristic and the second characteristic is given to the first avatar, and the environment of the first avatar is determined based on the action of the first avatar. is determined, one of the characteristics is also given to the second avatar, and the environment of the second avatar is determined based on the behavior of the second avatar.
  • the first property and the second property are properties that define the behavior of the avatar corresponding to the user with respect to the user's behavior. Specifically, it is the size of the avatar in the virtual space and the functions of the avatar (for example, movable speed, movable range, etc.).
  • the characteristics of the first avatar corresponding to the first user and the characteristics of the second avatar corresponding to the second user are different. Therefore, when the avatar moves in response to the user's motion, the environment of the avatar becomes independent as in the past. As a result, the environment experienced by the user becomes independent as in the past.
  • the characteristics of the first avatar corresponding to the first user and the characteristics of the second avatar corresponding to the second user are both the same. Therefore, when the avatar moves in response to the user's motion, the environment of the avatar matches or is similar. Consequently, the environment experienced by the user will also match or be similar.
  • the process of synchronizing the environments of the avatars is executed so that the environments of the virtual space recognized by each user are the same.
  • the first user and the second user who have experienced different environments until then, can now experience the same environment.
  • the predetermined trigger event is a predetermined action performed by one of the first avatar and the second avatar on the other of the first avatar and the second avatar.
  • predetermined action may be any action performed by one of the first avatar and the second avatar with reference to the other avatar.
  • the predetermined action is an action in which one avatar touches a second avatar, an action in which one avatar moves within a predetermined range relative to the other avatar, and one avatar moves in a complex space.
  • the other avatar is selected as the target (for example, the other avatar is photographed with a camera-type object, etc.).
  • the virtual space experience system of the second invention is A first virtual space corresponding to a physical space in which a plurality of users including a first user and a second user exist, and a plurality of second virtual spaces independent of the first virtual space and corresponding to the physical space.
  • a virtual space generation unit that generates a virtual space of an avatar generation unit that generates a plurality of avatars including a first avatar corresponding to the first user and a second avatar corresponding to the second user in the first virtual space or the second virtual space; a user action recognition unit that recognizes actions of the first user and the second user in the physical space; Avatar action for determining control of the first avatar in the virtual space based on the action of the first user and controlling action of the second avatar in the virtual space based on the action of the second user.
  • a control unit an avatar environment determination unit that determines environments of the first avatar and the second avatar in the virtual space; a user environment determination unit that determines an environment to be recognized by the one user based on the environment of the one avatar and determines an environment to be recognized by the second user based on the environment of the second avatar; a trigger event recognition unit that recognizes a predetermined trigger event;
  • a virtual space experience system comprising an environment output device that allows the first user and the second user to recognize the environment of the virtual space, Before the predetermined trigger event is recognized, The avatar generation unit generates the first avatar in one of the first virtual space and the second virtual space, and generates the second avatar in the other of the first virtual space and the second virtual space.
  • the avatar environment determination unit determines the environment of the first avatar based on the one virtual space and determines the environment of the second avatar based on the other virtual space; After the predetermined trigger event is recognized, The avatar generation unit generates the first avatar and the second avatar in one of the first virtual space and the second virtual space, The avatar environment determination unit determines the environment of the first avatar based on the one virtual space, and determines the environment of the second avatar based on the one virtual space. .
  • the avatar's “environment” refers to things that affect the avatar in the virtual space.
  • the avatar's environment is the type of existing virtual space itself (whether it is on the ground or in the water).
  • the “environment to be recognized” by the user refers to the environment of the virtual space that is recognized by the user through the five senses.
  • the user's environment is images, sounds, etc. of the virtual space that the user recognizes.
  • the motion of each avatar is controlled based on the motion of the corresponding user. Also, the environment of the virtual space to be recognized by each user is determined based on the environment of the corresponding avatar in the virtual space.
  • the virtual space experienced by each user does not necessarily have to be based on one virtual space. Instead, it may be based on an independent virtual space for each user.
  • a first avatar corresponding to the first user may be present in the virtual space of the sea to allow the first user to experience the virtual sea environment, and at the same time, a second avatar corresponding to the second user may be placed in the virtual space of the mountain. , so that the second user can experience a virtual mountain.
  • each user can communicate (specifically, a conversation beyond the type of space).
  • a user may desire to experience the environment of a user other than himself, or a user may desire to allow a user other than himself to experience his own environment.
  • the first avatar is generated in one of the first virtual space and the second virtual space, and The environment of the first avatar is determined based on the virtual space, the second avatar is generated in the other virtual space of the first virtual space and the second virtual space, and the second avatar is generated based on the other virtual space. Determines the avatar's environment.
  • the first avatar and the second avatar are generated in one of the first virtual space and the second virtual space, and based on the one virtual space , determine the environment of the first avatar and the second avatar.
  • the virtual space in which the first avatar corresponding to the first user exists and the virtual space in which the second avatar corresponding to the second user exists are independent before the predetermined trigger event is recognized. It has become a thing. As such, the avatar's environment becomes independent, as in the past. As a result, the environment experienced by the user becomes independent as in the past.
  • the virtual space in which the first avatar corresponding to the first user exists and the virtual space in which the second avatar corresponding to the second user exists are the same virtual space. become.
  • the avatar's environment will match or be similar. Consequently, the environment experienced by the user will also match or be similar.
  • the process of synchronizing the environments of the avatars is executed so that the environments of the virtual space recognized by each user are the same.
  • the first user and the second user who have experienced different environments until then, can now experience the same environment.
  • the avatar generation unit corresponds to one of the first user and the second user, and generates a ghost, which is an avatar independent of the first avatar and the second avatar, of the first user and the second user.
  • the predetermined trigger event is a predetermined action performed on the ghost by the avatar corresponding to the other of the first user and the second user.
  • the "predetermined action” may be any action performed by the avatar based on the ghost.
  • the predetermined action is an action in which the avatar touches a ghost, an action in which the avatar moves within a predetermined range based on the ghost, an avatar operating an object existing in the virtual space, An operation of selecting a ghost as the target (for example, an operation of photographing a ghost with a camera-type object, etc.).
  • each user has other users.
  • For the purpose of notifying the may generate a ghost that is
  • the user corresponding to the avatar can experience the environment of the user corresponding to the ghost as a result of executing the action. You will be able to intuitively understand what you can do.
  • the complex spatial sensation system of the third invention is a real space recognition unit that recognizes a real space in which a plurality of users including a first user and a second user exist; an object generation unit that generates a first object and a second object, which are virtual objects to be recognized by the first user and the second user; generating a plurality of complex spaces including a first complex space that is a space in which the first object exists in the physical space and a second complex space that is a space in which the second object exists in the physical space; a composite space generator; a user environment determination unit that determines an environment to be recognized by the first user and the second user; a trigger event recognition unit that recognizes a trigger event; A complex spatial sensation system comprising an environment output device that allows the first user and the second user to recognize the environment of the complex space, Before the predetermined trigger event is recognized, The user environment determination unit determines an environment in one of the first complex space and the second complex space as an environment to be recognized by the first user, and determines the first complex space and the second complex complex
  • the user environment determining unit determines an environment in one of the first complex space and the second complex space as an environment to be recognized by the first user and the second user.
  • the "environment recognized by the user” refers to the environment of the complex space that the user recognizes through the five senses.
  • the user's environment is an image of an object existing in a complex space recognized by the user, a sound generated based on the object, and the like.
  • the complex space experienced by each user does not necessarily have to be based on a single complex space. It may be based on different composite spaces that are independent for each user.
  • a first user is allowed to experience a first composite space in which a first object exists in a predetermined region of the physical space, and at the same time, a second user is instructed to experience the same region of the physical space as the region used for the first composite space.
  • a second composite space in which a second object is present is experienced.
  • the complex space recognized by each user can be different (that is, when the environment experienced by each user can be different), for example, when each user communicates (specifically, a conversation, etc.), a certain user A user may desire to experience another user's environment, or a user may wish to have another user experience his/her own environment.
  • the environment in one of the first complex space and the second complex space is determined as the environment to be recognized by the first user.
  • the environment in the other of the first complex space and the second complex space is determined as the environment to be recognized by the second user.
  • the environment in one of the first and second complex spaces is determined as the environment to be recognized by the first and second users.
  • the composite space experienced by the first user and the composite space experienced by the second user become composite spaces in which existing virtual objects are independent of each other.
  • the complex space experienced by the first user and the complex space experienced by the second user become the same complex space in which existing virtual objects match.
  • the process for synchronizing the objects existing in the complex space is executed, and the environment of the complex space is made to be recognized by each user. become similar, the first user and the second user, who have experienced different environments up until now, can now experience similar environments.
  • a user action recognition unit that recognizes actions of the user in the physical space;
  • the predetermined trigger event is a predetermined action performed by one of the first user and the second user on the other of the first user and the second user in the complex space.
  • the "predetermined action” may be any action performed by one of the first user and the second user based on the other.
  • the predetermined action is an action in which the first user touches the second user, an action in which the first user moves within a predetermined range with respect to the second user, and an action in which the first user moves into a complex space.
  • an action to select the second user as the target for example, an action to take a picture of the second user with the camera of a virtual mobile terminal or a real mobile terminal etc.).
  • the first user when the first user sets a predetermined action to be performed by the second user as a predetermined trigger event, the first user also experiences the second user's environment as a result of executing the action. You will be able to intuitively understand what you can do.
  • FIG. 1 is a schematic diagram showing a schematic configuration of a VR system according to a first embodiment
  • FIG. FIG. 2 is a block diagram showing the configuration of a processing unit of the VR system in FIG. 1
  • FIG. 3 is a flowchart showing processing executed by the VR system of FIG. 2 in normal use
  • FIG. FIG. 3 is a schematic diagram showing states of a real space and a virtual space in a normal usage state of the VR system of FIG. 2
  • FIG. 3 is a flow chart showing the processing performed by the VR system of FIG. 2 when synchronizing avatar characteristics
  • FIG. FIG. 3 is a schematic diagram showing the state of the virtual space when a trigger event occurs in the VR system of FIG. 2
  • FIG. 3 is a schematic diagram showing the state of the virtual space after a trigger event has occurred in the VR system of FIG. 2;
  • 9 is a flow chart showing processing executed by the VR system of FIG. 8 in normal use.
  • FIG. 9 is a schematic diagram showing the states of the real space and the virtual space in the normal usage state of the VR system of FIG. 8;
  • FIG. 9 is a flow chart showing processing executed when the VR system in FIG. 8 synchronizes the virtual space;
  • FIG. FIG. 9 is a schematic diagram showing the state of the virtual space when a trigger event occurs in the VR system of FIG. 8;
  • FIG. 9 is a schematic diagram showing the state of the virtual space after a trigger event has occurred in the VR system of FIG. 8;
  • the schematic diagram which shows schematic structure of the MR system which concerns on 3rd Embodiment.
  • FIG. 15 is a block diagram showing the configuration of the processing unit of the MR system of FIG. 14;
  • FIG. 16 is a flow chart showing processing executed by the MR system of FIG. 15 in normal use;
  • FIG. 16 is a schematic diagram showing the states of the real space and the composite space in normal use of the MR system of FIG. 15;
  • FIG. 16 is a flow chart showing processing executed when the MR system of FIG. 15 synchronizes complex spaces;
  • FIG. FIG. 16 is a schematic diagram showing the state of the virtual space when a trigger event occurs in the MR system of FIG. 15;
  • FIG. 16 is a schematic diagram showing the state of the virtual space after a trigger event has occurred in the VR system of FIG. 15;
  • a virtual space sensation system according to the first embodiment corresponding to the first invention a virtual space sensation system according to the second embodiment corresponding to the second invention, and a virtual space sensation system according to the third embodiment
  • a complex spatial sensation system that
  • the VR system S1 is for a first user U1 and a second user U2 (hereinafter collectively referred to as "users U") existing together in a predetermined area (for example, one room, etc.) of the physical space RS.
  • One virtual space VS corresponding to the area is made to recognize that they exist together via the first avatar A1 corresponding to the first user U1 and the second avatar A2 corresponding to the second user U2 ( See Fig. 4, etc.).
  • the virtual space experience system of the first invention is not limited to such a configuration, and the number of users may be three or more.
  • a VR system S1 includes a plurality of markers 1 attached to a user U existing in a physical space RS, and a camera 2 that captures the user U (strictly speaking, the marker 1 attached to the user U). , a server 3 that determines the image and sound of the virtual space VS (see FIG. 4, etc.), and a head-mounted display (hereinafter referred to as “HMD 4”) that allows the user to recognize the determined image and sound. .
  • HMD 4 head-mounted display
  • the camera 2, server 3, and HMD 4 can mutually transmit and receive information wirelessly, such as through the Internet network, public lines, and short-range wireless communication. However, any one of them may be configured to transmit and receive information to and from each other by wire.
  • a plurality of markers 1 are attached to the user U's head, both hands, and both feet via the HMD 4, gloves, and shoes that the user U wears.
  • the plurality of markers 1 are used for recognizing the coordinates and posture of the user U in the physical space RS, as well as actions (for example, movement of coordinates, changes in posture including orientation, etc.). be. Therefore, the attachment position of the marker 1 may be changed as appropriate according to other devices that configure the VR system S1.
  • the camera 2 is installed so that the user U's operable range (that is, the range in which the user U can move, change the posture, etc.) in the physical space RS where the user U exists can be photographed from multiple directions.
  • the user U's operable range that is, the range in which the user U can move, change the posture, etc.
  • the server 3 recognizes the sign 1 from the image captured by the camera 2, and recognizes the coordinates and orientation of the user U based on the position of the recognized sign 1 in the physical space RS. Also, the server 3 determines images and sounds of the virtual space VS to be recognized by the user U, based on the coordinates and orientation.
  • the HMD 4 is an environment output device that allows the user to recognize the environment of the virtual space VS.
  • the HMD 4 is worn on the user's U head.
  • the HMD 4 includes a monitor 40 for allowing the user U to recognize the image of the virtual space VS determined by the server 3, and a speaker for allowing the user U to recognize the sound of the virtual space VS determined by the server 3. 41 (see FIG. 2).
  • the VR system S1 When experiencing the virtual space VS using the VR system S1, the user U recognizes only the images and sounds of the virtual space VS, and is made to recognize that the user U himself exists in the virtual space VS. That is, the VR system S1 is configured as a so-called immersive system.
  • the virtual space experience system of the first invention is not limited to such a configuration.
  • the number and arrangement of signs and cameras different from the above configuration may be used.
  • the posture and coordinates of the user may be recognized by recognizing feature points from the user's image itself without using the mark.
  • a device that recognizes only the user's coordinates may be used instead of the motion capture device.
  • a sensor such as GPS may be installed in the HMD, and the user's coordinates, posture, etc. may be recognized based on the output from the sensor. Also, such a sensor may be used in combination with the motion capture device as described above.
  • the server 3 is composed of one or more electronic circuit units including CPU, RAM, ROM, interface circuits, and the like. As shown in FIG. 2, the server 3 includes a virtual environment generation unit 30, a user action recognition unit 31, and an avatar action control unit 32 as functions (processing units) implemented by the implemented hardware configuration or programs. , an avatar environment determination unit 33 , a user environment determination unit 34 , and a trigger event recognition unit 35 .
  • the virtual environment generation unit 30 has a virtual space generation unit 30a and an avatar generation unit 30b.
  • the virtual space generator 30a generates a virtual space VS corresponding to the physical space RS in which the user U exists. Specifically, the virtual space generator 30a generates images that are the background of the virtual space VS and objects that exist in the virtual space VS, and sounds related to those images.
  • the virtual space sensory system is provided with a configuration that realizes a predetermined feel (for example, a cushion that changes in hardness, etc.) and that generates a predetermined smell.
  • the virtual space generation unit may generate the virtual space using not only the images and sounds but also their touches and smells.
  • the avatar generation unit 30b generates a first avatar A1 corresponding to the first user U1 and a second avatar A2 corresponding to the second user U2 in the virtual space VS (see FIG. 4, etc.).
  • the first avatar A1 and the second avatar A2 (hereinafter collectively referred to as "avatar A") move in the virtual space VS in response to the corresponding user U's motion in the real space RS.
  • the user motion recognition unit 31 recognizes image data of the user U including the sign 1 captured by the camera 2, and recognizes the motion of the user U in the physical space RS based on the image data.
  • the user action recognition section 31 has a user posture recognition section 31a and a user coordinate recognition section 31b.
  • the user posture recognition unit 31a extracts the marker 1 from the input image data of the user U, and recognizes the posture including the orientation of the user U based on the extraction result.
  • the user coordinate recognition unit 31b extracts the marker 1 from the input image data of the user U, and recognizes the coordinates of the user U based on the extraction result.
  • the avatar action control unit 32 gives characteristics to the avatar A generated by the avatar generation unit 30b.
  • the properties are assigned when the VR system S1 is started to be used (for example, when the type of avatar A to be used is selected).
  • its characteristics may change during use of the VR system S1. For example, it is changed when a trigger event, which will be described later, is recognized.
  • the "characteristic” is a characteristic that defines the action of the avatar corresponding to the user in response to the user's action, and there are multiple types. Specifically, the characteristics are the size of the avatar in the virtual space VS, and the functions of the avatar (for example, movable speed, movable range, etc.).
  • the avatar motion control unit 32 also recognizes the posture of the user U in the physical space RS recognized by the user posture recognition unit 31a, the coordinates of the user U in the physical space RS recognized by the user coordinate recognition unit 31b, and the assigned Based on the characteristics, the action of the avatar A corresponding to the user U in the virtual space VS (for example, movement of coordinates, change of posture including orientation, etc.) is determined.
  • the avatar environment determining unit 33 determines the environment of avatar A in the virtual space VS based on the result of avatar A's action (for example, coordinates, posture, etc. at that time).
  • the avatar's "environment” in this embodiment refers to what affects the avatar in the virtual space.
  • the environment of the avatar includes the state of objects existing in the virtual space, physical laws (speed of time flow, magnitude of gravity, etc.) in the virtual space VS.
  • the user environment determination unit 34 determines the environment (image and sound) of the virtual space RS to be recognized by the user U corresponding to the avatar A through the monitor 40 and the speaker 41 of the HMD 4. .
  • the "environment to be recognized" by the user refers to the environment of the virtual space that the user recognizes through the five senses.
  • the environment of the user U is images, sounds, etc. of the virtual space that the user recognizes.
  • the "image of the virtual space” includes, in addition to the image of the background of the virtual space, the image of another avatar, the image of an object that exists only in the virtual space, and the image that exists in the virtual space corresponding to the real space. Images of objects, etc. are included.
  • the trigger event recognition unit 35 recognizes that a predetermined trigger event has occurred when a condition predetermined by the system designer is satisfied.
  • the "trigger event” may be one that the user is unaware of. Therefore, as a trigger event, for example, events caused by user actions such as the user performing a predetermined action in the real space (that is, the avatar corresponding to the user performing a predetermined action in the virtual space) are also applicable. In addition, it also applies to events that are not caused by the user's actions, such as the passage of a predetermined period of time.
  • each processing unit that constitutes the virtual space experience system of the first invention is not limited to the configuration as described above.
  • part of the processing unit provided in the server 3 in this embodiment may be provided in the HMD 4.
  • a plurality of servers may be used, or the servers may be omitted and the CPUs mounted on the HMDs may cooperate with each other.
  • a speaker other than the speaker mounted on the HMD may be provided.
  • devices that affect sight and hearing devices that affect smell and touch such as generating smells, wind, etc. according to the virtual space may also be included.
  • the virtual environment generation unit 30 of the server 3 generates the virtual space VS, the first avatar A1 and the second avatar A2 (FIG. 3/STEP 100).
  • the virtual space generation unit 30a of the virtual environment generation unit 30 generates an image that serves as the background of the virtual space VS and various objects existing in the virtual space VS. Also, the avatar generation unit 30b of the virtual environment generation unit 30 generates a first avatar A1 corresponding to the first user U1 and a second avatar A2 corresponding to the second user U2.
  • the avatar action control unit 32 of the server 3 assigns the first property to the first avatar A1 and the second property to the second avatar A2 (FIG. 3/STEP 101).
  • the characteristics given to avatar A are linked to the shape of avatar A.
  • the shape of the first avatar A1 is an anthropomorphic shape of a turtle, and the first avatar A1 is given a first characteristic corresponding to the shape.
  • the first characteristic is that it is larger than a predetermined object (for example, grass shown in FIG. 4) placed in the virtual space VS, and the corresponding user's motion in the real space RS is reflected in the avatar's motion. It is a characteristic that includes a characteristic that it takes a long time to complete (that is, the reflection speed of the operation is slow).
  • the shape of the second avatar A2 is an anthropomorphic shape of a rabbit, and the second avatar A2 is given a second characteristic corresponding to the shape.
  • the second characteristic is that it is smaller than a predetermined object placed in the virtual space VS, and that the corresponding user's motion in the real space RS is reflected in the avatar's motion in a short time (that is, the motion is reflected quickly).
  • the sizes of the avatars A that is, the height of the viewpoint
  • the characteristic of the reflection speed of the action of the corresponding avatar A is different.
  • the environment of the scenery change speed experienced by the first user U1 as A1 is different from the environment of the scenery change speed experienced by the second user U2 as the second avatar A2 (that is, the relative movement speed of the object). It will be.
  • the avatar action control unit 32 of the server 3 determines the action of the first avatar A1 based on the action of the first user U1 and the characteristics given to the first avatar A1, and the action of the second user U2 and The action of the second avatar A2 is determined based on the characteristics assigned to the second avatar A2 (FIG. 3/STEP 102).
  • actions recognized by the user action recognition unit 31 of the server 3 based on image data captured by the camera 2 are used.
  • the avatar environment determination unit 33 of the server 3 determines the environment of the first avatar A1 based on the motion of the first avatar A1, and determines the environment of the second avatar A2 based on the motion of the second avatar A2. Determine (FIG. 3/STEP 103).
  • the avatar environment determining unit 33 determines the environment of the change speed of the scenery in the virtual space VS based on the characteristics of the action process of the avatar A and the reflection speed of the action of the avatar A, and the characteristics of the size of avatar A, determine the environment of landscape that avatar A can overlook.
  • the user environment determination unit 34 of the server 3 determines the environment to be recognized by the first user U1 based on the environment of the first avatar A1, and determines the environment to be recognized by the second user U2 as the environment of the second avatar A2. It is determined based on the environment (FIG. 3/STEP 104).
  • the user environment determining unit 34 determines the image and sound of the virtual space VS representing the environment of the first avatar A1 as the environment to be recognized by the first user U1, and determines the environment to be recognized by the second user U2. , determine the images and sounds of the virtual space VS representing the environment of the second avatar A2.
  • the HMD 4 worn by the user U outputs the determined environment (FIG. 3/STEP 105).
  • the HMD 4 causes the monitor 40 mounted on the HMD 4 to display the determined image, and causes the speaker 41 mounted on the HMD 4 to generate the determined sound.
  • the user action recognition unit 31 of the server 3 determines whether or not the first user U1 or the second user U2 has performed some action (FIG. 3/STEP 106).
  • the process returns to STEP102, and the processes after STEP102 are executed again.
  • the server 3 determines whether or not it has recognized a signal instructing the end of the process (Fig. 3/ STEP 107).
  • a plurality of objects including a first avatar A1 corresponding to the first user U1 and a second avatar A2 corresponding to the second user U2 are installed in the virtual space VS.
  • the first user U1 and the second user U2 can visualize themselves via the corresponding first avatars A1 and second avatars A2 by the images displayed and the sounds generated by the HMDs 4 they wear. It comes to recognize that it exists in space VS and can move freely.
  • the first avatar A1 is given the first property
  • the second avatar A2 is given the second property different from the first property
  • a user may want to experience the environment of a user other than himself, or a user may want a user other than himself to experience his own environment.
  • the first avatar A1 is given a first property that it is larger than a predetermined object (for example, grass shown in FIG. 4) placed in the virtual space VS.
  • the avatar A2 is given a second property that it is smaller than a predetermined object placed in the virtual space VS.
  • the object for example, the first object O1, which is a distant building object, cannot be visually recognized by the second avatar A2 (and by extension, the corresponding second user U2).
  • the first user U1 corresponding to the first avatar A1 may desire to view the first object O1 viewed by the second user U2 in the same manner as the second user U2. Sometimes.
  • the characteristics imparted to the first avatar A1 and the characteristics imparted to the second avatar A2 are processed by the processing described below. It is configured to be able to execute a process of matching the characteristics of the
  • the trigger event recognition unit 35 of the server 3 determines whether or not the trigger event has been recognized (FIG. 5/STEP 200).
  • the trigger event recognition unit 35 recognizes the action of the first avatar A1 corresponding to the first user U1 in the virtual space VS, and the action is defined in advance as the trigger event. It is determined whether or not it corresponds to the predetermined operation.
  • the trigger event recognition unit 35 determines whether or not the recognized trigger event is a predetermined trigger event (FIG. 5/STEP 201).
  • the action of one of the first avatar A1 corresponding to the first user U1 and the second avatar A2 corresponding to the second user U2 touching the other is set as a predetermined trigger event. It is
  • the avatar motion control unit 32 synchronizes the characteristics of the first avatar A1 and the characteristics of the second avatar A2 (FIG. 5/STEP202).
  • the characteristics of the first avatar A1 are maintained as the first characteristics, and the characteristics of the second avatar A2 are changed from the second characteristics to the same first characteristics as the first avatar A1.
  • the characteristics of Avatar A are linked to the shape of Avatar A. Therefore, as shown in FIG. 7, when the process of STEP 202 is executed, the shape of the second avatar A2 changes from the shape of an anthropomorphic rabbit to the shape of an anthropomorphic turtle like the first avatar A1. .
  • the avatar action control unit 32 controls the characteristics of the first avatar A1 and the second avatar A2 according to the type of the recognized trigger event. change at least one of the characteristics of (FIG. 5/STEP 203).
  • the action of the first avatar A1 or the second avatar A2 pressing a switch-type object is set as another trigger event, and the recognized trigger event is the other trigger event. If there is, the avatar action control unit 32 performs a process of once again giving the characteristics at the start of use of the VR system S1 to the avatar A that caused the other trigger event.
  • the characteristics up to that point are maintained, and if the trigger event is recognized (YES in STEP 200), processing to change the characteristics of the avatar. (processing of STEP 202 or STEP 203), and then the avatar motion control unit 32 of the server 3 controls the motion of the first avatar A1 based on the motion of the first user U1 and the characteristics given to the first avatar A1. Then, based on the behavior of the second user U2 and the characteristics given to the second avatar A2, the behavior of the second avatar A2 is determined (FIG. 5/STEP 204).
  • the avatar environment determination unit 33 of the server 3 determines the environment of the first avatar A1 based on the motion of the first avatar A1, and based on the motion of the second avatar A2, The environment of the second avatar A2 is determined (FIG. 5/STEP 205).
  • the first avatar A1 can visually recognize an object that could not be visually recognized due to its small size.
  • the user environment determining unit 34 of the server 3 determines the environment to be recognized by the first user U1 based on the environment of the first avatar A1, and determines the environment to be recognized by the second user U2. is determined based on the environment of the second avatar A2 (FIG. 5/STEP 206).
  • the HMD 4 worn by the user U outputs the determined environment (FIG. 5/STEP 207).
  • the user action recognition unit 31 of the server 3 determines whether or not the first user U1 or the second user U2 has performed some action (FIG. 5/STEP 208).
  • the process returns to STEP200, and the processes after STEP200 are executed again.
  • the server 3 has recognized a signal instructing the end of the process as in the process in STEP 107. It is determined whether or not (FIG. 5/STEP 209).
  • the first characteristic is given to the first avatar A1, and based on the action of the first avatar A1, , determines the environment of the first avatar A1, gives a second characteristic different from the first characteristic to the second avatar A2, and based on the action of the second avatar A2, Determine A2's environment.
  • the first characteristic is given to the first avatar A1, and the environment of the first avatar A1 is determined based on the action of the first avatar A1, Similarly, the first property is given to the second avatar A2, and the environment of the second avatar A2 is determined based on the behavior of the second avatar A2.
  • the characteristics of the first avatar A1 corresponding to the first user U1 and the characteristics of the second avatar A2 corresponding to the second user U2 are different.
  • the characteristics of the first avatar A1 corresponding to the first user U1 and the characteristics of the second avatar A2 corresponding to the second user U2 are both the same. .
  • the process of synchronizing the environment of the avatar A is executed, and the virtual space VS that is recognized by the first user U1 and the second user U2 is displayed. Since the environments become similar, the first user U1 and the second user U2 who have experienced different environments can now experience similar environments.
  • an action in which one of the first avatar A1 corresponding to the first user U1 and the second avatar A2 corresponding to the second user U2 touches the other is set as a predetermined trigger event.
  • the "predetermined action” that becomes a predetermined trigger event is not limited to the action of one of the first avatar and the second avatar touching the other. Any action performed by one of the second avatars relative to the other may be acceptable.
  • the predetermined action is an action in which one avatar touches the other avatar, an action in which one avatar moves within a predetermined range relative to the other avatar, and one avatar moves in a complex space.
  • an action in which one avatar touches the other avatar an action in which one avatar moves within a predetermined range relative to the other avatar, and one avatar moves in a complex space.
  • a ghost that acts in response to the user's action but does not serve as a reference may also be generated.
  • the process of synchronizing the characteristics of the avatar may be executed even when the ghost performs a predetermined action.
  • the predetermined trigger event in the first invention does not necessarily have to be a predetermined action performed by one of the first avatar and the second avatar on the other of the first avatar and the second avatar. Therefore, for example, when the user gives an instruction to change the characteristics via an interface provided separately, the fact that the instruction is given may be set as the predetermined trigger event.
  • the characteristics of the second avatar A2 are synchronized with the first characteristics given to the first avatar A1, and the shape of the second avatar A2 is changed to the first characteristic.
  • 1 avatar A1 is changed to a shape similar to that of A1. This is because the characteristics given to avatar A are linked to the shape of avatar A in this embodiment.
  • the first invention only needs to synchronize the characteristics of the avatar when a predetermined trigger event is recognized, and the shape of the avatar does not necessarily have to be synchronized with the characteristics.
  • avatar features eg, size only
  • the characteristics of the second avatar A2 are synchronized with the first characteristics given to the first avatar A1. That is, they have the same characteristics.
  • the process of synchronizing properties is not limited to changing the properties of the second avatar in that way. Therefore, for example, the characteristics of the first avatar may be synchronized with the second characteristics given to the second avatar.
  • the process of synchronizing the characteristics is not limited to the process of completely matching the characteristics. Therefore, for example, it may be a process of synchronizing some of a plurality of characteristics. Specifically, out of the movable speed and the size, only the size may be synchronized. Alternatively, for example, the process may be a process of approximating values that define characteristics. Specifically, it may be a process of bringing the values of movable velocities close to each other.
  • a corresponding trigger event may be defined for each of these processings. For example, when the second avatar touches the first avatar with the right hand, the characteristics of the second avatar are synchronized with the characteristics given to the first avatar, and when the second avatar touches the first avatar with the left hand, Alternatively, the properties of the first avatar may be synchronized with the properties imparted to the second avatar.
  • avatar characteristics are adopted as avatar characteristics, but avatar characteristics are not limited to such configurations, and , the characteristics that define the behavior of the avatar corresponding to the user.
  • the visible range of visible light may be defined as a function of the avatar. Specifically, one avatar is set to be able to see only light in the normal visible light wavelength band, and the other avatar is set to be able to see infrared light in addition to light in the normal visible light wavelength band. If so, each avatar's environment (eg, how objects look, etc.) will be different.
  • a VR system S2 which is a virtual space sensation system according to the second embodiment, will be described below with reference to FIGS. 8 to 13.
  • FIG. 8 A VR system S2 which is a virtual space sensation system according to the second embodiment
  • the VR system S2 of the present embodiment is the same as the VR system S1 of the first embodiment except that only one type of characteristic is given to the avatar and that a plurality of virtual spaces are generated. It has a configuration of Therefore, in the following description, the same reference numerals are given to the same or corresponding configurations as those of the VR system S1 of the first embodiment, and detailed description thereof will be omitted.
  • the server 3 is composed of one or more electronic circuit units including CPU, RAM, ROM, interface circuits, and the like. As shown in FIG. 8, the server 3 includes a virtual environment generation unit 30, a user action recognition unit 31, and an avatar action control unit 32 as functions (processing units) realized by the implemented hardware configuration or programs. , an avatar environment determination unit 33 , a user environment determination unit 34 , and a trigger event recognition unit 35 .
  • the virtual environment generation unit 30 has a virtual space generation unit 30a and an avatar generation unit 30b.
  • the virtual space generator 30a generates a first virtual space VS1 corresponding to the physical space RS (see FIG. 1) in which the user U exists, and a second virtual space VS1 independent of the first virtual space VS1 corresponding to the physical space RS. Create space VS2.
  • the virtual space generation unit 30a becomes the background of the first virtual space VS1 and the second virtual space VS2 (hereinafter collectively referred to as "virtual space VS") and objects existing in the virtual space VS. Generate images and sounds associated with those images.
  • the avatar generation unit 30b generates a first avatar A1 corresponding to the first user U1 and a second ghost G2 corresponding to the second user U2 in the first virtual space VS1, and generates a second ghost G2 corresponding to the second user U2 in the second virtual space VS2.
  • a second avatar A2 corresponding to the second user U2 and a first ghost G1 corresponding to the first user U1 are generated (see FIG. 9, etc.).
  • a “ghost” is one of the avatars that act in response to the user, and does not affect the environment perceived by the corresponding user.
  • the ghost G is used to indicate the existence, actions, etc. of the corresponding user U to the users U other than the user U corresponding to the ghost G.
  • the first avatar A1 and second avatar A2 (hereinafter collectively referred to as “avatar A”), and the first ghost G1 and second ghost G2 (hereinafter collectively referred to as “ghost G”) are , operate in the first virtual space VS1 or the second virtual space VS2 corresponding to the action of the corresponding user U in the physical space RS.
  • the virtual space VS in which the first avatar A1 and the second avatar A2 exist may change during the use of the VR system S2. For example, it is changed when a predetermined trigger event, which will be described later, is recognized. Also, the first ghost G1 and the second ghost G2 may be erased along with the change.
  • the "information indicating which user it corresponds to” is not limited to such indirect information. As such, it may be direct information such as, for example, a message that is displayed all the time or at the user's request.
  • the avatar action control unit 32 gives characteristics to the avatar A generated by the avatar generation unit 30b.
  • the properties are assigned when the use of the VR system S2 is started (for example, when the type of avatar A to be used is selected).
  • the avatar motion control unit 32 also recognizes the posture of the user U in the physical space RS recognized by the user posture recognition unit 31a, the coordinates of the user U in the physical space RS recognized by the user coordinate recognition unit 31b, and the assigned Based on the characteristics, the actions of the avatar A and the ghost G corresponding to the user U in the virtual space VS (for example, movement of coordinates, change of posture including orientation, etc.) are determined.
  • the avatar motion control unit controls the user's motion and the characteristics of the avatar as well as the virtual space. may be configured to refer to the characteristics of
  • the characteristics of the virtual space are set so that the effect of gravity on the avatar in the virtual space is different from the effect of gravity on the user in the virtual space.
  • the characteristics of the virtual space are reflected.
  • the avatar environment determining unit 33 determines the environment of avatar A in the virtual space VS based on the result of avatar A's action (for example, coordinates, posture, etc. at that time).
  • the avatar's “environment” in this embodiment refers to what affects the avatar in the virtual space.
  • the avatar's environment is the type of existing virtual space itself (whether it is on the ground or in the water).
  • the virtual environment generation unit 30 of the server 3 generates a first virtual space VS1 and a second virtual space VS2, a first avatar A1 and a second avatar A2, and a first ghost G1 and a second ghost G2. is generated (FIG. 9/STEP 300).
  • the virtual space generation unit 30a of the virtual environment generation unit 30 generates an image that serves as the background of the virtual space VS and various objects existing in the virtual space VS.
  • a background image and objects are generated so that the first virtual space VS1 is a virtual space imitating the sea, and a background image and objects are generated so that the second virtual space VS2 is a virtual space imitating mountains. images and objects are generated.
  • the avatar generation unit 30b of the virtual environment generation unit 30 generates a first avatar A1 corresponding to the first user U1 and a second ghost G2 corresponding to the second user U2 in the first virtual space VS1.
  • a second avatar A2 corresponding to U2 and a first ghost G1 corresponding to the first user U2 are generated in the second virtual space VS2.
  • the avatar action control unit 32 of the server 3 assigns the first property to the first avatar A1 and the second property to the second avatar A2 (FIG. 9/STEP 301).
  • the characteristics imparted to the first avatar A1 and the characteristics imparted to the second avatar A2 are the same, and are not changed thereafter.
  • the characteristics given to the first avatar A1 and the second avatar A2 may be different from each other as in the first embodiment and may be changed thereafter.
  • the avatar motion control unit 32 of the server 3 determines the motion of the first avatar A1 and the first ghost G1 based on the motion of the first user U1 and the characteristics given to the first avatar A1, The actions of the second avatar A2 and the second ghost G2 are determined based on the action of the user U2 and the characteristics given to the second avatar A2 (FIG. 9/STEP302).
  • the avatar environment determination unit 33 of the server 3 determines the environment of the first avatar A1 based on the motion of the first avatar A1 and the situation of the first virtual space VS1, and the motion of the second avatar A2 and the second virtual space VS1.
  • the environment of the second avatar A2 is determined based on the situation of the virtual space VS2 (FIG. 9/STEP303).
  • the user environment determination unit 34 of the server 3 determines the environment to be recognized by the first user U1 based on the environment of the first avatar A1, and determines the environment to be recognized by the second user U2 as the environment of the second avatar A2. It is determined based on the environment (FIG. 9/STEP 304).
  • the user environment determination unit 34 determines the image and sound of the first virtual space VS1 representing the environment of the first avatar A1 as the environment to be recognized by the first user U1, and makes the second user U2 recognize the environment. As the environment, the image and sound of the second virtual space VS2 representing the environment of the second avatar A2 are determined.
  • the HMD 4 worn by the user U outputs the determined environment (FIG. 9/STEP 305).
  • the HMD 4 causes the monitor 40 mounted on the HMD 4 to display the determined image, and causes the speaker 41 mounted on the HMD 4 to generate the determined sound.
  • the user action recognition unit 31 of the server 3 determines whether or not the first user U1 or the second user U2 has performed some action (FIG. 9/STEP 306).
  • the server 3 determines whether or not it has recognized a signal instructing the end of the process (Fig. 9/ STEP 307).
  • a plurality of objects including a first avatar A1 corresponding to the first user U1 and a second ghost G2 corresponding to the second user U2 are installed in the first virtual space VS1.
  • a plurality of objects including a second avatar A2 corresponding to the second user U2 and a first ghost G1 corresponding to the first user U1 are installed.
  • the first user U1 and the second user U2 through the corresponding first avatar A1 and second avatar A2, respectively, by the image displayed and the sound generated by the HMD 4 worn by the first user U1 and the second user U2, become the second user. It exists in the first virtual space VS1 or the second virtual space VS2 and recognizes that it can move freely.
  • one of the first user U1 and the second user U2 is in a state of being able to communicate with the other of the first user U1 and the second user U2 via message transmission or the like.
  • the first user U1 and the second user U2 are mutually aware that they are using the VR system S2 at the same time, although they are in different virtual spaces VS.
  • the first avatar A1 exists in the first virtual space VS1, and the second avatar A2 exists in the second virtual space VS2. Existing.
  • the first user A desire may arise for one of U1 and second user U2 to experience the environment of the other, or for one to allow the other to experience its own environment.
  • the first user U1, through the first avatar A1 can create an object existing only in the first virtual space VS1 (for example, the second object O2, which is a dolphin swimming in the sea). ), the second user U2 may not be able to see the object.
  • the second object O2 which is a dolphin swimming in the sea.
  • the first user U1 informs the second user U2 of the existence of the second object O2 by using a message function or the like.
  • a desire to visually recognize the visually recognizing object in the same way as the first user U1 may arise.
  • the first virtual space VS1 in which the first avatar A1 exists and the second avatar are created by the processing described below. It is configured to be able to execute a process of synchronizing with the second virtual space VS2 in which A2 exists.
  • the trigger event recognition unit 35 of the server 3 determines whether or not a predetermined trigger event has been recognized (FIG. 11/STEP 400).
  • the trigger event recognition unit 35 recognizes the motion of the avatar A corresponding to the user U in the virtual space VS, and determines whether or not the motion corresponds to a predetermined motion defined in advance as a predetermined trigger event. to judge.
  • the action of the avatar A touching the ghost G is set as a predetermined trigger event.
  • the avatar generation unit 30b of the virtual environment generation unit 30 of the server 3 When a predetermined trigger event is recognized (YES in STEP 400), the avatar generation unit 30b of the virtual environment generation unit 30 of the server 3 generates the first virtual space VS1 in which the first avatar A1 exists and the second avatar A2. is synchronized with the second virtual space VS2 in which is present (FIG. 11/STEP 401).
  • the avatar generation unit 30b erases the second ghost G2 and generates the second avatar A2 in the first virtual space VS1.
  • the avatar generator 30b erases the second avatar A2 and the first ghost G1 in the second virtual space VS2.
  • both the first avatar A1 and the second avatar A2 are present in the first virtual space VS1.
  • the second virtual space VS2 is in a state where avatar A and ghost G do not exist after the synchronization process is executed.
  • a trigger event other than a predetermined trigger event for example, when avatar A presses a switch-type object
  • the avatar A may be generated again in the second virtual space VS2.
  • the first virtual space VS1 and the second virtual space VS2 can be synchronized.
  • the process of generating the second avatar A2 and the process of erasing the second ghost G2 in the first virtual space VS1 and the process of generating the first avatar A1 and erasing the first ghost G1 in the second virtual space VS2.
  • the process may or may not be performed.
  • the avatar action control unit 32 of the server 3 After executing the process of synchronizing the virtual space VS (the process of STEP 401), the avatar action control unit 32 of the server 3 performs the first The action of the avatar A1 is determined, and the action of the second avatar A2 is determined based on the action of the second user U2 and the characteristics given to the second avatar A2 (FIG. 11/STEP 402).
  • the avatar environment determination unit 33 of the server 3 determines the environment of the first avatar A1 based on the motion of the first avatar A1 and the situation of the first virtual space VS1, and the motion of the second avatar A2 and the first virtual space VS1.
  • the environment of the second avatar A2 is determined based on the situation of the virtual space VS1 (FIG. 11/STEP 403).
  • the avatar action control unit 32 of the server 3 applies the action of the first user U1 and the first avatar A1. Based on the characteristics given, the behavior of the first avatar A1 and the first ghost G1 is determined, and based on the behavior of the second user U2 and the characteristics given to the second avatar A2, the second avatar A2 and the second ghost Determine the operation of G2 (FIG. 11/STEP 404).
  • the avatar environment determination unit 33 of the server 3 determines the environment of the first avatar A1 based on the motion of the first avatar A1 and the situation of the first virtual space VS1, and the motion of the second avatar A2 and the second virtual space VS1.
  • the environment of the second avatar A2 is determined based on the situation of the virtual space VS2 (FIG. 11/STEP405).
  • the user environment determination unit 34 of the server 3 determines the environment to be recognized by the first user U1 as the environment of the first avatar A1.
  • the environment is determined based on the environment, and the environment to be recognized by the second user U2 is determined based on the environment of the second avatar A2 (FIG. 11/STEP 406).
  • the user environment determination unit 34 determines the image and sound of the first virtual space VS1 representing the environment of the first avatar A1 as the environment to be recognized by the first user U1, and makes the second user U2 recognize the environment. As the environment, the image and sound of the first virtual space VS1 representing the environment of the second avatar A2, or the image or sound of the second virtual space VS2 is determined.
  • the HMD 4 worn by the user U outputs the determined environment (FIG. 11/STEP 407), similar to the process in STEP 305.
  • the user action recognition unit 31 of the server 3 determines whether or not the first user U1 or the second user U2 has performed some action (FIG. 11/STEP 408).
  • the trigger event recognition unit 35 performs the process of synchronizing the first virtual space VS1 and the second virtual space VS2 (STEP401 process)) has been executed (FIG. 11/STEP 409).
  • the trigger event recognizing unit 35 determines based on whether or not a predetermined trigger event serving as a key to executing the process of synchronizing the first virtual space VS1 and the second virtual space VS2 has been recognized. , to determine whether the process has been executed.
  • the server 3 determines whether or not it has recognized a signal instructing the end of the process (Fig. 11/STEP 410).
  • the VR system S2 ends the current process.
  • the first avatar A1 is generated in the first virtual space VS1, and the environment of the first avatar A1 is determined.
  • the second avatar A2 is generated in the second virtual space VS2, and the environment of the second avatar A2 is determined.
  • the first avatar A1 and the second avatar A2 are generated in the first virtual space VS1, and the environments of the first avatar A1 and the second avatar A2 are determined.
  • the virtual space in which the first avatar A1 corresponding to the first user U1 exists (for example, the first virtual space VS1 in this embodiment) and the second user U2 is independent of the virtual space (for example, the second virtual space VS2 in this embodiment) in which the second avatar A2 corresponding to . Therefore, the environment of Avatar A becomes independent as before. As a result, the environment experienced by the user U becomes independent as in the conventional case.
  • the virtual space in which the first avatar A1 corresponding to the first user U1 exists and the virtual space in which the second avatar A1 corresponding to the second user U2 exists are: It becomes the same virtual space (for example, the first virtual space VS1 in this embodiment). Therefore, Avatar A's environment will match or be similar. As a result, the environment experienced by the user U also matches or is similar.
  • the process of synchronizing the environment of the avatar A is executed, and the virtual space VS recognized by the first user U1 and the second user U2 is displayed. Since the environments become similar, the first user U1 and the second user U2 who have experienced different environments can now experience similar environments.
  • the "predetermined action” that is a predetermined trigger event is not limited to the action of the avatar touching the ghost, but any action that the avatar performs with reference to the ghost. good.
  • the predetermined action is an action in which the avatar touches a ghost, an action in which the avatar moves within a predetermined range based on the ghost, an avatar operating an object existing in the virtual space,
  • An action of selecting a ghost as the target for example, an action of photographing a ghost with a camera-type object, etc. may be used.
  • the trigger event in the second invention must be a predetermined action performed by the avatar corresponding to one of the first user and the second user to the ghost corresponding to the other of the first user and the second user. no. Therefore, for example, when the user gives an instruction to change the characteristics via an interface provided separately, the fact that the instruction is given may be set as the trigger event.
  • the virtual space bodily sensation system of the second invention is not limited to such a configuration. It may be configured such that a process for synchronizing is also executed.
  • the MR system S3 includes a third user U3 and a fourth user U4 (first user and second user in the third invention; hereinafter collectively referred to as In this case, it is referred to as “user U”). This is for recognition (see FIG. 17, etc.).
  • the complex spatial sensation system of the third invention is not limited to such a configuration, and the number of users may be three or more.
  • the MR system S3 includes a user U existing in the physical space RS, a camera 5 that captures the physical space RS, and a server 6 that determines images and sounds of the complex space MS (see FIG. 4, etc.). , and a head-mounted display (hereinafter referred to as “HMD 7”) that allows the user to recognize the determined image and sound.
  • HMD 7 head-mounted display
  • the camera 5, server 6, and HMD 7 can mutually transmit and receive information wirelessly, such as through the Internet network, public lines, and short-range wireless communication. However, any one of them may be configured to transmit and receive information to and from each other by wire.
  • the camera 5 is installed in such a way that the user U in the real space RS in which the user U exists can photograph the operable range (that is, the range in which the user U can move the coordinates, change the posture including the orientation, etc.) from multiple directions.
  • the operable range that is, the range in which the user U can move the coordinates, change the posture including the orientation, etc.
  • the server 6 recognizes the feature points on the body of the user U from the image captured by the camera 5, and recognizes the coordinates and posture of the user U based on the positions of the recognized feature points in the physical space RS. Also, the server 6 recognizes the feature points of the physical space RS from the image captured by the camera 5, and recognizes the coordinates and orientation of a real object (for example, a bookshelf) existing in the physical space RS.
  • a real object for example, a bookshelf
  • the server 6 determines images and sounds of the complex space MS to be recognized by the user U, based on the coordinates and posture of the user U and feature points in the physical space RS.
  • the HMD 7 is an environment output device that allows the user to recognize the environment of the complex space MS.
  • the HMD 7 is worn on the user's U head.
  • the HMD 7 includes a monitor 70 for allowing the user U to recognize the image of the composite space MS determined by the server 6, and a speaker for allowing the user U to recognize the sound of the composite space MS determined by the server 6. 71 (see FIG. 15).
  • the user U When experiencing the complex space MS using the MR system S3, the user U is made to recognize the image and sound of the virtual object corresponding to the complex space MS to be experienced, together with the image and sound of the real space RS.
  • the virtual object is configured to perform a predetermined action (for example, movement of coordinates, etc.) in accordance with the user's U action.
  • the complex spatial sensation system of the third invention is not limited to such a configuration.
  • a motion capture device in addition to the configuration described above, a configuration in which the number and arrangement of cameras are different from those shown in the drawings may be used.
  • a marker attached to the user or a real object existing in the real space may be used.
  • a device that recognizes only the coordinates of the user may be used instead of the motion capture device.
  • a sensor such as GPS may be installed in the HMD, and the user's coordinates, posture, etc. may be recognized based on the output from the sensor. Also, such a sensor may be used in combination with the motion capture device as described above.
  • the server 6 is composed of one or more electronic circuit units including CPU, RAM, ROM, interface circuits, and the like. As shown in FIG. 15, the server 6 includes a real space recognition unit 60, a complex environment generation unit 61, and a user action recognition unit 62 as functions (processing units) realized by implemented hardware configurations or programs. , an object control unit 63 , a user environment determination unit 64 , and a trigger event recognition unit 65 .
  • the physical space recognition unit 60 recognizes the image data of the physical space RS captured by the camera 5, and recognizes the situation of the physical space RS based on the image data.
  • the situation of the physical space RS is, for example, the coordinates and attitudes of other users U and real objects existing in the physical space RS.
  • the complex environment generator 61 has an object generator 61a and a complex space generator 61b.
  • the object generation unit 61a generates a virtual object that exists in the complex space MS and is recognized by the user U (see FIG. 17, etc.).
  • the virtual objects include a third object O3 existing in a first complex space MS1 and a fourth object O4 existing in a second complex space MS2 (the first object and the second object in the third invention). included.
  • At least a part of the third object O3 and the fourth object O4 (hereinafter collectively referred to as "object O") is configured to move according to the user U's motion in the complex space MS.
  • the composite space generation unit 61b makes the third object O3 exist in the image of the physical space RS recognized by the physical space recognition unit 60 to generate the first composite space MS1.
  • the composite space generation unit 61b makes the fourth object O4 exist in the image of the physical space RS recognized by the physical space recognition unit 60 to generate the second composite space MS2.
  • the user motion recognition unit 62 recognizes the image data of the user U captured by the camera 5, and recognizes the motion of the user U in the physical space RS based on the image data.
  • the user action recognition section 62 has a user posture recognition section 62a and a user coordinate recognition section 62b.
  • the user posture recognition unit 62a extracts feature points such as the body of the user U from input image data of the user U, and recognizes the posture including the orientation of the user U based on the extraction result.
  • the user coordinate recognition unit 62b extracts the result of extracting feature points such as the body of the user U extracted from the input image data of the user U, and the state of the real space RS recognized by the real space recognition unit 60 (for example, the actual , etc.), the coordinates of the user U are recognized.
  • the object control unit 63 determines the position of the user U based on the posture of the user U in the physical space RS recognized by the user posture recognition unit 62a and the coordinates of the user U in the physical space RS recognized by the user coordinate recognition unit 62b. to control the motion of the object O existing in the complex space MS experienced by .
  • the object control unit 63 moves the coordinates of the object O in the composite space MS.
  • the user environment determination unit 64 determines the environment of the complex space MS to be recognized by the user U through the monitor 70 and speaker 71 of the HMD 7.
  • the "environment recognized by the user” refers to the environment of the complex space that the user recognizes through the five senses.
  • the user's environment is an image of an object existing in a complex space recognized by the user, a sound generated based on the object, and the like.
  • the user environment determining unit 64 recognizes the motion of the user U and the type of the composite space MS that the user U is allowed to experience, and based on the motion of the user U, determines which region of the composite space MS the user U Decide whether you want to experience it.
  • the trigger event recognition unit 65 recognizes that a predetermined trigger event has occurred when a condition predetermined by the system designer is satisfied.
  • the "trigger event” may be one that the user is unaware of. Therefore, the trigger event includes, for example, events caused by the user's actions such as the user performing a predetermined action in the real space, and events not caused by the user's actions such as the passage of a predetermined time.
  • each processing unit that constitutes the complex spatial sensation system of the third invention is not limited to the configuration as described above.
  • part of the processing unit provided in the server 6 in this embodiment may be provided in the HMD 7 .
  • a plurality of servers may be used, or the servers may be omitted and the CPUs mounted on the HMDs may cooperate with each other.
  • a speaker other than the speaker mounted on the HMD may be provided.
  • devices that affect sight and hearing devices that affect smell and touch, such as generating smells, wind, etc., according to a complex space may also be included.
  • the composite environment generation unit 61 of the server 6 generates the first composite space MS1 and the second composite space MS2 (FIG. 16/STEP 500).
  • the object generation unit 61a of the composite environment generation unit 61 generates a third object O3 to exist in the first composite space MS1 and a third object O3 to exist in the second composite space MS2. 4 Create an object O4.
  • the third object O3 is a potted flower object (hereinafter simply referred to as "flower O30") whose coordinates and orientation are controlled according to the motion of the user U who recognizes the first complex space MS1. ) and a first message board O31 added to another user U.
  • the first message board O31 displays the status of the user U to whom the first message board O31 is added (for example, the type of complex space MS being experienced, etc.).
  • the fourth object O4 is a book object (hereinafter simply referred to as "book O40" whose coordinates and orientation are controlled according to the motion of the user U who is aware of the second composite space MS2. ) and a second message board O41 added to another user U.
  • the second message board O41 displays the status of the user U to whom the second message board O41 is attached (for example, the type of complex space MS being experienced).
  • the complex space generation unit 61b of the complex environment generation unit 61 superimposes the third object O3 on the image of the physical space RS recognized by the physical space recognition unit 60 to generate the first complex space MS1.
  • the composite space generation unit 61b also superimposes the fourth object O4 on the image of the physical space RS recognized by the physical space recognition unit 60 to generate the second composite space MS2.
  • the user environment determination unit 64 of the server 6 determines the composite space MS to be recognized by the third user U3 and the composite space MS to be recognized by the fourth user U4 (FIG. 16/STEP 501).
  • the user environment determination unit 64 selects the composite space MS to be recognized by the user U based on an instruction from the user U input via a separately provided interface (not shown).
  • the method of determining the composite space to be recognized by the user is not limited to such a configuration, and may be appropriately set by the system designer. Therefore, for example, the complex space to be recognized by the user is automatically determined according to the characteristics of the user input in advance (physical characteristics such as the user's age, characteristics such as the purpose of use of the complex spatial sensation system, etc.). It may be configured as
  • the user environment determining unit 64 of the server 6 makes the third user U3 recognize the first composite space based on the state of the third user U3 and the state of the first composite space MS1 at the start of use of the MR system MS3.
  • the environment in MS1 is determined, and the environment in the second complex space MS2 to be recognized by the fourth user U4 is determined based on the state of the fourth user U4 and the situation of the second complex space MS2 at the start of use of the MR system MS3. (FIG. 16/STEP 502).
  • the user environment determining unit 64 as the environment to be recognized by the third user U3, based on the coordinates and orientation of the physical space RS of the third user U3, creates an image of a region to be recognized in the physical space RS, The image of the third object O3 in that area and the accompanying sound are determined.
  • the user environment determination unit 64 as the environment to be recognized by the fourth user U4, based on the coordinates and posture of the physical space RS of the fourth user U4, the image of the region to be recognized in the physical space RS, The image of the fourth object O4 and the accompanying sound are determined.
  • the HMD 7 worn by the user U outputs the determined environment (FIG. 16/STEP 503).
  • the HMD 7 causes the monitor 70 mounted on the HMD 7 to display the determined image, and the speaker 71 mounted on the HMD 7 to generate the determined sound.
  • the user action recognition unit 31 of the server 6 determines whether or not the third user U3 or the fourth user U4 has performed some action (FIG. 16/STEP 504).
  • the object control unit 63 of the server 6 determines whether or not the action of the user U is to operate the object O (FIG. 16/STEP 505).
  • the object control unit 63 determines whether or not the action of the user U corresponds to the predetermined action with respect to the object O.
  • the object control unit 63 controls the object O according to the type of action of the user U (FIG. 16/STEP506). .
  • the object control unit 63 changes the coordinates of the object O according to the content of the action.
  • the user environment determination unit 64 determines Based on the state of the third user U3 and the situation of the first composite space MS1 at that time, the environment in the first composite space MS1 to be recognized by the third user U3 is determined, and the state of the fourth user U4 at that time and the second composite space Based on the situation of the space MS2, the environment in the second composite space MS2 to be recognized by the fourth user U4 is determined (FIG. 16/STEP 507).
  • the HMD 7 executes processing similar to STEP 503 and outputs the determined environment (FIG. 16/STEP 508).
  • the server 6 After executing the process of outputting the determined environment (the process of STEP 508), or when the user U did not act (NO in STEP 504), the server 6 recognized a signal instructing the end of the process. (FIG. 16/STEP 509).
  • the MR system S3 recognizes the signal instructing the end (YES in STEP509), the MR system S3 ends the current process.
  • the third user U3 can create a first composite space MS1 in which the third object O3 exists in the physical space RS by means of the image displayed and the sound generated by the HMD 4 worn by the third user U3. is recognized, and the third object O3 can be manipulated in the first composite space MS1.
  • the fourth user U4 recognizes the second composite space MS2 in which the fourth object O4 exists in the physical space RS from the image displayed and the sound generated by the HMD 4 worn by the user, and recognizes the second composite space MS2. , the fourth object O4 can be operated.
  • the other of the third user U3 and the fourth user U4 is provided with the first message board, which is an object indicating the type of the complex space MS experienced by the third user U3 or the fourth user U4.
  • O31 or a second message board O41 is added.
  • the third user U3 and the fourth user U4 can mutually recognize that they are using the MR system S3 at the same time and, if so, which complex space MS they are experiencing. It's becoming
  • the third user U3 is experiencing the first composite space MS1
  • the fourth user U4 is experiencing the second composite space MS2. I feel it.
  • the third user U3 and the fourth user U4 A desire may arise for one to experience the environment of the other, or for one to allow the other to experience its own environment.
  • the third user U3, who is experiencing the first complex space MS1 arranges flowers O30, which are virtual objects, on shelves that actually exist in the physical space RS.
  • the fourth user U4 who is experiencing the second composite space MS2 is experiencing the second composite space MS2
  • the shelves that exist in common in the first composite space MS1 and the second composite space MS2 can be recognized, the flower O30, which is an object of the first composite space MS1 arranged there, cannot be recognized.
  • the processing described below is performed so that the first complex space MS1 experienced by the third user U3 and the fourth space It is configured to be able to execute a process of synchronizing with the second complex space MS2 experienced by the user U4.
  • the trigger event recognition unit 65 of the server 6 determines whether or not a predetermined trigger event has been recognized (FIG. 18/STEP 600).
  • the trigger event recognition unit 65 recognizes the motion of the user U in the complex space MS, and determines whether or not the motion corresponds to a predetermined motion defined in advance as a predetermined trigger event. In this embodiment, as shown in FIG. 19, an action in which one of the third user U3 and the fourth user U4 touches the other is set as a predetermined trigger event.
  • the user environment determining unit 64 of the server 6 synchronizes the type of complex space MS that the user U experiences (FIG. 18/STEP601).
  • the user environment determining unit 64 maintains the composite space MS experienced by the third user U3 as the first composite space MS1, and changes the composite space MS experienced by the fourth user U4 to the second composite space MS2. to the first composite space MS1.
  • the user environment determining unit 64 selects the object to be recognized by the third user U3 as the object for generating the second composite space MS2 (for example, the book O40, etc.), but not the object for generating the second complex space MS2. 1 Change to an object (for example, a flower O30, etc.) for generating a composite space MS1.
  • the user environment determination unit 64 determines the conditions of the third user U3 and the fourth user U4 based on the conditions of the third user U3 and the fourth user U4 and the conditions of the first composite space MS1 at that time.
  • the environment in the first composite space MS1 to be recognized by the user U4 is determined (FIG. 18/STEP 602).
  • the user environment determining unit 64 as the environment to be recognized by the third user U3, based on the coordinates and orientation of the physical space RS of the third user U3, creates an image of a region to be recognized in the physical space RS, The image of the third object O3 in that area and the accompanying sound are determined.
  • the user environment determination unit 64 as the environment to be recognized by the fourth user U4, based on the coordinates and posture of the physical space RS of the fourth user U4, the image of the region to be recognized in the physical space RS, The image of the third object O3 and the accompanying sound are determined.
  • the fourth user U4 can experience the first complex space MS1 together with the third user U3.
  • the fourth user U4 can visually recognize the flower O30, which is a virtual object arranged on the shelf, which is a real object, arranged by the third user U3.
  • the user environment determining unit 64 determines the third The environment in the first composite space MS1 to be recognized by the user U3 is determined, and based on the state of the fourth user U4 and the situation of the second composite space MS2 at that time, the environment in the second composite space MS2 to be recognized by the fourth user U4 The environment is determined (FIG. 18/STEP 603).
  • the HMD 7 executes the same process as STEP 503 and outputs the determined environment (FIG. 18/STEP 604 ).
  • the user action recognition unit 62 of the server 6 determines whether or not the third user U3 or the fourth user U4 has performed some action (FIG. 18/STEP 605).
  • the user action recognition unit 62 continues to perform a predetermined The determination in STEP 605 is repeated at the control cycle of .
  • the user motion recognition unit 62 determines whether or not the motion is a predetermined motion corresponding to a predetermined trigger event (FIG. 18/STEP606). .
  • the trigger event recognizing unit 65 executes the process of synchronizing the first composite space MS1 and the second composite space MS2 (the process of STEP601). It is determined whether or not it is completed (FIG. 18/STEP 607).
  • the trigger event recognition unit 65 determines whether or not a predetermined trigger event that is a key to executing the process of synchronizing the first composite space MS1 and the second composite space MS2 has been recognized. , to determine whether the process has been executed.
  • the object control unit 63 of the server 6 determines that the action of the user U is an action to operate the object O, similar to the process in STEP 505. (FIG. 18/STEP 608).
  • the object control unit 63 controls the object O according to the type of action of the user U, similar to the process in STEP 506. (FIG. 18/STEP 609).
  • the user environment determination unit 64 determines the environment in the composite space MS that the user U is made to recognize based on the state of the user U at that time and the situation of the composite space MS experienced by the user U ( FIG. 18/STEP 610).
  • the HMD 7 executes the same processing as STEP604 and outputs the determined environment (FIG. 18/STEP611).
  • the server 6 determines whether or not it has recognized a signal instructing the end of processing (FIG. 18/STEP 612).
  • the MR system S3 recognizes the signal instructing the end (YES in STEP612), the MR system S3 ends the current process.
  • the environment in the first composite space MS1 is determined as the environment to be recognized by the third user U3, and the environment in the second composite space MS2 is determined. is determined as the environment to be recognized by the fourth user U4.
  • the environment in the first complex space MS is determined as the environment recognized by the third user U3 and the fourth user U4.
  • the complex space experienced by the third user U3 (for example, the first complex space MS1 in this embodiment) and the complex space experienced by the fourth user U4 (for example, The second complex space M2) in the form is a complex space in which existing virtual objects are independent of each other.
  • the composite space experienced by the third user U3 and the composite space experienced by the fourth user U4 are the same composite space in which existing virtual objects match (for example, a real object). It becomes the first virtual space MS1) in the embodiment.
  • a process for synchronizing the objects existing in the composite space MS is executed to make the third user U3 and the fourth user U4 recognize the composite event. Since the environment of the space MS becomes similar, the third user U3 and the fourth user U4 who have experienced different environments can now experience similar environments.
  • the "predetermined action” that is a predetermined trigger event is not limited to the action of one of the third user U3 and the fourth user U4 touching the other. Any action performed by one of the user U3 and the fourth user U4 with reference to the other is sufficient.
  • the predetermined action is an action of the third user contacting the fourth user, an action of the third user moving within a predetermined range with respect to the fourth user, and an action of the third user moving into the complex space.
  • an action to select the fourth user as the target for example, an action to take a picture of the fourth user with a camera of a virtual mobile terminal or a real mobile terminal etc.
  • a ghost may be generated as a virtual object that operates in response to the user's motion, but whose coordinates in the complex space are different from those of the user.
  • the process of synchronizing the user characteristics may be executed even when a predetermined action is performed on the ghost.
  • the predetermined trigger event in the third invention does not necessarily have to be a predetermined action performed by one of the third user and the fourth user on the other of the first user and the fourth user. Therefore, for example, when the user gives an instruction to synchronize the complex space via an interface provided separately, the fact that the instruction is given may be set as a predetermined trigger event.
  • Virtual environment generator 30a Virtual space generator 30b Avatar generator 31, 62 User motion recognition unit 31a, 62a User posture recognition unit 31b, 62b User coordinate recognition unit 32
  • Avatar motion control unit 33 Avatar environment determination unit 34, 64 User environment determination unit 35, 65 Trigger event recognition unit 40, 70 Monitor 41, 71 Speaker 60
  • third object O30... flower, O31... first message board, O4... fourth object, O40... book, O41... second message board, RS... real space, S1, S2... VR system (virtual space experience system), S3... MR system (multiple spatial experience system), U1... first user, U2... second user, U3... third user, U4... fourth user, VS... virtual space, VS1... first virtual space, VS2 . . . second virtual space.

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Computer Graphics (AREA)
  • Computer Hardware Design (AREA)
  • Software Systems (AREA)
  • User Interface Of Digital Computer (AREA)
  • Processing Or Creating Images (AREA)
  • Position Input By Displaying (AREA)
PCT/JP2021/029641 2021-08-11 2021-08-11 仮想空間体感システム、及び、複合空間体感システム Ceased WO2023017584A1 (ja)

Priority Applications (3)

Application Number Priority Date Filing Date Title
JP2022529032A JP7138392B1 (ja) 2021-08-11 2021-08-11 仮想空間体感システム
PCT/JP2021/029641 WO2023017584A1 (ja) 2021-08-11 2021-08-11 仮想空間体感システム、及び、複合空間体感システム
JP2022084262A JP2023026312A (ja) 2021-08-11 2022-05-24 複合空間体感システム

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2021/029641 WO2023017584A1 (ja) 2021-08-11 2021-08-11 仮想空間体感システム、及び、複合空間体感システム

Publications (1)

Publication Number Publication Date
WO2023017584A1 true WO2023017584A1 (ja) 2023-02-16

Family

ID=83318379

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2021/029641 Ceased WO2023017584A1 (ja) 2021-08-11 2021-08-11 仮想空間体感システム、及び、複合空間体感システム

Country Status (2)

Country Link
JP (2) JP7138392B1 (https=)
WO (1) WO2023017584A1 (https=)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024209524A1 (ja) * 2023-04-04 2024-10-10 株式会社ソニー・インタラクティブエンタテインメント 情報処理装置、情報処理方法およびコンピュータプログラム
JP7598410B2 (ja) * 2023-06-19 2024-12-11 任天堂株式会社 ゲームプログラム、ゲームシステム、およびゲーム方法
US20250363752A1 (en) * 2023-10-19 2025-11-27 Abal Inc. Experience space switching method, experience space switching system, experience space switching apparatus, and experience space switching program and recording medium with the same recorded therein

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006293605A (ja) * 2005-04-08 2006-10-26 Canon Inc 情報処理方法およびシステム
WO2014119098A1 (ja) * 2013-02-01 2014-08-07 ソニー株式会社 情報処理装置、端末装置、情報処理方法及びプログラム
JP2018013900A (ja) * 2016-07-20 2018-01-25 株式会社Ａｂａｌ 仮想空間体感システム
JP2018029907A (ja) * 2016-08-26 2018-03-01 株式会社コロプラ 情報処理方法、当該情報処理方法をコンピュータに実行させるためのプログラム及びコンピュータ
WO2019116689A1 (ja) * 2017-12-15 2019-06-20 ソニー株式会社 情報処理装置、情報処理方法、及びプログラム
JP2020017244A (ja) * 2018-07-25 2020-01-30 株式会社バーチャルキャスト サーバ、方法、プログラム、動画像放送システム
WO2021095175A1 (ja) * 2019-11-13 2021-05-20 株式会社Ａｂａｌ 仮想空間体感システム

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002312295A (ja) * 2001-04-09 2002-10-25 Nec Interchannel Ltd 仮想３次元空間会話システム
JP2004199159A (ja) * 2002-12-16 2004-07-15 Canon Inc 情報処理方法、情報処理装置、プログラムおよび記憶媒体
JP7060778B2 (ja) * 2017-02-28 2022-04-27 キヤノンマーケティングジャパン株式会社 情報処理システム、情報処理システムの制御方法及びプログラム
JP6711885B2 (ja) * 2018-11-06 2020-06-17 キヤノン株式会社 送信端末、受信端末、送受信システム、およびそのプログラム
JP7029379B2 (ja) * 2018-11-28 2022-03-03 株式会社コロプラ プログラム、情報処理装置、および方法

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006293605A (ja) * 2005-04-08 2006-10-26 Canon Inc 情報処理方法およびシステム
WO2014119098A1 (ja) * 2013-02-01 2014-08-07 ソニー株式会社 情報処理装置、端末装置、情報処理方法及びプログラム
JP2018013900A (ja) * 2016-07-20 2018-01-25 株式会社Ａｂａｌ 仮想空間体感システム
JP2018029907A (ja) * 2016-08-26 2018-03-01 株式会社コロプラ 情報処理方法、当該情報処理方法をコンピュータに実行させるためのプログラム及びコンピュータ
WO2019116689A1 (ja) * 2017-12-15 2019-06-20 ソニー株式会社 情報処理装置、情報処理方法、及びプログラム
JP2020017244A (ja) * 2018-07-25 2020-01-30 株式会社バーチャルキャスト サーバ、方法、プログラム、動画像放送システム
WO2021095175A1 (ja) * 2019-11-13 2021-05-20 株式会社Ａｂａｌ 仮想空間体感システム

Also Published As

Publication number Publication date
JP2023026312A (ja) 2023-02-24
JP7138392B1 (ja) 2022-09-16
JPWO2023017584A1 (https=) 2023-02-16

Similar Documents

Publication Publication Date Title
JP7366196B2 (ja) 広範囲同時遠隔ディジタル提示世界
CN108304063B (zh) 信息处理装置、信息处理方法和计算机可读介质
JP2022549853A (ja) 共有空間内の個々の視認
JP7138392B1 (ja) 仮想空間体感システム
JP2010253277A (ja) ビデオゲームにおいてオブジェクトの動きを制御する方法およびシステム
US20210286432A1 (en) Information processing device, information processing method, and program
CN116964544A (zh) 信息处理装置、信息处理终端、信息处理方法和程序
JP6933850B1 (ja) 仮想空間体感システム
JP7055527B1 (ja) 仮想空間体感システム
JP2025027481A (ja) 情報処理方法、情報処理システム、情報処理装置及びプログラム

Legal Events

Date Code Title Description
ENP Entry into the national phase

Ref document number: 2022529032

Country of ref document: JP

Kind code of ref document: A

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21953479

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21953479

Country of ref document: EP

Kind code of ref document: A1