WO2021240601A1

WO2021240601A1 - Virtual space body sensation system

Info

Publication number: WO2021240601A1
Application number: PCT/JP2020/020547
Authority: WO
Inventors: 良哉尾小山
Original assignee: 株式会社Abal
Priority date: 2020-05-25
Filing date: 2020-05-25
Publication date: 2021-12-02
Also published as: JPWO2021240601A1

Abstract

Provided is a virtual space body sensation system with which it is possible to operate an object in a virtual space without reducing the feeling of immersion. A VR system S comprises: an avatar control unit 32 that controls the state of an avatar in the virtual space in accordance with the state of a user; and an object control unit 34 that controls a first object and a second object in the virtual space. The second object is positioned at different coordinates than the first object in the virtual space, has a shape corresponding to the first object, and can be operated by the user via the actions of the avatar. The object control unit 34 changes the state of the first object in accordance with changes in the state of the second object according to an operation by the user.

Description

Virtual space experience system

The present invention relates to a virtual space experience system that allows a user to recognize that he / she exists in a virtual space.

Conventionally, a virtual space is generated by a server or the like, and an image of the virtual space and an avatar corresponding to the user in the virtual space are presented to the user via a head-mounted display (hereinafter, may be referred to as “HMD”). There is a virtual space experience system that allows the user to recognize the image of the server and recognize that the user himself / herself exists in the virtual space to experience the virtual reality.

In this kind of virtual space experience system, a motion capture device or the like recognizes a user's state in the real space (for example, body movement, coordinate movement, posture change, etc.), and depending on the recognized state, Some change the state of the avatar (see, for example, Patent Document 1).

In the virtual space experience system of Patent Document 1, the state of the game pad type object which is the operation medium in the virtual space is changed according to the state of the user or the controller in the real space, and the state in the virtual space is changed according to the change. The character to be operated is made to perform a predetermined action.

Japanese Unexamined Patent Publication No. 2019-033881

By the way, in the system described in Patent Document 1, since a game pad type object is adopted as an operation medium, the operation of the operation target (character) operated via the object is the shape and function of the game pad. It will be simplified and restricted accordingly.

As a result, it gives the user the impression that the intuitive operation of the object to be operated is hindered, and there is a risk that the user's immersive feeling in the virtual space may be reduced.

The present invention has been made in view of the above points, and an object of the present invention is to provide a virtual space experience system capable of manipulating objects in a virtual space without reducing the immersive feeling.

The virtual space experience system of the present invention is
A virtual space generator that creates a virtual space that corresponds to the real space in which the user exists and includes an avatar and a predetermined object corresponding to the user.
A user state recognition unit that recognizes the user's state,
An avatar control unit that controls the state of the avatar according to the recognized state of the user,
An object control unit that controls the state of the predetermined object,
An image determination unit that determines an image of the virtual space to be recognized by the user based on the state of the avatar and the predetermined object.
A virtual space experience system including an image display that causes the user to recognize a determined image of the virtual space.
The predetermined object is located at a coordinate different from that of the first object in the first object and the virtual space, has a shape corresponding to at least a part of the first object, and has a shape corresponding to at least a part of the first object, through the movement of the avatar. Contains a second object that can be manipulated by the user
The object control unit is characterized in that when the second object is operated by the user, the state of the first object is changed according to the change in the state of the second object due to the operation.

Here, the "state" refers to something that can change according to the intention or operation of the user. It refers to the operating state of the body or mechanism, the position of coordinates, posture, direction, shape, color, size, etc. Therefore, the "change" of the state refers to the start, progress and stop of the operation, the movement of coordinates, the posture, the direction, the shape, the color, the change in size, and the like.

Further, here, the "corresponding shape" includes not only the same shape, but also a similar shape (that is, a shape having the same shape but a different size), a deformed shape, the same shape as a partially cut out shape, and the like. include.

As described above, in the virtual sky experience system of the present invention, when the second object is operated by the user through the movement of the avatar, the state of the first object is changed according to the change in the state of the second object due to the operation. Is configured to change. That is, the first object is the operation target, and the second object is the operation medium.

Here, the second object has a shape corresponding to at least a part of the first object. Therefore, the user can perform the same operation on the second object, which is the operation medium, as on the first object, which is the actual operation target.

This gives the user the impression that the first object is being directly operated. As a result, it is possible to prevent the user from giving an impression that the intuitive operation of the first object to be operated is hindered. As a result, it is possible to suppress the user's feeling of immersion in the virtual space.

Further, in the virtual space experience system of the present invention,
When the second object is operated by the user, the object control unit keeps at least one of the coordinates, the posture, and the direction of the state of the first object fixed, and the second object is operated by the operation. It is preferable to change the other state of the first object according to the change of the state.

If the state of the first object to be operated and the state of the second object to be the operation medium are completely synchronized, it may be difficult for the user to grasp the change in the state of the first object. be.

Therefore, in this way, while fixing at least one of the coordinates, postures, and directions of the object among the states of the first object, the user can change the other states in correspondence with the second object. It becomes easier to observe the state of the first object while freely moving the two objects. As a result, the user can easily grasp the change in the state of the first object.

Further, in the virtual space experience system of the present invention,
It is equipped with an item state recognition unit that recognizes the state of items existing in the real space.
It is preferable that the virtual space generation unit generates the second object in correspondence with the recognized item.

If there is no item corresponding to the second object as the operation medium in the real space (for example, a model having a shape corresponding to the first object), the second object is made to correspond to the first object to be operated. Can be freely generated. On the other hand, since it is not possible to give a tactile sensation to the user, there is a possibility that the immersive feeling of the user in the virtual space may be reduced.

Therefore, if the second object, which is the operation medium (that is, the object touched by the user U), is generated corresponding to the item existing in the real space in this way, the user can be given a tactile sensation. , It is possible to enhance the user's immersive feeling in the virtual space.

Further, in the virtual space experience system of the present invention,
It is preferable that the second object has the same or similar shape as the first object and is smaller than the first object.

When the first object is large and the second object is the same size, it may be difficult for the user to operate the avatar corresponding to the user himself / herself with respect to the second object. Therefore, with this configuration, the user can easily operate the avatar with respect to the second object.

The schematic diagram which shows the schematic structure of the VR system which concerns on embodiment. The block diagram which shows the structure of the processing part of the VR system of FIG. The schematic diagram of the virtual space generated in the VR system of FIG. The flowchart which shows the process to execute when the VR system of FIG. 1 is started to use. It is a schematic diagram of the virtual space to be recognized by the user in the VR system of FIG. 1, and is the figure which shows the state before operating the 2nd object. The flowchart which shows the process which the VR system of FIG. 1 executes when an object is manipulated. It is a schematic diagram of the virtual space to be recognized by the user in the VR system of FIG. 1, and is the figure which shows the state which performed the operation which changed the coordinates of the 2nd object. It is a schematic diagram of the virtual space to be recognized by the user in the VR system of FIG. 1, and is the figure which shows the state which performed the operation which changes the direction of the 2nd object.

Hereinafter, the VR system S, which is a virtual space experience system according to the embodiment, will be described with reference to the drawings. The VR system S is a system that allows the user to experience virtual reality (so-called VR (virtual reality)) by recognizing that the user himself / herself exists in the virtual space.

[Overview of system configuration]
First, a schematic configuration of the VR system S will be described with reference to FIGS. 1 and 2.

As shown in FIG. 1, the VR system S has a plurality of markers 1 attached to the user U and the item I existing in the real space RS, and the user U and the item I (strictly speaking, the markers 1 attached to them). The camera 2 that captures the image, the server 3 that determines the image and sound of the virtual space VS (see FIG. 3 and the like) to be recognized by the user U, and the head-mounted display that causes the player to recognize the determined image and sound (hereinafter, "" It is called "HMD4").

In the VR system S, the camera 2, the server 3, and the HMD 4 can wirelessly transmit and receive information to and from each other. However, any one of them may be configured to be able to send and receive information to and from each other by wire.

Of the plurality of signs 1, those attached to the user U are attached to the head, both hands, and both feet of the user U via the HMD4, gloves, and shoes worn by the user U.

Further, among the plurality of signs 1, those attached to the item I are attached to the positions that are the feature points in the image of the item I taken by the camera 2. In the present embodiment, since a plurality of rectangular parallelepiped building blocks placed on the plate are adopted as the item I, they are attached to the edges, the vicinity of the corners, and the like.

Note that the sign 1 is used to recognize the posture, coordinates, and direction of the user U or the item I in the real space RS as described later. Therefore, the mounting position of the sign 1 may be appropriately changed according to other devices constituting the VR system S.

The camera 2 multi-directions the operable range of the user U and the item I in the real space RS in which the user U and the item I exist (that is, a range in which the posture change, the coordinate movement, the direction change, etc. can be executed). It is installed so that you can shoot from.

The server 3 recognizes the sign 1 from the image taken by the camera 2, and recognizes the posture, coordinates, and direction of the user U or the item I based on the position of the recognized sign 1 in the real space RS. Further, the server 3 determines the image and the sound to be recognized by the user U based on the posture, the coordinates and the direction.

The HMD4 is attached to the head of the user U. As shown in FIG. 2, the HMD 4 has a monitor 40 (image display) for causing the user U to recognize the image of the virtual space VS determined by the server 3, and a virtual space determined by the server 3. It has a speaker 41 (voice generator) for causing the user U to recognize the voice of VS.

When the user U is made to experience the virtual reality by using this VR system S, the user U is made to recognize only the image and the sound of the virtual space VS, and the user U himself exists in the virtual space as the avatar A. Be recognized. That is, the VR system S is configured as a so-called immersive system.

The VR system S includes a so-called motion capture device composed of a marker 1, a camera 2, and a server 3 as a system for recognizing the states of the user U and the item I in the real space RS.

Here, the "state" refers to something that can change according to the intention or operation of the user U. For example, it refers to the operating state of the body or mechanism, the position of coordinates, the posture, the direction, the shape, the color, the size, and the like. Therefore, the "change" of the state refers to the start, progress and stop of the operation, the movement of coordinates, the posture, the direction, the shape, the color, the change in size, and the like.

However, the virtual space experience system of the present invention is not limited to such a configuration. For example, when a motion capture device is used, in addition to the above configuration, one having a different number of signs and cameras from the above configuration (for example, one is provided for each) may be used.

Further, a device other than the motion capture device may be used to recognize the state in the user's real space. Specifically, for example, a sensor such as GPS may be mounted on the HMD, and the state of the player may be recognized based on the output from the sensor. Further, such a sensor may be used in combination with a motion capture device as described above.

[Configuration of processing unit]
Next, the configuration of the server 3 will be described in detail with reference to FIGS. 2 and 3.

The server 3 is composed of one or a plurality of electronic circuit units including a CPU, RAM, ROM, an interface circuit, and the like. As shown in FIG. 2, the server 3 has a virtual space generation unit 30, a user state recognition unit 31, an avatar control unit 32, and an item state recognition unit as functions realized by the implemented hardware configuration or program. 33, an object control unit 34, and an output information determination unit 35 are provided.

As shown in FIG. 3, the virtual space generation unit 30 includes an image as a background of the virtual space VS corresponding to the real space RS in which the user U exists, an avatar 5 existing in the virtual space VS, and a predetermined object. Generate an image of. The virtual space generation unit 30 also generates sounds related to those images.

In the present embodiment, the movement range of the avatar 5 in the virtual space VS is the range corresponding to the room of the real space RS in which the user U exists. Therefore, the size of the image of the virtual space VS generated by the virtual space generation unit 30 (that is, the size of the virtual space VS) corresponds to the room of the real space RS.

In the present embodiment, the avatar 5 generated by the virtual space generation unit 30 is anthropomorphic to an animal, and operates in response to a human motion. When a plurality of users U exist, the virtual space generation unit 30 generates a plurality of avatars so as to correspond to each user U.

The predetermined object generated by the virtual space generation unit 30 includes a first object 6 and a second object 7 located at a coordinate different from that of the first object 6 in the virtual space VS. Specifically, the first object 6 is generated at a position far from the avatar 5 so that the user U can recognize the whole image through the avatar 5. The second object 7 is generated within the reach of the avatar 5.

In the present embodiment, the first object 6 is composed of the first building 61a, the second building 61b, and the third building 61c established on the base 60. In the following, these buildings are collectively referred to as "building 61".

The second object 7 is composed of a first model 71a, a second model 71b, and a third model 71c mounted on a plate 70 corresponding to the base 60. In the following, these models are collectively referred to as "model 71".

The second object 7 is a reduction of the first object 6 to the extent that it can be operated through the hand of the avatar 5. That is, the shape of the second object 7 is the same as that of the first object 6, but the size of the second object 7 is smaller than that of the first object 6.

As shown in FIG. 2, the user state recognition unit 31 recognizes the state of the user U based on the image data of the user U including the sign 1 taken by the camera 2. The user state recognition unit 31 has a user posture recognition unit 31a, a user coordinate recognition unit 31b, and a user direction recognition unit 31c.

The user posture recognition unit 31a, the user coordinate recognition unit 31b, and the user direction recognition unit 31c extract the marker 1 attached to the user U from the image data of the user U, and based on the extraction result, the posture of the user U, Recognize coordinates and directions.

The avatar control unit 32 controls the state of the avatar 5 (specifically, the change in posture, coordinates, and direction) according to the change in the state of the user U recognized by the user state recognition unit 31.

The item state recognition unit 33 recognizes the state of the item I based on the image data of the item I including the sign 1 taken by the camera 2. The item state recognition unit 33 has an item posture recognition unit 33a, an item coordinate recognition unit 33b, and an item direction recognition unit 33c.

The item posture recognition unit 33a, the item coordinate recognition unit 33b, and the item direction recognition unit 33c extract the marker 1 attached to the item I from the image data of the item I, and based on the extraction result, the posture of the item I, Recognize coordinates and directions.

The object control unit 34 changes the states of the first object 6 and the second object 7 according to the operation of the avatar 5 recognized by the avatar control unit 32 and the state of the item I recognized by the item state recognition unit 33. Control. The object control unit 34 has a first object control unit 34a and a second object control unit 34b.

The second object control unit 34b controls the state of the second object 7 according to the operation of the avatar 5 or the change of the state of the item I. The first object control unit 34a controls the state of the first object 6 according to the change of the state of the second object 7.

As will be described later, the posture, coordinates, and direction of the second object 7 correspond to the posture, coordinates, and direction of the item I. Further, the posture (direction of each building 61) and coordinates of the first object 6 correspond to the posture (direction of each model 71) and coordinates of the second object 7. On the other hand, the direction of the first object 6 (phase around the yaw axis of the base 60) is fixed and always constant regardless of the direction of the second object 7 (phase around the yaw axis of the plate 70) (FIG. 5, FIG. 7 and 8).

The output information determination unit 35 determines the information regarding the virtual space VS to be recognized by the user U via the HMD4. The output information determination unit 35 has an image determination unit 35a and an audio determination unit 35b.

The image determination unit 35a determines an image of the virtual space VS to be recognized by the user U corresponding to the avatar 5 via the monitor 40 of the HMD 4 based on the states of the avatar 5, the first object 6, and the second object 7. do.

The voice determination unit 35b is related to the image of the virtual space VS to be recognized by the user U corresponding to the avatar 5 via the speaker 41 of the HMD 4 based on the states of the avatar 5, the first object 6, and the second object 7. Determine the voice to be played.

Note that each processing unit constituting the virtual space experience system of the present invention is not limited to the above configuration.

For example, a part of the processing unit provided in the server 3 in the present embodiment may be provided in the HMD 4. Further, a plurality of servers may be used, or the servers may be omitted and the CPUs mounted on the HMD may be linked to each other.

Further, a speaker other than the speaker mounted on the HMD may be provided. Further, in addition to the device that affects the visual sense and the auditory sense, the device that affects the sense of smell and the sense of touch that causes odor, wind, etc. according to the virtual space may be included.

[Process to be executed]
Next, with reference to FIGS. 2 to 8, the process executed by the VR system S when the user U is made to experience the virtual reality by using the VR system S will be described. It should be noted that this VR system S is for verifying the arrangement of each building constituting the building group in the virtual space VS.

[Processing when use starts]
First, with reference to FIGS. 2 to 5, when the user U starts using the VR system S, the processing executed by each processing unit of the VR system S will be described.

In this process, first, the virtual space generation unit 30 of the server 3 generates the virtual space VS, the avatar 5 to exist in the virtual space VS, and various objects (FIG. 4 / STEP101).

Specifically, as shown in FIG. 3, the virtual space generation unit 30 generates an image as a background of the virtual space VS. Further, the virtual space generation unit 30 generates an image of the avatar 5 existing in the virtual space VS based on the image of the user U taken by the camera 2. Further, the virtual space generation unit 30 generates images of the first object 6 and the second object 7 based on the image of the item I taken by the camera 2.

Next, the avatar control unit 32 of the server 3 determines the state of the avatar 5 in the virtual space VS based on the state of the user U in the real space RS (FIG. 4 / STEP102).

Specifically, first, the user state recognition unit 31 recognizes the state of the user U in the real space RS based on the state of the user U taken by the camera 2. After that, the avatar control unit 32 determines the state of the avatar 5 based on the recognized state of the user U.

Next, the object control unit 34 of the server 3 determines the states of the first object 6 and the second object 7 in the virtual space VS based on the state of the item I in the real space RS (FIG. 4 / STEP103).

Specifically, first. The item state recognition unit 33 recognizes the state of the item I in the real space RS based on the state of the item I taken by the camera 2. After that, the first object control unit 34a of the object control unit 34 determines the state of the first object 6 based on the recognized state of the item I. Further, the second object control unit 34b of the object control unit 34 determines the state of the second object 7 based on the recognized state of the item I.

Next, the image and voice that the image determination unit 35a and the voice determination unit 35b of the output information determination unit 35 of the server 3 make the user U recognize are the images and sounds of the avatar 5, the first object 6, and the second object 7 in the virtual space VS. Determined based on the condition (FIG. 4 / STEP104).

Next, the HMD 4 worn by the user U displays the determined image on the monitor 40, generates the determined voice on the speaker 41 (FIG. 4 / STEP105), and ends the current process. do.

By these processes, as shown in FIG. 5, the user U has the first object 6 to be operated at a position where the whole image can be seen, and the second object 7 which is an operation medium has a second object 7. The virtual space VS, which exists within the reach of the avatar 5, is in a state of being recognized as if it exists.

[Processing when an object is manipulated]
Next, referring to FIGS. 2 and 5 to 8, when the user U starts using the VR system S and then operates the second object 7 which is an operation medium via the avatar 5, the VR system is used. The processing executed by each processing unit of S will be described.

Here, the first object 6 to be operated in the present embodiment is composed of a base 60 and a plurality of buildings 61 established on the base 60. Further, the second object 7 is mounted on a plate 70 having the same shape or a similar shape as the base 60, and a model 71 of a plurality of buildings having the same shape or a similar shape as the building 61. It is composed of and.

Further, the "operation" for the second object 7 means the rotation of the entire second object 7 around the yaw axis by rotating the plate 70, and the plate by changing the coordinates and posture (direction with respect to the plate 70) of the model 71. 70 Refers to the layout change on.

In this process, first, the user state recognition unit 31 determines whether or not the state of the user U has changed (FIG. 6 / STEP201).

If the state of the user U has not changed (NO in STEP201), the process returns to STEP201 and the determination of STEP201 is executed again at a predetermined control cycle.

On the other hand, when the state of the user U changes (YES in STEP201), the avatar control unit 32 changes the state of the avatar 5 based on the change in the state of the user U (FIG. 6 / STEP202).

Next, the object control unit 34 determines whether or not the change in the state of the avatar 5 (that is, the operation) executed by the avatar control unit 32 is an operation such as operating the second object 7 which is an operation medium. (Fig. 6 / STEP203).

Specifically, the object control unit 34 determines whether or not the change in the posture, coordinates, and direction of the avatar 5 with respect to the second object 7 in the virtual space VS corresponds to the change in the predetermined posture, coordinates, and direction. It is determined whether or not the operation of the avatar 5 is an operation such as operating the second object 7 which is an operation medium.

When the operation of the avatar 5 is such that the operation of the second object 7 is performed (YES in STEP 203), the second object control unit 34b of the object control unit 34 controls the second object based on the operation by the avatar 5. The posture, coordinates, and direction of No. 7 are changed (FIG. 6 / STEP204).

Next, the first object control unit 34a of the object control unit 34 changes the posture and coordinates of the first object 6 based on the change of the posture and coordinates of the second object 7 (FIG. 6 / STEP205).

Specifically, in the processing of STEP 203 to 205, for example, as shown in FIG. 5, from the state before the operation medium, the second object 7, is operated by the avatar 5, as shown in FIG. It is assumed that the positions of the first model 71a and the positions of the second model 71b included in the second object 7 are replaced, and their orientations are adjusted. That is, it is assumed that the coordinates and postures (positions or orientations thereof with respect to the plate 70) of the first model 71a and the second model 71b are changed.

In this case, even in the first object 6 which is the operation target, the first building 61a and the second building corresponding to the changes in the coordinates and postures of the first model 71a and the second model 71b of the second object 7. The coordinates and orientations of 61b (positions and orientations thereof with respect to the base 60) change from the state shown in FIG. 5 to the state shown in FIG. 7.

However, for example, as shown in FIG. 5, the direction (yaw) of the plate 70 included in the second object 7 is as shown in FIG. 8 from the state before the operation medium, the second object 7, is operated by the avatar 5. It is assumed that the (phase around the axis) is changed to change the direction of the entire second object 7, and then the coordinates of the third model 71c are changed.

In this case, in the first object 6 to be operated, the direction (phase around the yaw axis) of the base 60 corresponding to the plate 70 is the direction of the plate 70 of the second object 7, regardless of the change in the direction of the plate 70. It does not change from the state shown in the above, and only the coordinates of the third building 61c corresponding to the third model 71c (the position relative to the base 60 or the first building 61a or the second building 61b) change. That is, the direction of the first object 6 with respect to the user U does not change.

Next, the image and voice that the image determination unit 35a and the voice determination unit 35b of the output information determination unit 35 of the server 3 make the user U recognize are the images and sounds of the avatar 5, the first object 6, and the second object 7 in the virtual space VS. Determined based on the condition (FIG. 6 / STEP206).

Similarly, when the operation of the avatar 5 is not an operation such as operating the second object 7 (NO in STEP 203), the image determination unit 35a and the voice determination unit 35b of the output information determination unit 35 of the server 3 are used by the user U. The image and sound to be recognized by the server are determined based on the states of the avatar 5, the first object 6, and the second object 7 in the virtual space VS (FIG. 6 / STEP206).

Next, the HMD 4 worn by the user U displays the determined image on the monitor 40, generates the determined voice on the speaker 41 (FIG. 6 / STEP207), and ends the current process. do.

The VR system S repeatedly executes the above processing in a predetermined control cycle until the end instruction by the user U is recognized.

As described above, in the VR system S, when the second object 7 is operated by the user U via the operation of the avatar 5, the first object is changed according to the change in the state of the second object 7 due to the operation. It is configured to change the state of 6. That is, the first object 6 is the operation target, and the second object 7 is the operation medium.

Here, the second object 7 has a shape corresponding to the first object 6. Therefore, the user U can perform the same operation on the second object 7, which is the operation medium, as on the first object 6, which is the actual operation target.

This gives the user U the impression that the first object 6 is being directly operated. As a result, it is possible to prevent the user U from giving an impression that the intuitive operation of the first object 6 to be operated is hindered. As a result, it is possible to suppress the user U from reducing the immersive feeling in the virtual space VS.

By the way, when the state of the first object 6 to be operated and the state of the second object 7 to be the operation medium are completely synchronized, the user U reverses the change in the state of the first object 6. It may be difficult to grasp.

Specifically, in the present embodiment, if the direction of the first object 6 and the direction of the second object 7 are completely synchronized, the direction of the first object 6 is frequently switched, and the user U can use the user U. On the contrary, it may be difficult to grasp the change in the state of the first object 6 as a whole.

Therefore, in the VR system S, the posture and coordinates of the state of the first object 6 are configured to be changed according to the second object 7 while the direction is fixed. This makes it easier for the user U to observe the state of the first object 6 while freely moving the second object 7. As a result, the user U can easily grasp the change in the state of the first object 6.

Further, when the first object 6 is large as in the VR system S and the second object 7 has the same size as the first object, the user U sets the avatar 5 corresponding to the user U himself. 2 It may be difficult to operate on the object 7.

Therefore, in the VR system S, the second object 7 has the same shape as or similar to that of the first object 6, and is smaller than the first object 6 so that it can be operated by the hand of the avatar 5. Has been done. As a result, the user U can easily operate the avatar 5 with respect to the second object 7.

[Other embodiments]
Although the illustrated embodiment has been described above, the present invention is not limited to such an embodiment.

For example, in the above embodiment, the shape of the second object 7 which is the operation medium is the same as that of the first object 6 which is the operation target, but the size of the second object 7 is smaller than that of the first object 6. ing.

However, the second object of the present invention is not limited to such a configuration, is located at coordinates different from the first object in the virtual space, and has a shape corresponding to at least a part of the first object. It suffices as long as it can be operated by the user through the operation of the avatar.

Here, the "corresponding shape" includes not only the same shape, but also a similar shape (that is, a shape having the same shape but a different size), a deformed shape, the same shape as a partially cut out shape, and the like.

Therefore, for example, the first object and the second object may have the same size as well as the shape. Further, the second object may be generated by imitating only a part of the first object. Specifically, for example, the first object may be the entire car, and the second object may be only the part for designing the car (for example, the internal structure of the bonnet).

Further, in the above embodiment, one second object 7 which is an operation medium is generated for one first object 6 which is an operation target. However, the object of the present invention is not limited to such a configuration. For example, a plurality of second objects as operation media may be generated for six first objects to be operated. On the contrary, only one second object may be generated for a plurality of first objects.

Further, in the above embodiment, the posture and coordinates of the first object 6 which is the operation target are changed according to the change of the posture and coordinates of the second object 7 which is the operation medium. On the other hand, the direction of the first object 6 is fixed regardless of the change in the direction of the second object 7. However, the change in the state of the first object in the present invention is not limited to such a configuration, and may be any one corresponding to the change in the state of the second object.

Here, as described above, the "state" in the present invention refers to a state that can change according to the intention or operation of the user U. For example, it refers to the operating state of the body or mechanism, the position of coordinates, the posture, the direction, the shape, the color, the size, and the like. Therefore, the "change" of the state refers to the start, progress and stop of the operation, the movement of coordinates, the posture, the direction, the shape, the color, the change in size, and the like.

Therefore, for example, in the above embodiment, the direction of the first object 6 may also be changed according to the change in the state of the second object 7. Further, for example, the shape, color, size, etc. of the first object may be changed according to the state of the second object.

At this time, the changes do not necessarily have to be in perfect agreement. For example, the state of the first object may be changed later than the change of the state of the second object. Further, for example, as in the above embodiment, only a part of the plurality of states may be changed in correspondence with each other. It should be noted that, by the user's operation by the operation of the avatar or the like, it may be possible to switch between the case where only a part is made to correspond and the case where the change is made and the case where the whole is made to correspond.

Further, in the above embodiment, the virtual space generation unit 30 generates the second object 7 corresponding to the recognized item I, and the second object control unit 34b is the item I recognized by the item state recognition unit 33. The state of the second object 7 is controlled according to the state of.

This gives the user U a tactile sensation by generating the second object 7, which is the operation medium (that is, the object to be touched by the user U), in correspondence with the item I existing in the real space. This is to enhance the immersive feeling of the user U in the virtual space VS.

However, the virtual space experience system of the present invention is not limited to such a configuration, and the item corresponding to the second object may not exist. With this configuration, the second object, which is the operation medium, can be freely generated in correspondence with the first object, which is the operation target.

Further, in this configuration, the control of the first object and the second object cannot be performed based on the state of the item as in the above embodiment, so that the control thereof is the operation of the avatar (and by extension, the avatar). It suffices to control only according to the change in the state of the user, which is the basis of the control of the above.

1 ... sign, 2 ... camera, 3 ... server, 4 ... HMD, 5 ... avatar, 6 ... first object, 7 ... second object, 30 ... virtual space generation unit, 31 ... user state recognition unit, 31a ... user attitude Recognition unit, 31b ... User coordinate recognition unit, 31c ... User direction recognition unit, 32 ... Avatar control unit, 33 ... Item state recognition unit, 33a ... Item attitude recognition unit, 33b ... Item coordinate recognition unit, 33c ... Item direction recognition unit , 34 ... object control unit, 34a ... first object control unit, 34b ... second object control unit, 35 ... output information determination unit, 35a ... image determination unit, 35b ... voice determination unit, 40 ... monitor (image display) , 41 ... speaker (voice generator), 60 ... base, 61 ... building, 61a ... first building, 61b ... second building, 61c ... third building, 70 ... plate, 71 ... model, 71a ... first model, 71b ... 2nd model, 71c ... 3rd model, I ... item, U ... user, RS ... real space, S ... VR system (virtual space experience system), VS ... virtual space.

Claims

A virtual space generator that creates a virtual space that corresponds to the real space in which the user exists and includes an avatar and a predetermined object corresponding to the user.
A user state recognition unit that recognizes the user's state,
An avatar control unit that controls the state of the avatar according to the recognized state of the user,
An object control unit that controls the state of the predetermined object,
An image determination unit that determines an image of the virtual space to be recognized by the user based on the state of the avatar and the predetermined object.
A virtual space experience system including an image display that causes the user to recognize a determined image of the virtual space.
The predetermined object is located at a coordinate different from that of the first object in the first object and the virtual space, has a shape corresponding to at least a part of the first object, and has a shape corresponding to at least a part of the first object, through the movement of the avatar. Contains a second object that can be manipulated by the user
The object control unit is characterized in that when the second object is operated by the user, the state of the first object is changed according to the change in the state of the second object due to the operation. Spatial experience system.
In the virtual space experience system according to claim 1,
When the second object is operated by the user, the object control unit keeps at least one of the coordinates, the posture, and the direction of the state of the first object fixed, and the second object is operated by the operation. A virtual space experience system characterized in that the other state of the first object is changed according to the change of the state.
In the virtual space experience system according to claim 1 or 2.
It is equipped with an item state recognition unit that recognizes the state of items existing in the real space.
The virtual space generation unit is a virtual space experience system characterized in that the second object is generated in correspondence with the recognized item.
In the virtual space experience system according to any one of claims 1 to 3.
The virtual space experience system, wherein the second object has the same shape or a similar shape as the first object, and is smaller than the first object.