WO2022107294A1 - Ｖｒ映像空間生成システム - Google Patents

Ｖｒ映像空間生成システム Download PDF

Info

Publication number
WO2022107294A1
WO2022107294A1 PCT/JP2020/043282 JP2020043282W WO2022107294A1 WO 2022107294 A1 WO2022107294 A1 WO 2022107294A1 JP 2020043282 W JP2020043282 W JP 2020043282W WO 2022107294 A1 WO2022107294 A1 WO 2022107294A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
video
user
space
viewing device
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2020/043282
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
晃弘 安藤
智絵 水上
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hashilus Co ltd
Original Assignee
Hashilus Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hashilus Co ltd filed Critical Hashilus Co ltd
Priority to PCT/JP2020/043282 priority Critical patent/WO2022107294A1/ja
Priority to JP2022563511A priority patent/JP7547501B2/ja
Publication of WO2022107294A1 publication Critical patent/WO2022107294A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating three-dimensional [3D] models or images for computer graphics

Definitions

  • the present invention relates to a VR video space generation system for constructing a virtual space that can be used by a user by VR (Virtual Reality), and in particular, the user can freely move and operate in the generated VR video space.
  • VR Virtual Reality
  • it is generated in a general file format including PNG, MP4, etc., and can be easily replaced without changing the VR video space generation system main body, readable independent, plane projection video, 3D video Or, by projecting image / video data (hereinafter referred to as "material") such as dome images as they are in the VR image space, the user can feel as if they are watching and experiencing them in the virtual space.
  • the present invention relates to a VR video space generation system that enables a user to effectively experience presentations, amusements, exhibitions, trainings, etc. using various materials in a VR space by performing the operation of.
  • Virtual reality refers to the technology for making users recognize and perceive in a virtual space created by a computer as if they were in that space, and the technology for creating such an environment and for that purpose.
  • Various business tools and amusement devices using such techniques have been developed to give convenience and entertainment to users, and have the possibility of further use and utilization in the future.
  • Japanese Patent Application Laid-Open No. 2018-142722 exists as a technology related to a system that provides a support method that demonstrates superiority to competitors in presentations of buildings, etc.
  • a 3D CG perspective which is two panoramic images that are stereograms based on the original drawing data, is generated.
  • the 3D CG perspective is converted into a panoramic VR image, and the converted panoramic VR image is associated with each other to generate a presentation document related to the building, and then associated with the presentation document in which the presentation document is used.
  • a technique relating to a system for displaying a panoramic VR image on a display device is disclosed.
  • the present invention is a VR video space generation system for constructing a virtual space that can be used by users by virtual reality, and in particular, one or more users are free to use the generated VR video space in the generated VR video space.
  • a readable, independent, planar projection image, or dome image that can be moved and operated, and is generated in a general file format and can be easily replaced without changing the system itself.
  • the VR video space generation system is a VR video space generation system that generates VR video for constructing a virtual space accessible to one or more users, and is a user.
  • a VR viewing device including one or a plurality of VR viewing devices, a video generating means for generating an initial image that can be displayed on the VR viewing device, a position information acquiring means for acquiring the position information of each of the VR viewing devices, and the above. Based on each position information acquired by the position information acquisition means, it is generated by the VR image generation means that generates a VR image by synthesizing an avatar image with the initial image generated by the image generation means, and the VR image generation means. It is configured to include a video output means for outputting the VR video to a VR viewing device.
  • the VR video space generation system numerically defines an area defining means that numerically defines a region that can be moved by the user as XYZ coordinates, and each position information acquired by the position information acquisition means numerically as XYZ coordinates.
  • Each position information defined by the position definition means is introduced (applied) as a coordinate value into the position definition means and the area defined by the area definition means, and the area and each position information are introduced into the video.
  • the configuration includes a mapping means associated with the initial image generated by the generation means.
  • the VR video space generation system is a VR video space generation system that generates VR video for constructing a virtual space accessible to one or a plurality of users, and is one or a plurality of VR video space generation systems worn by the user.
  • a VR viewing device consisting of a VR viewing device, a video generation means for generating an initial image that can be displayed on the VR viewing device, and an avatar image of each user who attaches the VR viewing device to the initial video generated by the video generation means. It is composed of a VR image generation means for generating a VR image and a video output means for outputting the VR image generated by the VR image generation means to a VR viewing device.
  • the VR video space generation system numerically defines the area defining means for numerically defining the area in which the user can move as XYZ coordinates and the position information of the user who wears the VR viewing device as XYZ coordinates.
  • Each position information defined by the position definition means is introduced (applied) as a coordinate value into the position definition means and the area defined by the area definition means, and the area and each position information are introduced into the video.
  • the configuration includes a mapping means associated with the initial image generated by the generation means.
  • the position defining means acquires information relating to the position of a specific part of the body of the user wearing the VR viewing device, calculates information regarding the position of each other part of the body, and generates the VR image generation means. However, it is configured to draw based on the above information and generate a VR image to be displayed on the VR viewing device.
  • the initial image is composed of an image selected from any of a plane projection image, a 3D image, and a dome image.
  • the initial video is created in a general format and is a plane projection containing readable, independent, user-instructed illustrations, 2D images, and textual information that can be easily replaced without changing other systems. Any one or a video, a presentation image consisting of a 2D moving image that operates according to the user's instruction, a 3D image including a CG stereoscopic model that operates according to the user's instruction, an all-sky image, or a part thereof. It is a configuration that includes a plurality. Further, the initial video is configured to be composed of independent video data that can be read by the video generation means.
  • the VR viewing device includes a sensor that detects the position and movement of the fingers of both hands of the user who wears the device, and the sensor detects the movement of the user's finger and then acquires the movement information as tracking information.
  • the VR video space generation system holds a plurality of action information consisting of finger movement information, and each of the action information is associated with a specific change process of the initial video, and the VR is described.
  • the video space generation system is configured to perform change processing of the associated initial video when the tracking information matches any of the action information.
  • the action information is composed of instruction information generated by simultaneously operating the fingers of both hands of the user.
  • the initial video includes an image such as a presentation and a moving image, and the action information is associated with the progress and backward processing of the initial video.
  • the present invention has the configuration as described in detail above, it has the following effects. 1.
  • the position information of the VR viewing device is acquired and the avatar image is synthesized with the initial image so as to correspond to the position information, it is possible to reflect the current position of the user in the VR image space and display it. , Multiple people can experience and share the VR video space at the same time. Further, by performing the calibration process, even a user at a remote place can experience and share the VR video space in the same manner.
  • the area defining means and the position defining means define the area of the VR video space and the position of the user in the area, it is possible to match the actually movable area with the area in the virtual space.
  • the VR video space generation system can be configured without using location information acquisition means, even users who do not acquire the user's realistic current location information or continuously acquire the current location information can also virtualize in the virtual space. It becomes possible to participate in the VR space using the specific location information.
  • the position defining means can be configured to numerically define each position information of the user who wears the VR viewing device as XYZ coordinates without using the position information acquisition means, when the user is in a remote place in Japan or abroad. However, it will be possible for multiple people to experience and share the VR video space at the same time.
  • the position defining means is configured to acquire information related to the position of a specific part of the user's body and calculate the position of other parts of the body, which one is used for each VR viewing device based on such accurate position information. It is possible to draw and display in detail whether such an image should be displayed. 6. Since the initial image is composed of a plane projection image, a 3D image, a dome image, etc., it is possible to experience all types and types of materials in the VR space.
  • 3D including illustrations that are switched according to the user's instructions, 2D images and a planar projection image that includes text information, a 2D moving image that operates according to the user's instructions, a 3D model of CG that operates according to the user's instructions, etc.
  • a VR video space where multiple users can share presentations, amusements, exhibitions, and training that include various materials because the configuration includes one or more of video, all-sky video, or some of them. It will be possible to do it within.
  • the initial video the video that is switched according to the user's instruction is displayed at a fixed position on the VR viewing device, so these materials are shared by multiple users as images / videos for presentations, amusements, exhibitions, and training. It becomes possible to display in space.
  • One VR video space generation system because the initial video is configured so that the video generation means conforms to a general file format and is generated from readable independent material that can be easily replaced without changing the system body. It is possible to display various VR video spaces and let the user experience them. 10. Since the sensor acquires the movement of the user's finger as tracking information and contrasts it with the preset action information, the user moves the finger to give a processing instruction such as the movement / change of the initial image. It becomes possible.
  • the action information is composed of instruction information generated by simultaneously moving the fingers of both hands of the user, it reacts only to the processing instruction by moving both hands, and the risk of issuing an erroneous processing instruction can be reduced.
  • the action information is configured to include instruction information corresponding to the progress and backward processing of the initial video, presentations, amusements, exhibitions, and trainings that involve sequential changes of images in the VR video space can be performed only with the movements of the user's hands and fingers. It is possible to proceed with.
  • FIG. 1a is a schematic diagram of a VR video space generation system according to the present invention
  • FIG. 1b is a schematic diagram of a VR video space generation system provided with an external computer
  • FIG. 2 is a schematic view showing a display example of a VR image
  • FIG. 3 is a diagram showing area / position information
  • FIG. 4a is a schematic diagram of a VR video space generation system provided to a remote user
  • FIG. 4b is a schematic diagram of a VR video space generation system provided to a remote user provided with an external computer
  • FIG. 5 is a schematic diagram of a VR video space generation system that performs switching processing of video materials and scenes.
  • the VR video space generation system 1 includes a VR viewing device 100, a video generation means 200, a position information acquisition means 300, a VR video generation means 400, and a video output. It is a system for constructing a virtual space using virtual reality (VR) technology, which comprises means 500 and means. It generates a VR video for constructing a virtual space that can be accessed by one or more users, and one or more users can freely move and operate in the generated VR video space, and it is also virtual. It is a system that makes it possible to give the user the feeling of being in space.
  • the VR video space generation system according to the present invention makes it possible to realize user communication in a virtual space, presentation, experience of attraction, exhibition, training, and the like.
  • the "VR" image displayed in the present invention also includes AR (Augmented Reality), MR (Mixed Reality), and SR (Substitutional Reality). It is a concept that includes.
  • the VR viewing device 100 is a device composed of one or a plurality of devices worn by a user, and is a device for playing / displaying a video used for playing a VR video 20 generated based on the initial video 10. ..
  • the device of this embodiment mainly consists of a goggle-type projection device, but is not limited to this, and may be worn by a user, for example, a retinal projection method for directly forming and projecting an image on the retina. Any projection device that is possible and gives a feeling of immersion in the image can be appropriately selected and used.
  • the VR viewing device 100 is equipped with a storage means 610 including an arithmetic unit (not shown) and an arbitrary storage medium, and the storage means 610 is equipped with an initial image 10 And the VR image 20 generated from this is stored.
  • the VR viewing device 100 is a projection device for enabling viewing of VR images, and in the present embodiment, it is mainly configured as a goggle type projection device, but the present invention is not limited to this. However, it is possible to use a projection device having another structure.
  • the image displayed on the VR viewing device 100 is mainly composed of an image that can be viewed in whole or in part at 360 degrees, and (1) a plane projection version (text, illustration and / or 2D image). It is possible to configure either (2) a 3D image or (3) a three-dimensional spherical dome image.
  • the VR viewing device 100 can be configured to be wirelessly or wiredly connected to the computer 600 or the cloud.
  • the computer 600 or the cloud is equipped with a storage means 610 composed of an arithmetic unit (not shown) and an arbitrary storage medium, and the storage means 610 is equipped with an initial image 10 and a VR generated from the initial image 10.
  • It is a configuration in which the video 20 is stored, and is also a configuration used for acquiring, calculating, and managing the position information and the like of the VR viewing device 100.
  • the initial video 10 and the VR video 20 generated from the initial video 10 can be managed, calculated, and held by the VR viewing device 100.
  • the image generation means 200 is a device that generates an initial image 10 that can be displayed on the VR viewing device 100.
  • the initial image 10 is composed of materials (images, moving images, etc.) such as a plane projection image, a 3D image, or a dome image, and can be, for example, a 360-degree all-sky image. Yes, it can be a flat image, it can be any of 3 dof and 6 dof, and it can be an image with disparity.
  • the image generation means 200 initially changes these basic image data into a flat output image (360 degree image or the like is also a kind of flat output image) which is a format that can be displayed on the VR viewing device 100. Generate video 10.
  • the image generation means 200 is configured such that the arithmetic unit of the VR viewing device 100 performs arithmetic processing based on various data stored in the storage medium 610 incorporated in the VR viewing device 100 to generate the initial image 10.
  • the format is not limited to this, and for example, a computer 600 or a cloud is provided outside, and an arithmetic unit performs arithmetic processing based on various data stored in the storage medium 610 on the computer 600 or the cloud. It is also possible to have a configuration that generates the initial video 10.
  • the video generation means 200 is configured to include software that the arithmetic unit reads from the storage medium 610 and processes the video, but the video generation means 200 is not limited thereto.
  • the position information acquisition means 300 is a means for acquiring each position information P of the VR viewing device 100.
  • the user of the VR video space generation system 1 according to the present invention is equipped with the VR viewing device 100, and grasps the current position information regarding a specific part of the body of each user by each VR viewing device or an external sensor.
  • the position of each other part of the body is calculated based on the position information P, the VR image generation means draws based on the information, and the avatar V of each user is displayed on the VR viewing device 100. Can be displayed.
  • the position information acquisition means 300 is equipped in each VR viewing device 100, and the position information P is calculated by the camera of the VR viewing device 100, but the present invention is not limited to this. It is also possible to install a sensor on the outside and use other technologies such as tracking by laser irradiation. Further, in the present embodiment, the position information acquisition means 300 is configured to include software that is read from the storage medium 610 by the arithmetic unit incorporated in the VR viewing device 100 and processed, but is not limited thereto.
  • the position information acquisition means 300 can include a configuration for acquiring, calculating and managing the position information of the user in a remote place.
  • the position information of the user in a remote place is acquired, the position information acquisition means 300 performs the calibration process, and the position information is assigned to the VR space in which another user exists so as to exist in that space. Perform video processing.
  • the calibration process is performed in the same manner, and the image processing is performed so as to move in the same way in the VR space.
  • one or more users in remote areas can experience and share the VR image space at the same time via the avatar image V, even if the users are in remote positions.
  • the arithmetic unit shifts the center of the calibrated world of each user (for example, if it moves a few centimeters to the right and a few centimeters to the left in the real world, it moves from the center of the world in VR space to the right. (Move a few centimeters, a few centimeters to the left), or when the user overlaps with another user's avatar, landscape, or object, it is possible to make it transparent (thin), etc. It is also possible to incorporate a configuration that performs video processing that enhances the user's immersive feeling in VR.
  • each position information P of the VR viewing device 100 can be numerically defined as the coordinates consisting of the XYZ axes.
  • the VR image generation means 400 is a means for synthesizing the avatar image V with the initial image 10 generated by the image generation means 200 to generate the VR image 20.
  • the VR image generation means 400 specifies a position and an orientation (or a posture) to be arranged in the initial image 10 based on the position information P of each VR viewing device 100 acquired by the position information acquisition unit 300. Then, a VR image 20 is generated by synthesizing the avatar image V of each user with the initial image 10. As a result, a VR image 20 in a state in which the realistic position of each user wearing the VR viewing device 100 is reflected in the initial image 10 is generated, and as shown in FIG. 2, each user is a VR image. It is possible to configure a state of participating in a virtual space consisting of 20.
  • the user can view the image of the virtual space displayed by another user and includes the image of the virtual space including the avatar of another person that can be seen from the position in the virtual space where he / she is, and he / she also enters the virtual space. It becomes possible to obtain a feeling as if it were, and the immersive feeling in the VR image 20 is enhanced.
  • the VR image generation means 400 is configured to include software that is read from the storage medium 610 and processed by the arithmetic unit incorporated in the VR viewing device 100, but is limited to this.
  • a computer 600 or a cloud may be provided externally, and the VR image generation means 400 installed in the computer 600 or on the cloud may perform arithmetic processing to generate the VR image 20.
  • the computer 600 or the cloud can acquire and manage the location information P, and the VR video 20 can be configured to be generated by the VR viewing device 100 that has acquired the information, and VR can be generated in any other manner. It is possible to select a configuration for generating the moving image 20.
  • the video output means 500 is a means for outputting the VR video 20 generated by the VR video generation means 400, in which the avatar video V of each user is synthesized with the initial video 10, to each VR viewing device 100. Since the position and orientation of each VR viewing device 100 are different, the images output to the VR viewing device 100 will be different (see FIG. 2).
  • the video output means 500 is configured by software that is read from the storage medium 610 by the arithmetic unit incorporated in the VR viewing device 100 and processed, but is not limited to this configuration, for example. It is also possible to provide an external computer 600 and have the video output means 500 equipped in the computer 600 perform arithmetic processing to output the VR video 20 to the VR viewing device 100 by wire or wirelessly. It is also possible to perform the processing on the cloud.
  • the VR video space generation system 1 prepares in advance after the arithmetic unit analyzes the video of the space acquired by the camera mounted on the VR viewing device 100 worn by the user. It is configured to project the spatial image onto the VR viewing device 100 in association with the image of the VR space. Further, as described above, the camera mounted on each VR viewing device 100 acquires the depth information, creates a mesh model that virtually exists according to the viewpoint position, and acquires and calculates the position information P. Then, a flat output image (360 degree image or the like is also a kind of flat output image) that can be seen from the angle is generated and displayed.
  • a flat output image 360 degree image or the like is also a kind of flat output image
  • the virtual space that each user sees through the VR viewing device 100 corresponds to the scenery seen in the real space including other users, and becomes the VR image 20 that combines the image of the virtual space with the avatar image V. ..
  • the VR video space generation system 1 includes an area defining means 310, a position defining means 320, and an associating means 410 in the VR viewing device 100.
  • the area defining means 310 is, for example, as shown in FIG. 3, a means for defining a user-movable area F in the virtual space generated by the VR video space generation system 1.
  • the area F coincides with a certain area provided in the real space.
  • the area is the actual space in which the user arbitrarily moves or sits.
  • the area defining means 310 defines a virtual space that matches this space as the area F.
  • the region F is numerically defined as, for example, coordinates consisting of XYZ axes, but is not limited to this, and it is possible to configure the VR space to be grasped and managed by using other region management means. ..
  • the area defining means 310 is composed of software that is read from the storage medium 610 by the arithmetic unit of the VR viewing device 100 and processed, and in particular, is configured to include a software module or the like incorporated in the position information acquisition means 300.
  • the present invention is not limited to this, and it may be configured to be processed on independent software, software embedded in separately provided hardware, or the cloud.
  • the position defining means 320 is a means for numerically defining each position information P of the VR viewing device 100 as coordinates having XYZ axes. When the X value, Y value and / or Z value of the user's position information exceeds the maximum value, an exception handling such as displaying a warning message on the VR viewing device 100 can be considered.
  • the position defining means 320 is composed of software that is read from the storage medium 610 by the arithmetic unit of the VR viewing device 100 and processed, and in particular, is composed of a software module or the like incorporated in the position information acquiring means 300.
  • the present invention is not limited to this, and it is of course possible to use independent software, software embedded in separately provided hardware, or a configuration for processing on the cloud.
  • the associating means 410 is a means for introducing and applying one or a plurality of position information Ps to the area F and then associating them with the initial image 10. Specifically, the associating means 410 introduces (applies) the position information P defined by the position defining means 320 as a coordinate value to the area F defined by the area defining means 310. Then, the area F and each position information P are associated with the initial image 10 generated by the image generation means 200. Specifically, for example, in this embodiment, a configuration using a technique related to inverse kinematics is possible. The details of this mapping will be described later.
  • each position information P exists in the real space in the area F defined to match the real space. Defined to match the user's position. Then, in order to apply each of the information to the initial image 10, the avatar image V of each user is displayed at a position where the avatar image V of each user is actually present in the initial image 10.
  • the associating means 410 is composed of software that is read from the storage medium 610 by the arithmetic unit of the VR viewing device 100 and processed, and in particular, is composed of a software module or the like incorporated in the VR video generating means 400.
  • the present invention is not limited to this, and it is of course possible to use independent software, software embedded in separately provided hardware, or a configuration for processing on the cloud.
  • the VR video space generation system 1 is configured to include an area defining means 310, a position defining means 320, and an associating means 410 in the computer 600.
  • the VR video space generation system 2 can be configured without using the position information acquisition means 300 as shown in FIGS. 4a and 4b. That is, the configuration is such that the current and current realistic position of each VR viewing device 100 worn by each user is not acquired, and the position defining means 320 calculates the user's virtual position information P in the virtual space and XYZ axes.
  • the VR image generation means 400 uses the position information P to numerically define the coordinates as the coordinates consisting of the coordinates, and the VR image generation means 400 attaches the VR viewing device 100 to the initial image 10 generated by the image generation means 200 as an avatar image of each user. V is combined to generate a VR image 20. At this time, it is conceivable to set an arbitrary position so that the avatars of users in remote areas who are not there do not overlap each other.
  • the position defining means 320 arbitrarily sets the position information P on the assumption that the position information P is at an arbitrary place in the real space while communicating with the other VR viewing device 100. It is numerically defined as the coordinates consisting of the XYZ axes.
  • the arithmetic unit performs a process of shifting the center of the calibrated world of each user, and a process of making (thinning) the avatar, landscape, or object of another user when the user overlaps with the user.
  • the calibration process may be performed by the position defining means 320.
  • each position information P of the VR viewing device 100 can be numerically defined as the coordinates consisting of the XYZ axes.
  • FIG. 4b it is possible to provide a configuration in which an external computer 600 is provided or a cloud is used to centrally manage various types of information.
  • the computer 600 or the position defining means 320 on the cloud communicates with each VR viewing device 100 and then the position information P is located at an arbitrary place in the real space. It is configured to be arbitrarily set and numerically defined as the coordinates consisting of the XYZ axes.
  • the position defining means 320 is configured to acquire information related to the position of a specific part of the body of the user wearing the VR viewing device 100 and to calculate the position of each other part of the body. ..
  • the associating means 410 associates the area F with the position information P including these information with the initial image 10 generated by the image generating means 200.
  • the VR image generation means 400 draws based on these information and generates a VR image 20 to be displayed on the VR viewing device 100. With this configuration, the VR image 20 displayed on the VR viewing device 100 worn by the user becomes accurate according to the movement of the user, and the user is immersed in the VR image 20 as if the virtual space is real. It is possible to get a feeling.
  • the technique related to inverse kinematics can be used to generate the avatar video V.
  • Inverse kinematics is a technique for calculating the position and rotation angle of a higher-level object by designating the target position of a lower-level object in an object having a hierarchical structure, and is used for calculating the operation of the avatar video V.
  • the VR viewing device 100 or the computer 600 incorporates an arithmetic unit, and the position defining means 320 acquires information related to the position of a specific part of the user's body.
  • the arithmetic unit performs arithmetic processing on this information using the inverse kinematics technique, identifies the positions of other parts of the body, and the mapping means 410 performs the mapping processing, and then the VR viewing device 100. It is a configuration to generate a VR image 20 to be displayed in.
  • the user in the presentation venue watches the presentation by the VR viewing device 100 in the virtual space superimposed on the real space, and is in a remote place.
  • the user also watches the presentation by the VR viewing device 100 in the same virtual space superimposed on the real space. Since the avatar image V is displayed on the VR image 20 for the user in the remote place, the user in the presentation venue can recognize that the user in the remote place is also in the same place.
  • a user in a remote place can get the experience of being in the presentation venue through the VR viewing device 100.
  • the VR video space generation systems 1 and 2 have a configuration corresponding to 6 dof or 3 dof.
  • the 3DOF is a VR viewing device 100 corresponding to three movements of the X-axis, the Y-axis, and the Z-axis, and senses the rotation and tilt of the head equipped with the VR viewing device 100.
  • the 6 dof has a configuration corresponding to 6 movements in which movements in the X-axis, Y-axis, and Z-axis directions are added in addition to the movements of 3 dof.
  • the user can enjoy the 3DOF image while maintaining the immersive feeling of 6DOF. That is, it is possible to use the illusion of the experience of 6DOF to show a large number of existing 3DOF images while maintaining the immersive feeling of 6DOF.
  • an image equivalent to what is shown in the range that can be seen in the captured 3dof image is CG generated in 6dof, and then the 3dof is guided to the position where the 3dof was photographed in the 6dof experience, and the immersive feeling can be maintained. It is possible to replace 6dof with 3dof in a suitable position and show this continuously to show 3dof while maintaining the immersive feeling of 6dof.
  • the VR video space generation systems 1 and 2 give each user the feeling of being in a virtual space, allowing the users to communicate with each other in the virtual space, and a plurality of users in remote locations. Can be collected in one VR space, and it is possible to experience presentations, amusements, exhibitions, training, etc. in the VR space.
  • the VR video space generation system 1 can be operated only locally without using an external network such as the Internet. That is, it is possible to give a presentation or the like using the VR video space generation system 1 even in a situation where there is no network environment that can be connected to the outside such as the Internet.
  • the initial video 10 is generated in a general file format including PNG, MP4, etc., and can be easily replaced without changing the main body of the VR video space generation system, and is a readable independent plane. It is composed of projected images, 3D images, dome images, and the like. With this configuration, it has become possible for users who use the VR video space generation systems 1 and 2 to experience materials of all formats and types in the VR space.
  • the initial video 10 is generated in a general file format, and can be easily replaced without changing the VR video space generation system main body. It is a readable independent illustration, 2D image and characters that are switched according to the user's instruction. It is possible to configure a configuration including a plane projection image including information and a plane projection image composed of a 2D moving image that operates according to a user's instruction. Further, it is possible to configure the configuration to include a stereoscopic model of CG that operates according to the user's instruction, or to include a spherical image that operates according to the user's instruction or a part of the image.
  • Each of these images is generated by a general file format such as PNG or MP4, and is fixedly embedded in the VR space (in the initial image 10) or always displayed in the direction in which the user turns his face. It is an independent image that can be displayed or any display method can be selected. Moreover, since it can be embedded arbitrarily, it can be easily replaced without changing other systems.
  • the user can view the plane projection image, 3D image, and spherical image embedded in the virtual space, in order to introduce the structure and specifications of automobiles, real estate, etc. Presentations, amusements, exhibitions, and training can now be easily experienced and shared by users. In addition, it has become possible for multiple users to experience and share various attractions.
  • the initial video 10 can be configured to be composed of independent video data that can be read by the video generation means 200.
  • the video generation means 200 can read an arbitrary initial video 10 from various existing materials including image / video data generated in a general file format, and one VR video space. It has become possible to display an arbitrary VR video space desired by the generation system and let the user experience it.
  • presentation materials created in another format are inserted at a fixed position and displayed in the VR space in which the user participates (for example, the VR space is composed of an auditorium).
  • Presentation materials are fixed and embedded behind the lecture hall).
  • the VR video space generation system 1 and 2 it is possible to include an image in the VR space.
  • the video of the instructor embedded in the virtual space is always displayed in front of the user. It is possible to provide a VR video space generation system 1 and 2 that can always display a projected image at a specific position (for example, in front of the eyes) regardless of the user's posture and is highly convenient for the user. Is now possible.
  • the VR viewing device 100 includes a sensor 110 in this embodiment.
  • the sensor 110 is a sensor that detects the position and movement of the fingers of both hands of the user wearing the VR viewing device 100.
  • the sensor 110 detects the movements of the hands and fingers of the user wearing the VR viewing device 100, and then acquires the information related to the movements as the tracking information T.
  • the tracking information T is information related to the movements of the fingers of both hands of the user, and is composed of information related to a series of movements of the hands and fingers within a certain period of time.
  • the sensor 100 may be installed in a device other than the VR viewing device 100 to detect the positions and movements of the fingers of both hands of the user who wears the VR viewing device 100.
  • the path of a hand or finger from a certain point to a certain point is traced for each of the left and right hands and acquired and saved as tracking information T. do.
  • the VR video space generation systems 1 and 2 hold the action information A.
  • the action information A is information that tracks a series of finger movements within a certain period of time, and in this embodiment, a plurality of patterns of action information A are retained.
  • the action information A is composed of data stored in the storage medium 610 of the VR viewing device 100 or the computer 600.
  • each of the plurality of action information A is configured to be associated with a specific change process of the initial image 10. Further, when the acquired tracking information T matches any of the stored action information A, the change processing of the associated initial video 10 is performed.
  • the initial video 10 includes a presentation image and the like
  • the action information A is associated with the progress and backward processing of the initial video.
  • the action of moving the finger from right to left is registered and held as the action information A
  • the action information A changes the image so as to sequentially switch the presentation image displayed in a part of the initial image 10.
  • the sensor 110 detects the position and movement of the finger, and the information related to the operation is used as the tracking information T. get.
  • the VR video space generation systems 1 and 2 perform processing for comparing the tracking information T that traces the movement of the finger and the action information A, and when it is determined that they are the same, the associated presentation video is sequentially switched. It is a configuration that processes images.
  • action information A such as the movement of making a circle by contacting the thumb with the index finger or middle finger.
  • the action information A is configured to include instruction information generated by simultaneously operating the fingers of both hands of the user.
  • the initial video 10 includes a presentation video composed of a plurality of images, and the action information A is associated with the progress and backward processing of the presentation video.
  • the action of contacting the thumb and the index finger of both hands or the thumb and the middle finger is registered as the action information A, and this action is made to correspond to the progress (or backward) processing of the presentation image.
  • the VR video space generation systems 1 and 2 detect tracking information T that traces the movement of the finger.
  • comparison with the action information A is performed, it is confirmed that they match, and the progress processing of the presentation video associated with the action information A is performed.
  • the user can feel as if he / she is in the virtual space, communicate with each other in the virtual space, and multiple users in remote areas gather in one VR space.
  • various materials including independent images and videos that can be chilled, generated in a general file format, and readable, into the VR video space as they are, you can watch them in the virtual space.
  • you can watch them in the virtual space.
  • Schematic diagram of the VR video space generation system Schematic diagram of a VR video space generation system equipped with an external computer Schematic diagram showing a display example of a VR image Diagram showing area / location information
  • Schematic diagram of VR video space generation system provided to users in remote areas Schematic diagram of a VR video space generation system provided to users in remote areas equipped with an external computer Schematic diagram of a VR video space generation system that switches video materials and scenes

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Graphics (AREA)
  • Computer Hardware Design (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Processing Or Creating Images (AREA)
PCT/JP2020/043282 2020-11-19 2020-11-19 Ｖｒ映像空間生成システム Ceased WO2022107294A1 (ja)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/JP2020/043282 WO2022107294A1 (ja) 2020-11-19 2020-11-19 Ｖｒ映像空間生成システム
JP2022563511A JP7547501B2 (ja) 2020-11-19 2020-11-19 Ｖｒ映像空間生成システム

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2020/043282 WO2022107294A1 (ja) 2020-11-19 2020-11-19 Ｖｒ映像空間生成システム

Publications (1)

Publication Number Publication Date
WO2022107294A1 true WO2022107294A1 (ja) 2022-05-27

Family

ID=81708626

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2020/043282 Ceased WO2022107294A1 (ja) 2020-11-19 2020-11-19 Ｖｒ映像空間生成システム

Country Status (2)

Country Link
JP (1) JP7547501B2 (https=)
WO (1) WO2022107294A1 (https=)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024131204A1 (zh) * 2022-12-23 2024-06-27 南京欧珀软件科技有限公司 虚拟场景设备交互方法及相关产品

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP6203369B1 (ja) * 2016-12-08 2017-09-27 株式会社コロプラ 情報処理方法及び当該情報処理方法をコンピュータに実行させるためのプログラム
JP2017529635A (ja) * 2014-06-14 2017-10-05 マジック リープ， インコーポレイテッドＭａｇｉｃ Ｌｅａｐ，Ｉｎｃ． 仮想および拡張現実を作成する方法およびシステム
WO2020129115A1 (ja) * 2018-12-17 2020-06-25 株式会社ソニー・インタラクティブエンタテインメント 情報処理システム、情報処理方法およびコンピュータプログラム

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10156908B2 (en) * 2015-04-15 2018-12-18 Sony Interactive Entertainment Inc. Pinch and hold gesture navigation on a head-mounted display
JP6470374B1 (ja) * 2017-10-03 2019-02-13 株式会社コロプラ 仮想現実を提供するためにコンピュータで実行されるプログラムおよび情報処理装置

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2017529635A (ja) * 2014-06-14 2017-10-05 マジック リープ， インコーポレイテッドＭａｇｉｃ Ｌｅａｐ，Ｉｎｃ． 仮想および拡張現実を作成する方法およびシステム
JP6203369B1 (ja) * 2016-12-08 2017-09-27 株式会社コロプラ 情報処理方法及び当該情報処理方法をコンピュータに実行させるためのプログラム
WO2020129115A1 (ja) * 2018-12-17 2020-06-25 株式会社ソニー・インタラクティブエンタテインメント 情報処理システム、情報処理方法およびコンピュータプログラム

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024131204A1 (zh) * 2022-12-23 2024-06-27 南京欧珀软件科技有限公司 虚拟场景设备交互方法及相关产品

Also Published As

Publication number Publication date
JP7547501B2 (ja) 2024-09-09
JPWO2022107294A1 (https=) 2022-05-27

Similar Documents

Publication Publication Date Title
JP7681609B2 (ja) ３ｄオブジェクト注釈
US7626569B2 (en) Movable audio/video communication interface system
JP7539478B2 (ja) セッションマネージャ
Anthes et al. State of the art of virtual reality technology
JP7627720B2 (ja) 複合現実における空間命令およびガイド
US12488699B2 (en) Class system, viewing terminal, information processing method, and program
JP2022549853A (ja) 共有空間内の個々の視認
JP5739922B2 (ja) 仮想インタラクティブプレゼンスのシステムおよび方法
KR100809479B1 (ko) 혼합 현실 환경을 위한 얼굴 착용형 디스플레이 장치
Komiyama et al. JackIn space: designing a seamless transition between first and third person view for effective telepresence collaborations
Stavness et al. pCubee: a perspective-corrected handheld cubic display
US20060250392A1 (en) Three dimensional horizontal perspective workstation
JP7544071B2 (ja) 情報処理装置、情報処理システム及び情報処理方法
JP2019175323A (ja) シミュレーションシステム及びプログラム
JP7762780B2 (ja) 非均一ステレオレンダリング
Peterson Virtual Reality, Augmented Reality, and Mixed Reality Definitions
JP7104539B2 (ja) シミュレーションシステム及びプログラム
CN108830944A (zh) 光学透视式三维近眼显示系统及显示方法
JP7530754B2 (ja) 教育支援システム、方法およびプログラム
Darwish Virtual Reality Technologies and Real Life Applications
JP7547501B2 (ja) Ｖｒ映像空間生成システム
JP7011746B1 (ja) コンテンツ配信システム、コンテンツ配信方法、及びコンテンツ配信プログラム
CN114638734A (zh) 一种桌面3d教育系统
Wong HandsOn: a portable system for collaboration on virtual 3D objects using binocular optical head-mounted display
Nesamalar et al. An introduction to virtual reality techniques and its applications

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20962457

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2022563511

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20962457

Country of ref document: EP

Kind code of ref document: A1