CN109427095B - Method and system for displaying mixed reality scene - Google Patents
Method and system for displaying mixed reality scene Download PDFInfo
- Publication number
- CN109427095B CN109427095B CN201710748651.2A CN201710748651A CN109427095B CN 109427095 B CN109427095 B CN 109427095B CN 201710748651 A CN201710748651 A CN 201710748651A CN 109427095 B CN109427095 B CN 109427095B
- Authority
- CN
- China
- Prior art keywords
- equipment
- positioning
- scene
- player
- virtual
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/006—Mixed reality
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
- A63F13/20—Input arrangements for video game devices
- A63F13/21—Input arrangements for video game devices characterised by their sensors, purposes or types
- A63F13/213—Input arrangements for video game devices characterised by their sensors, purposes or types comprising photodetecting means, e.g. cameras, photodiodes or infrared cells
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
- A63F13/25—Output arrangements for video game devices
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
- A63F13/25—Output arrangements for video game devices
- A63F13/26—Output arrangements for video game devices having at least one additional display device, e.g. on the game controller or outside a game booth
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
- A63F13/50—Controlling the output signals based on the game progress
- A63F13/52—Controlling the output signals based on the game progress involving aspects of the displayed game scene
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
- A63F13/60—Generating or modifying game content before or while executing the game program, e.g. authoring tools specially adapted for game development or game-integrated level editor
- A63F13/65—Generating or modifying game content before or while executing the game program, e.g. authoring tools specially adapted for game development or game-integrated level editor automatically by game devices or servers from real world data, e.g. measurement in live racing competition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F2300/00—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
- A63F2300/10—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by input arrangements for converting player-generated signals into game device control signals
- A63F2300/1087—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by input arrangements for converting player-generated signals into game device control signals comprising photodetecting means, e.g. a camera
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F2300/00—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
- A63F2300/30—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by output arrangements for receiving control signals generated by the game device
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F2300/00—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
- A63F2300/60—Methods for processing data by generating or executing the game program
- A63F2300/69—Involving elements of the real world in the game world, e.g. measurement in live races, real video
Abstract
The invention relates to the field of mixed reality, in particular to a method and a system for displaying a mixed reality scene. The invention configures the authority for more than three MR devices; the permissions comprise player permissions, on-site observer permissions and off-site observer permissions; the player MR equipment with the player authority modifies the attribute value of a virtual object in the virtual scene corresponding to the player MR equipment; the spectator MR equipment and the server with the permission of the observers in the field respectively update the virtual scenes corresponding to the spectator MR equipment and the server according to the attribute values; the entity camera acquires a real scene; the positioning MR device with the authority of the off-site observer is physically fixed with the physical camera; the server superposes the real scene and the virtual scene corresponding to the server according to the real-time position information of the positioning MR equipment to obtain a mixed reality scene; and displaying the mixed reality scene by the display equipment. The mixed reality scene observed by the player wearing the MR equipment is displayed in a third person perspective.
Description
Technical Field
The invention relates to the field of mixed reality, in particular to a method and a system for displaying a mixed reality scene.
Background
VR (virtual reality) technology is known today to simulate an environment by computer-generated real-time dynamic three-dimensional stereo images and to use sensing devices to enable interaction. VR technology creates a new virtual world, which cannot be linked with the real world.
MR (Mix Reality mixed Reality) technology has been proposed in recent years by Microsoft corporation to merge virtual and Reality together. The general process is as follows:
1) MR-enabled devices model in real time by constantly scanning the real-world environment around the experiencer.
2) The physical coordinates of the device in real space are calculated using a Spatial Understanding (Spatial Understanding) technique.
3) The real world is mapped into a virtual coordinate system using a Spatial Mapping (Spatial Mapping) technique with the MR device as the origin of coordinates.
4) The virtual environment generated by calculation is superposed on the real world, and the interaction with virtual things is realized through gestures, so that the virtual environment has more natural user experience compared with a VR device.
The current MR game on Windows Store can be divided into a stand-alone version and a networked version. The networking version is that a PC is appointed as a server, then a plurality of MR devices are connected together through a local area network, and the server is responsible for managing and forwarding scene data, so that the consistent virtual scene seen on different terminals is realized.
However, only the players who actually participate in the existing halftone MR game can see the virtual and real combined pictures, and other people cannot obtain the same experience as the players through the spectator.
Disclosure of Invention
The technical problem to be solved by the invention is as follows: how to display a mixed reality scene viewed by a player wearing the MR device from a third person perspective.
In order to solve the technical problems, the invention adopts the technical scheme that:
the invention provides a method for displaying a mixed reality scene, which comprises the following steps:
s1, configuring authorities for more than three MR devices; the permissions include player permissions, on-site observer permissions and off-site observer permissions;
s2, the MR equipment of the player with the player authority modifies the attribute value of a virtual object in the virtual scene corresponding to the MR equipment of the player;
s3, the onlooker MR equipment with the in-field observer authority and the server respectively update the virtual scenes corresponding to the server according to the attribute values;
s4, the entity camera acquires a real scene; a positioning MR device with off-site observer authority is physically fixed with the physical camera;
s5, the server superimposes the real scene and a virtual scene corresponding to the server according to the real-time position information of the positioning MR equipment to obtain a mixed reality scene;
and S6, displaying the mixed reality scene by the display equipment.
The invention also provides a system for displaying the mixed reality scene, which comprises more than three MR devices, an entity camera, a display device and a server;
the server includes one or more processors and memory, the memory storing a program and configured to perform the following steps by the one or more processors:
s1, configuring authority for more than three MR devices; the permissions comprise player permissions, on-site observer permissions and off-site observer permissions;
s2, when the MR equipment of the player with the player authority modifies the attribute value of a virtual object in the virtual scene corresponding to the MR equipment of the player, acquiring the attribute value;
s3, updating a virtual scene corresponding to the server according to the attribute value; triggering the onlooker MR equipment with the in-field observer authority to update the virtual scene corresponding to the onlooker MR equipment according to the attribute value;
s4, acquiring a real scene from the entity camera; the positioning MR device with the authority of the off-site observer is physically fixed with the physical camera;
s5, superposing the real scene and the virtual scene corresponding to the server according to the real-time position information of the positioning MR equipment to obtain a mixed reality scene;
and S6, triggering display equipment to display the mixed reality scene.
The invention has the beneficial effects that: the physical camera and the MR equipment are physically fixed, so that a server can indirectly acquire the position information of the physical camera according to the position information of the MR equipment, a virtual scene matched with a real scene can be acquired when the real camera acquires the real scene at any position, the real scene and the virtual scene can be superposed, and a mixed reality scene which can be observed by a player wearing the MR equipment is displayed at the view angle of the current position of the physical camera; viewers who do not wear MR devices can also experience mixed reality scenes viewed by players through live display devices. Meanwhile, the user operation data of the player can be synchronously updated to the MR device with the permission of the observer in the field, and the commentator can better experience the mixed reality scene watched by the player through the MR device with the permission of the observer in the field.
Drawings
FIG. 1 is a block flow diagram of an embodiment of a method for displaying a mixed reality scene;
fig. 2 is a block diagram of a specific implementation of a terminal for displaying a mixed reality scene according to the present invention;
FIG. 3 is a diagram of a hardware deployment according to a second embodiment;
description of the reference symbols:
1. an MR device; 2. a physical camera; 3. a display device; 4. a server; 41. a processor; 42. a memory.
Detailed Description
In order to explain technical contents, achieved objects, and effects of the present invention in detail, the following description is made with reference to the accompanying drawings in combination with the embodiments.
The noun interpretation:
referring to fig. 1 to 3 of the drawings,
as shown in fig. 1, the present invention provides a method for displaying a mixed reality scene, including:
s1, configuring authorities for more than three MR devices; the permissions include player permissions, on-site observer permissions and off-site observer permissions;
s2, the MR equipment of the player with the player authority modifies the attribute value of a virtual object in the virtual scene corresponding to the MR equipment of the player;
s3, the onlooker MR equipment and the server with the permission of the observers in the field respectively update the virtual scenes corresponding to the onlooker MR equipment and the server according to the attribute values;
s4, the entity camera acquires a real scene; the positioning MR device with the authority of the off-site observer is physically fixed with the physical camera;
s5, the server superimposes the real scene and a virtual scene corresponding to the server according to the real-time position information of the positioning MR equipment to obtain a mixed reality scene;
and S6, displaying the mixed reality scene by the display equipment.
Further, the method also comprises the following steps:
acquiring a first space model corresponding to the player MR device;
acquiring a second space model corresponding to the onlooker MR device;
acquiring a third spatial model corresponding to the positioning MR device;
presetting a virtual object with a constant position;
and mapping the first space model, the second space model and the third space model to the same coordinate system by taking the virtual object with the constant position as a reference object.
As can be seen from the above description, a fixed virtual object is preset at a position in a real space, and the virtual object is used as a reference object to map a third space model corresponding to a positioning MR device physically fixed by a physical camera, a first space model corresponding to a player MR device worn by a player, and a second space model corresponding to a spectator MR device worn by an in-field commentator to the same coordinate system, so as to synchronize space models corresponding to different MR devices, and enable other MR devices to accurately update their own virtual scenes when data of the player MR devices is updated.
Further, the S5 specifically is:
acquiring the coordinates of the positioning MR equipment in the coordinate system in real time to obtain positioning coordinates;
acquiring a field angle of an entity camera;
converting a virtual scene corresponding to the server according to the field angle, the positioning coordinates and the current rotation offset of the positioning MR equipment to obtain a current virtual scene; rendering the current virtual scene to the real scene.
Further, still include:
horizontally placing a physical camera;
modifying the initial horizontal rotation offset of the positioning MR device to be zero;
calculating the vertical rotation offset of the positioning MR equipment relative to the entity camera according to the real scene and the virtual scene corresponding to the positioning MR equipment to obtain the relative vertical rotation offset;
the initial vertical rotational offset of the positioning MR device is modified depending on the relative vertical rotational offset.
As can be seen from the above description, since the physical fixation of the physical camera and the positioning MR device is difficult to ensure that the positioning MR device is parallel to the physical camera, the real-time rotational offset of the positioning MR device is difficult to accurately reflect the real-time rotational offset of the physical camera. However, the amount of rotational displacement has a great influence on the final effect, and for example, a rotational deviation of the subject from the camera 10m,1 ° causes a translational misalignment of at least 17 cm. According to the method, the solid camera is horizontally placed by means of the level meter, according to the principle that the XOZ plane is consistent with the real horizontal plane when the MR equipment establishes a self coordinate system, the initial horizontal rotation offset of the positioning MR equipment is modified to be zero no matter how much the current horizontal rotation offset of the positioning MR equipment is, and then the initial vertical rotation offset is modified, namely the initial orientation of the virtual camera corresponding to the positioning MR equipment is modified to be consistent with the initial orientation of the solid camera, so that the real-time rotation offset of the positioning MR equipment is consistent with the real-time rotation offset of the solid camera.
Further, the S3 specifically is:
when the server receives an attribute value from one MR device, verifying the authority of the MR device to obtain a verification result; the verification result comprises player authority, on-site observer authority and off-site observer authority;
if the verification result is the player authority, then:
the server updates a virtual scene corresponding to the server according to the attribute value;
the server sends the attribute value to the onlooker MR equipment;
and the onlooker MR device updates the virtual scene corresponding to the onlooker MR device according to the attribute value.
According to the description, when the server receives the updating data corresponding to the virtual scene, the authority of the MR device of the updating data source is verified, and the disorder of the game process caused by the misoperation of other MR devices except the player MR device is avoided.
As shown in fig. 2, the present invention further provides a system for displaying a mixed reality scene, which includes more than three MR devices 1, an entity camera 2, a display device 3 and a server 4;
the server 4 comprises one or more processors 41 and a memory 42, the memory 42 storing programs and being configured to perform the following steps by the one or more processors 41:
s1, configuring authorities for more than three MR devices; the permissions include player permissions, on-site observer permissions and off-site observer permissions;
s2, when the MR equipment of the player with the player authority modifies the attribute value of a virtual object in the virtual scene corresponding to the MR equipment of the player, acquiring the attribute value;
s3, updating a virtual scene corresponding to the server according to the attribute value; triggering the onlooker MR equipment with the in-field observer authority to update the virtual scene corresponding to the onlooker MR equipment according to the attribute value;
s4, acquiring a real scene from the entity camera; the positioning MR device with the authority of the off-site observer is physically fixed with the physical camera;
s5, superposing the real scene and the virtual scene corresponding to the server according to the real-time position information of the positioning MR equipment to obtain a mixed reality scene;
and S6, triggering display equipment to display the mixed reality scene.
Further, still include:
acquiring a first space model corresponding to the player MR device;
acquiring a second space model corresponding to the bystander MR equipment;
acquiring a third spatial model corresponding to the positioning MR device;
presetting a virtual object with a constant position;
and mapping the first space model, the second space model and the third space model to the same coordinate system by taking the virtual object with the constant position as a reference object.
Further, the S5 specifically is:
acquiring coordinates of the positioning MR equipment in the coordinate system in real time to obtain positioning coordinates;
acquiring the field angle of the entity camera;
converting a virtual scene corresponding to the server according to the field angle, the positioning coordinates and the current rotation offset of the positioning MR equipment to obtain a current virtual scene; rendering the current virtual scene to the real scene.
Further, still include:
horizontally placing a solid camera;
modifying the initial horizontal rotational offset of the positioning MR device to zero;
calculating the vertical rotation offset of the positioning MR equipment relative to the entity camera according to the real scene and the virtual scene corresponding to the positioning MR equipment to obtain the relative vertical rotation offset;
the initial vertical rotational offset of the positioning MR device is modified depending on the relative vertical rotational offset.
Further, the S3 specifically is:
when an attribute value from one MR device is received, verifying the authority of the MR device to obtain a verification result; the verification result comprises player authority, on-site observer authority and off-site observer authority;
if the verification result is the player authority, then:
updating a virtual scene corresponding to the server according to the attribute value;
sending the attribute value to a spectator MR device;
and triggering the onlooker MR device to update the virtual scene corresponding to the onlooker MR device according to the attribute value.
The first embodiment of the invention is as follows:
the embodiment provides a method for displaying a mixed reality scene, which comprises the following steps:
s1, configuring authority for more than three MR devices; the permissions include player permissions, on-site observer permissions and off-site observer permissions;
s2, acquiring a first space model corresponding to player MR equipment; acquiring a second space model corresponding to the onlooker MR device; acquiring a third spatial model corresponding to the positioning MR device;
the player MR equipment is configured with player authority, directly participates in game interaction, and can control part of game elements in the game; the onlooker MR equipment is configured with the authority of an observer in the field, can freely select an angle for watching a virtual-real combined scene, personally watches the game process of a player in the scene, but cannot operate game elements; the positioning MR equipment is configured with the authority of an off-site observer, and the position information of the positioning MR equipment is sent to the server in real time, so that the server can indirectly obtain the position information of the entity camera physically fixed with the positioning MR equipment according to the position information of the positioning MR equipment.
S3, presetting a virtual object with a constant position; mapping the first space model, the second space model and the third space model to the same coordinate system by taking the virtual object with the constant position as a reference object;
optionally, when running an application, the MR device, such as a Hololens developed by Microsoft corporation, may establish a left-hand coordinate system with its own position as an origin of coordinates, a projection of a right front side of the MR device on a horizontal plane as a positive Z-axis direction, and a vertically upward direction as a positive Y-axis direction; successively scanning the same space by the player MR equipment, the spectator MR equipment and the positioning MR equipment, respectively modeling the same space in respective coordinate systems, and respectively obtaining a first space model, a second space model and a third space model; a common world coordinate system is established through a world anchor point (namely a virtual object with a fixed position in a real space) which is placed in a space in advance, a conversion matrix from different MR equipment coordinate systems to the world coordinate system is calculated, and virtual scenes constructed by different MR equipment are superposed;
s4, horizontally placing an entity camera; modifying the initial horizontal rotation offset of the positioning MR device to be zero; calculating the vertical rotation offset of the positioning MR equipment relative to the entity camera according to the real scene and the virtual scene corresponding to the positioning MR equipment to obtain the relative vertical rotation offset; modifying an initial vertical rotational offset of the positioning MR device in accordance with the relative vertical rotational offset;
optionally, a vertical rotation offset of the positioning MR device with respect to the physical camera is calculated according to the real scene and the virtual scene corresponding to the positioning MR device, so as to obtain a relative vertical rotation offset, specifically:
placing a virtual vertical slender model in a virtual scene corresponding to the positioning MR equipment, wherein the virtual vertical slender model is positioned in the middle of the visual field of a virtual camera of the positioning MR equipment; a vertical rod is arranged at a position far away from the entity camera in a real scene and is positioned in the middle of the visual field of the entity camera;
superposing a virtual scene and a real scene corresponding to the positioning MR equipment, if the virtual vertical slender model and the vertical rod are not superposed, then:
keeping the position of the virtual camera for positioning the MR device still, and adjusting the rotation amount (namely, rotating around a vertical Y axis) of the virtual camera for positioning the MR device along the horizontal direction, wherein the movement direction of the virtual vertical slender model in the virtual image is opposite to the rotation direction of the virtual camera for positioning the MR device;
and rotating the virtual camera of the positioning MR equipment until the virtual vertical slender model is superposed with the vertical rod, and taking a negative value of the rotation amount of the virtual camera of the positioning MR equipment around the Y axis as the vertical rotation offset.
S5, the player MR equipment with the player authority modifies the attribute value of a virtual object in the virtual scene corresponding to the player MR equipment;
optionally, the attributes of a virtual object include coordinates, a rotation angle, and a scaling; when a player operates a virtual object, the attribute value of the virtual object changes correspondingly.
S6, the onlooker MR equipment with the in-field observer authority and the server respectively update the virtual scenes corresponding to the server according to the attribute values; the method specifically comprises the following steps:
when the server receives an attribute value from one MR device, verifying the authority of the MR device to obtain a verification result; the verification result comprises player authority, on-site observer authority and off-site observer authority;
if the verification result is the player authority, then:
the server updates a virtual scene corresponding to the server according to the attribute value;
the server sends the attribute value to the onlooker MR equipment;
and the onlooker MR device updates the virtual scene corresponding to the onlooker MR device according to the attribute value.
Optionally, if there is more than one player MR device with player authority, the attribute value may be sent to other player MR devices, and the other player MR devices update the virtual scene corresponding to themselves according to the attribute value.
S7, the entity camera acquires a real scene; the positioning MR device with the authority of the off-site observer is physically fixed with the physical camera;
s8, the server superimposes the real scene and a virtual scene corresponding to the server according to the real-time position information of the positioning MR equipment to obtain a mixed reality scene; the method specifically comprises the following steps:
acquiring coordinates of the positioning MR equipment in the coordinate system in real time to obtain positioning coordinates;
acquiring the field angle of the entity camera;
converting a virtual scene corresponding to a server according to the field angle, the positioning coordinates and the current rotation offset of the positioning MR equipment to obtain a current virtual scene; rendering the current virtual scene to the real scene;
optionally, the server includes a renderer for rendering a virtual scene corresponding to the server onto a real scene acquired by the physical camera, so as to obtain a mixed reality scene.
Converting a virtual scene corresponding to the server according to the field angle, the positioning coordinates and the current rotation offset of the positioning MR equipment to obtain a current virtual scene, which specifically comprises the following steps:
setting the field angle parameter, the coordinate and the rotation offset parameter of the virtual camera corresponding to the renderer to be the same as the field angle, the positioning coordinate and the rotation offset of the entity camera; the parameters of the virtual camera and the parameters of the physical camera are completely consistent, so that the virtual scene shooting from the angle of the physical camera is simulated.
S9, displaying the mixed reality scene by display equipment;
alternatively, the reality device is a large screen connected to the server, and the spectators who do not wear the MR device can see the mixed reality scene viewed by the player through the player MR device through the large screen.
The second embodiment of the invention is as follows:
the embodiment provides a system for displaying a mixed reality scene, which comprises more than three MR devices, an entity camera, a display device and a server;
alternatively, the hardware deployment diagram is shown in fig. 3, wherein the icon of the smartphone represents a Hololens device. The dotted line connection refers to connection through a wireless network, the solid line connection refers to connection through a wired network, and the connection between the user and the equipment is physical wearing or holding;
the server includes one or more processors and memory, the memory storing a program and configured to perform the following steps by the one or more processors:
s1, configuring authorities for more than three MR devices; the permissions include player permissions, on-site observer permissions, and off-site observer permissions.
S2, acquiring a first space model corresponding to player MR equipment; acquiring a second space model corresponding to the bystander MR equipment; a third spatial model corresponding to the positioning of the MR device is acquired.
S3, presetting a virtual object with a constant position; and mapping the first space model, the second space model and the third space model to the same coordinate system by taking the virtual object with the constant position as a reference object.
S4, horizontally placing an entity camera; modifying the initial horizontal rotational offset of the positioning MR device to zero; calculating the vertical rotation offset of the positioning MR equipment relative to the entity camera according to the real scene and the virtual scene corresponding to the positioning MR equipment to obtain the relative vertical rotation offset; the initial vertical rotational offset of the positioning MR device is modified depending on the relative vertical rotational offset.
And S5, when the player MR equipment with the player authority modifies the attribute value of a virtual object in the virtual scene corresponding to the player MR equipment, acquiring the attribute value.
S6, updating the virtual scene corresponding to the server according to the attribute value; triggering the onlooker MR equipment with the in-field observer authority to update the virtual scene corresponding to the onlooker MR equipment according to the attribute value; the method comprises the following specific steps:
when an attribute value from one MR device is received, verifying the authority of the MR device to obtain a verification result; the verification result comprises player authority, on-site observer authority and off-site observer authority;
if the verification result is the player authority, then:
updating a virtual scene corresponding to the server according to the attribute value;
sending the attribute value to a spectator MR device;
and triggering the onlooker MR device to update the virtual scene corresponding to the onlooker MR device according to the attribute value.
S7, acquiring a real scene from the entity camera; the positioning MR device with the authority of the off-site observer is physically fixed with the physical camera.
S8, superposing the real scene and the virtual scene corresponding to the server according to the real-time position information of the positioning MR equipment to obtain a mixed reality scene; the method specifically comprises the following steps:
acquiring coordinates of the positioning MR equipment in the coordinate system in real time to obtain positioning coordinates;
acquiring the field angle of the entity camera;
converting a virtual scene corresponding to the server according to the field angle, the positioning coordinates and the current rotation offset of the positioning MR device 1 to obtain a current virtual scene; rendering the current virtual scene to the real scene.
And S9, triggering display equipment to display the mixed reality scene.
The third embodiment of the invention is as follows:
the embodiment of the invention provides a system for displaying a mixed reality scene to carry out an MR interactive game of Chaojie character.
The hardware required for this embodiment is shown in table 1:
TABLE 1
The background director clicks 'question making' on the server, and disordered virtual scenes of Chinese character fragments appear on all the MR equipment and a large screen connected with the program recording system;
the player 1 operates one of the shards, the data of the shard is uploaded to the server, and then the server broadcasts the data to the MR games running on the MR devices worn by all guests and all Mac Pro through a wireless route. In this embodiment, since the game of "chang jie" character building according to this embodiment is a competitive game, player 1 and player 2 cannot check the progress of each other, and data is not shared. However, the data of the fragment operated by the player 1 and the player 2 is broadcasted to the MR device and the server worn by the guest;
and outputting a virtual-real combined picture after the virtual scene on the virtual scene renderer of the server is mixed with the real scene shot by the camera, wherein the picture is presented in front of each scene appearance through a program recording system.
In summary, according to the method and system for displaying a mixed reality scene provided by the invention, the physical camera and the MR device are physically fixed, so that the server can indirectly obtain the position information of the physical camera according to the position information of the MR device, and thus when the physical camera obtains a real scene at any position, a virtual scene matched with the real scene can be obtained, the real scene and the virtual scene can be superimposed, and the mixed reality scene viewable by a player wearing the MR device can be displayed at the view angle of the current position of the physical camera; viewers who do not wear MR devices can also experience mixed reality scenes viewed by players through live display devices. Meanwhile, the user operation data of the player can be synchronously updated to the MR device with the permission of the observer in the field, and the commentator can better experience the mixed reality scene watched by the player through the MR device with the permission of the observer in the field.
The above description is only an embodiment of the present invention, and is not intended to limit the scope of the present invention, and all equivalent modifications made by the present invention and the contents of the accompanying drawings, which are directly or indirectly applied to the related technical fields, are included in the scope of the present invention.
Claims (4)
1. A method of displaying a mixed reality scene, comprising:
s1, configuring authorities for more than three MR devices; the permissions include player permissions, on-site observer permissions and off-site observer permissions;
s2, the MR equipment of the player with the player authority modifies the attribute value of a virtual object in the virtual scene corresponding to the MR equipment of the player;
s3, the onlooker MR equipment and the server with the permission of the observers in the field respectively update the virtual scenes corresponding to the onlooker MR equipment and the server according to the attribute values;
the S3 specifically comprises the following steps:
when the server receives an attribute value from one MR device, verifying the authority of the MR device to obtain a verification result; the verification result comprises player authority, on-site observer authority and off-site observer authority;
if the verification result is the player authority, then:
the server updates a virtual scene corresponding to the server according to the attribute value;
the server sends the attribute value to the onlooker MR equipment;
updating a virtual scene corresponding to the onlooker MR equipment by the onlooker MR equipment according to the attribute value;
s4, the entity camera acquires a real scene; a positioning MR device with off-site observer authority is physically fixed with the physical camera;
s5, the server superimposes the real scene and a virtual scene corresponding to the server according to the real-time position information of the positioning MR equipment to obtain a mixed reality scene;
the S5 specifically comprises the following steps:
acquiring the coordinates of the positioning MR equipment in a coordinate system in real time to obtain positioning coordinates;
acquiring a field angle of an entity camera;
converting a virtual scene corresponding to a server according to the field angle, the positioning coordinates and the current rotation offset of the positioning MR equipment to obtain a current virtual scene; rendering the current virtual scene to the real scene;
s6, displaying the mixed reality scene by display equipment;
further comprising:
acquiring a first space model corresponding to the player MR device;
acquiring a second space model corresponding to the bystander MR equipment;
acquiring a third space model corresponding to the positioning MR equipment;
presetting a virtual object with a constant position;
and mapping the first space model, the second space model and the third space model to the same coordinate system by taking the virtual object with the constant position as a reference object.
2. The method of displaying a mixed reality scene of claim 1, further comprising:
horizontally placing a physical camera;
modifying the initial horizontal rotational offset of the positioning MR device to zero;
calculating the vertical rotation offset of the positioning MR equipment relative to the entity camera according to the real scene and the virtual scene corresponding to the positioning MR equipment to obtain the relative vertical rotation offset;
the initial vertical rotational offset of the positioning MR device is modified depending on the relative vertical rotational offset.
3. A system for displaying a mixed reality scene is characterized by comprising more than three MR devices, an entity camera, a display device and a server;
the server includes one or more processors and memory, the memory storing a program and configured to perform the following steps by the one or more processors:
s1, configuring authorities for more than three MR devices; the permissions include player permissions, on-site observer permissions and off-site observer permissions;
s2, when a player MR device with the player permission modifies the attribute value of a virtual object in a virtual scene corresponding to the player MR device, acquiring the attribute value;
s3, updating a virtual scene corresponding to the server according to the attribute value; triggering the onlooker MR equipment with the in-field observer authority to update the virtual scene corresponding to the onlooker MR equipment according to the attribute value;
the S3 specifically comprises the following steps:
when the server receives an attribute value from one MR device, verifying the authority of the MR device to obtain a verification result; the verification result comprises player authority, on-site observer authority and off-site observer authority;
if the verification result is the player authority, then:
the server updates a virtual scene corresponding to the server according to the attribute value;
the server sends the attribute value to the onlooker MR equipment;
updating a virtual scene corresponding to the onlooker MR equipment by the onlooker MR equipment according to the attribute value;
s4, acquiring a real scene from the entity camera; the positioning MR device with the authority of the off-site observer is physically fixed with the physical camera;
s5, superposing the real scene and the virtual scene corresponding to the server according to the real-time position information of the positioning MR equipment to obtain a mixed reality scene;
the S5 specifically comprises the following steps:
acquiring the coordinates of the positioning MR equipment in a coordinate system in real time to obtain positioning coordinates;
acquiring a field angle of an entity camera;
converting a virtual scene corresponding to a server according to the field angle, the positioning coordinates and the current rotation offset of the positioning MR equipment to obtain a current virtual scene; rendering the current virtual scene to the real scene;
s6, triggering display equipment to display the mixed reality scene;
further comprising:
acquiring a first space model corresponding to the player MR equipment;
acquiring a second space model corresponding to the onlooker MR device;
acquiring a third spatial model corresponding to the positioning MR device;
presetting a virtual object with a constant position;
and mapping the first space model, the second space model and the third space model to the same coordinate system by taking the virtual object with the constant position as a reference object.
4. The system for displaying a mixed reality scene of claim 3, further comprising:
horizontally placing a physical camera;
modifying the initial horizontal rotational offset of the positioning MR device to zero;
calculating the vertical rotation offset of the positioning MR equipment relative to the entity camera according to the real scene and the virtual scene corresponding to the positioning MR equipment to obtain the relative vertical rotation offset;
the initial vertical rotational offset of the positioning MR device is modified depending on the relative vertical rotational offset.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710748651.2A CN109427095B (en) | 2017-08-28 | 2017-08-28 | Method and system for displaying mixed reality scene |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710748651.2A CN109427095B (en) | 2017-08-28 | 2017-08-28 | Method and system for displaying mixed reality scene |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109427095A CN109427095A (en) | 2019-03-05 |
| CN109427095B true CN109427095B (en) | 2022-10-21 |
Family
ID=65502431
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710748651.2A Active CN109427095B (en) | 2017-08-28 | 2017-08-28 | Method and system for displaying mixed reality scene |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109427095B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112148122A (en) * | 2020-08-25 | 2020-12-29 | 中国电子科技集团公司第三十八研究所 | Third-party visual angle implementation method for wearable augmented/mixed reality equipment |
| CN114489342A (en) * | 2022-01-29 | 2022-05-13 | 联想(北京)有限公司 | Image processing method and device and electronic equipment |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104715479A (en) * | 2015-03-06 | 2015-06-17 | 上海交通大学 | Scene reproduction detection method based on augmented virtuality |
| CN106898049A (en) * | 2017-01-18 | 2017-06-27 | 北京商询科技有限公司 | A kind of spatial match method and system for mixed reality equipment |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160267720A1 (en) * | 2004-01-30 | 2016-09-15 | Electronic Scripting Products, Inc. | Pleasant and Realistic Virtual/Augmented/Mixed Reality Experience |
-
2017
- 2017-08-28 CN CN201710748651.2A patent/CN109427095B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104715479A (en) * | 2015-03-06 | 2015-06-17 | 上海交通大学 | Scene reproduction detection method based on augmented virtuality |
| CN106898049A (en) * | 2017-01-18 | 2017-06-27 | 北京商询科技有限公司 | A kind of spatial match method and system for mixed reality equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109427095A (en) | 2019-03-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110382066B (en) | Mixed reality observer system and method | |
| CN107976811B (en) | Virtual reality mixing-based method simulation laboratory simulation method of simulation method | |
| US10692288B1 (en) | Compositing images for augmented reality | |
| CN105137705B (en) | A kind of creation method and device of virtual ball curtain | |
| US20050219695A1 (en) | Horizontal perspective display | |
| WO2018000609A1 (en) | Method for sharing 3d image in virtual reality system, and electronic device | |
| US20210038975A1 (en) | Calibration to be used in an augmented reality method and system | |
| JPWO2017094543A1 (en) | Information processing apparatus, information processing system, information processing apparatus control method, and parameter setting method | |
| CN106210856A (en) | Internet video live broadcasting platform is watched the method and system of 3D panoramic video | |
| CN109246404A (en) | The more optical projection systems and method of spectators' seat including direction-agile | |
| CN109427095B (en) | Method and system for displaying mixed reality scene | |
| CN104317546A (en) | Situational interactive experience simulation system | |
| CN106530408A (en) | Museum temporary exhibition planning and design system | |
| WO2017062730A1 (en) | Presentation of a virtual reality scene from a series of images | |
| Constantine et al. | Project esky: Enabling high fidelity augmented reality on an open source platform | |
| Marner et al. | Exploring interactivity and augmented reality in theater: A case study of Half Real | |
| EP3542877A1 (en) | Optimized content sharing interaction using a mixed reality environment | |
| CN103871094A (en) | Swept-volume-based three-dimensional display system data source generating method | |
| CN109427094B (en) | Method and system for acquiring mixed reality scene | |
| IJsselsteijn et al. | A room with a cue: The efficacy of movement parallax, occlusion, and blur in creating a virtual window | |
| CN115423916A (en) | XR (X-ray diffraction) technology-based immersive interactive live broadcast construction method, system and medium | |
| CN114708407A (en) | Virtual three-dimensional space information display method, device and program product | |
| CN110197524A (en) | Stereo display method, unit and computer readable storage medium | |
| KR102492985B1 (en) | A digital theme park implementation system that provides a digital twin-based metaverse | |
| Khedwala et al. | Analysis of Auto Generation of 3D Model Using Multiple 2D Graphics to Manifest Through Augmented Reality |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |