CN110211238A - Display methods, device, system, storage medium and the processor of mixed reality - Google Patents
Display methods, device, system, storage medium and the processor of mixed reality Download PDFInfo
- Publication number
- CN110211238A CN110211238A CN201910419915.9A CN201910419915A CN110211238A CN 110211238 A CN110211238 A CN 110211238A CN 201910419915 A CN201910419915 A CN 201910419915A CN 110211238 A CN110211238 A CN 110211238A
- Authority
- CN
- China
- Prior art keywords
- image
- reality
- mixed reality
- equipment
- posture
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/006—Mixed reality
Abstract
The invention discloses a kind of display methods of mixed reality, device, system, storage medium and processors.Wherein, this method comprises: receiving the virtual scene information that mixed reality equipment is passed back；Receive the reality scene image and the first posture information that image capture device is passed back, wherein the first posture information is used to indicate position and the posture of image capture device；According to the first posture information of image capture device, adjusts the position of Softcam and posture in virtual scene information and obtain virtual scene image；Mixed reality image after showing reality scene image and virtual scene image superposition.The present invention solves the technical issues of corresponding mixed reality image of mixed reality equipment can not be shown by third party device.
Description
Technical field
The present invention relates to mixed reality field, in particular to a kind of display methods of mixed reality, device, system,
Storage medium and processor.
Background technique
Mixed reality (Mixed Reality), abbreviation MR, by introducing reality scene information in virtual environment, in void
The quasi- information circuits that an exchange feedback is set up between the world, real world and user, to enhance the sense of reality of user experience.
User checks the picture of mixed reality by wearing head-mounted display apparatus (such as MR glasses), but does not wear head
It wears formula and shows that the third party user of equipment will be unable to see the picture in mixed reality.
It is above-mentioned third party user can not be made to check mixed reality image aiming at the problem that, at present not yet propose effective solution
Scheme.
Summary of the invention
The embodiment of the invention provides a kind of display methods of mixed reality, device, system, storage medium and processor,
At least to solve the technical issues of corresponding mixed reality image of mixed reality equipment can not be shown by third party device.
According to an aspect of an embodiment of the present invention, a kind of display methods of mixed reality is provided, comprising: receive mixing
The virtual scene information that real world devices are passed back；The reality scene image and the first posture information that image capture device is passed back are received,
Wherein, first posture information is used to indicate position and the posture of described image acquisition equipment；It is set according to described image acquisition
The first standby posture information adjusts the position of Softcam and posture in the virtual scene information and obtains virtual scene figure
Picture；Mixed reality image after showing the reality scene image and the virtual scene image superposition.
Further, the first posture information that equipment is acquired according to described image adjusts empty in the virtual scene information
It includes: to determine Softcam according to first posture information that the position of quasi- camera and posture, which obtain virtual scene image,
Corresponding position and posture in the virtual scene information, obtain the second posture information；Based on second posture information, determine
The virtual scene image of the Softcam acquisition.
Further, the mixed reality image packet after showing the reality scene image and the virtual scene image superposition
It includes: using the reality scene image as Background From Layer, by the virtual scene image superposition in the upper of the virtual scene image
Layer, obtains the mixed reality image；Show the mixed reality image.
Further, described image acquisition equipment includes at least one of: the first image capture device, for being based on institute
The first person for stating mixed reality equipment acquires the reality scene image, obtains the first reality scene image；Second figure
As acquisition equipment, the reality scene image is acquired for the third person based on the mixed reality equipment, obtains the
Two reality scene images.
Further, in the case where described image acquisition equipment is second image capture device；Image is received to adopt
First posture information of reality scene image and described image acquisition equipment that collection equipment is passed back includes: to receive second image
The second reality scene image that acquisition equipment is passed back and the first posture information based on the second reality scene image；Root
Position and posture according to mixed reality equipment described in the second reality scene image recognition, and it is based on second reality scene
First posture information of image obtains the third posture information of the mixed reality equipment；It, will according to the third posture information
The second reality scene Image Adjusting is the first reality scene image；Determine that the third posture information is based on described
First posture information of the first reality scene image.
Further, the position of the mixed reality equipment according to the second reality scene image recognition and posture, obtain
Third posture information to the mixed reality equipment includes: to identify the mixed reality according to the second reality scene figure
Pose corresponding relationship between equipment and second image capture device, wherein the pose corresponding relationship is for indicating institute
State the position corresponding relationship and posture corresponding relationship between mixed reality equipment and second image capture device；According to described
Pose corresponding relationship and the first posture information based on the second reality scene image, determine the third posture information.
According to another aspect of an embodiment of the present invention, a kind of display device of mixed reality is additionally provided, comprising: first connects
Receive unit, the virtual scene information passed back for receiving mixed reality equipment；Second receiving unit is set for receiving Image Acquisition
The standby reality scene image passed back and the first posture information, wherein first posture information is for indicating that described image acquires
The position of equipment and posture；Adjustment unit adjusts described virtual for acquiring the first posture information of equipment according to described image
The position of Softcam and posture obtain virtual scene image in scene information；Display unit, for showing the real field
Mixed reality image after scape image and the virtual scene image superposition.
According to another aspect of an embodiment of the present invention, a kind of display system of mixed reality is additionally provided, comprising: image is adopted
Collect equipment, for acquiring reality scene image；Computer, the virtual scene information passed back for receiving mixed reality equipment, with
And the reality scene image passed back of image capture device and the first posture information, the first pose of equipment is acquired according to described image
Information adjusts the position of Softcam and posture in the virtual scene information and obtains virtual scene image, and will be described existing
Real scene image and the virtual scene image superposition, obtained mixed reality image, wherein first posture information is used for
Indicate position and the posture of described image acquisition equipment；Display, for showing the mixed reality image.
Other side according to an embodiment of the present invention, additionally provides a kind of storage medium, and the storage medium includes depositing
The program of storage, wherein equipment where controlling the storage medium in described program operation executes mixed reality described above
Display methods.
Another aspect according to an embodiment of the present invention additionally provides a kind of processor, and the processor is for running journey
Sequence, wherein described program executes the display methods of mixed reality described above when running.
In embodiments of the present invention, computer can receive the virtual scene information that mixed reality equipment is passed back, Yi Jitu
As the reality scene image passed back of acquisition equipment and the first posture information, further according to the first location information of image capture device,
The position of Softcam and posture in virtual scene information are adjusted, it is corresponding in virtual scene information to obtain Softcam
Virtual scene image obtains mixed reality image then by reality scene image and virtual scene image superposition, and shows that this is mixed
Real world images are closed, have been achieved the purpose that through Computer display mixed reality image, to realize to mixed reality equipment
The technical effect that mixed reality image is shown by computer, and then solve the corresponding mixed reality of mixed reality equipment
The technical issues of image can not be shown by third party device.
Detailed description of the invention
The drawings described herein are used to provide a further understanding of the present invention, constitutes part of this application, this hair
Bright illustrative embodiments and their description are used to explain the present invention, and are not constituted improper limitations of the present invention.In the accompanying drawings:
Fig. 1 shows a kind of hardware of the terminal (or mobile device) of display methods for realizing mixed reality
Structural block diagram；
Fig. 2 is a kind of flow chart of the display methods of mixed reality according to an embodiment of the present invention；
Fig. 3 is the schematic diagram of the display methods of mixed reality according to an embodiment of the present invention；
Fig. 4 is the schematic diagram according to an embodiment of the present invention for generating mixed reality image；
Fig. 5 is the schematic diagram of the display device of mixed reality according to an embodiment of the present invention；
Fig. 6 is the schematic diagram of the display system of mixed reality according to an embodiment of the present invention.
Specific embodiment
In order to enable those skilled in the art to better understand the solution of the present invention, below in conjunction in the embodiment of the present invention
Attached drawing, technical scheme in the embodiment of the invention is clearly and completely described, it is clear that described embodiment is only
The embodiment of a part of the invention, instead of all the embodiments.Based on the embodiments of the present invention, ordinary skill people
The model that the present invention protects all should belong in member's every other embodiment obtained without making creative work
It encloses.
It should be noted that description and claims of this specification and term " first " in above-mentioned attached drawing, "
Two " etc. be to be used to distinguish similar objects, without being used to describe a particular order or precedence order.It should be understood that using in this way
Data be interchangeable under appropriate circumstances, so as to the embodiment of the present invention described herein can in addition to illustrating herein or
Sequence other than those of description is implemented.In addition, term " includes " and " having " and their any deformation, it is intended that cover
Cover it is non-exclusive include, for example, the process, method, system, product or equipment for containing a series of steps or units are not necessarily limited to
Step or unit those of is clearly listed, but may include be not clearly listed or for these process, methods, product
Or other step or units that equipment is intrinsic.
According to embodiments of the present invention, a kind of display methods embodiment of mixed reality is additionally provided, it should be noted that
The step of process of attached drawing illustrates can execute in a computer system such as a set of computer executable instructions, also,
It, in some cases, can be to be different from shown in sequence execution herein although logical order is shown in flow charts
The step of out or describing.
The display methods embodiment of mixed reality provided herein can be in mobile terminal, terminal or class
As execute in arithmetic unit.The terminal that Fig. 1 shows a kind of display methods for realizing mixed reality is (or mobile
Equipment) hardware block diagram.As shown in Figure 1, terminal 10 (or mobile device 10) may include one or more (figures
Middle to use 102a, 102b ... ..., 102n is shown) (processor 102 can include but is not limited to microprocessor to processor 102
The processing unit of MCU or programmable logic device FPGA etc.), memory 104 for storing data and be used for communication function
Transmission module 106, wherein transmission module 106 may include: input/output interface, network interface.It in addition to this, can be with
It include: display, computer universal serial bus (BUS), power supply and/or camera.It will appreciated by the skilled person that
Structure shown in FIG. 1 is only to illustrate, and does not cause to limit to the structure of above-mentioned electronic device.For example, terminal 10 is also
It may include more perhaps less component or with the configuration different from shown in Fig. 1 than shown in Fig. 1.
It is to be noted that said one or multiple processors 102 and/or other data processing circuits lead to herein
Can often " data processing circuit " be referred to as.The data processing circuit all or part of can be presented as software, hardware, firmware
Or any other combination.In addition, data processing circuit for single independent processing module or all or part of can be integrated to meter
In any one in other elements in calculation machine terminal 10 (or mobile device).As involved in the embodiment of the present application,
The data processing circuit controls (such as the selection for the variable resistance end path connecting with interface) as a kind of processor.
Memory 104 can be used for storing the software program and module of application software, such as the mixing in the embodiment of the present invention
Corresponding program instruction/the data storage device of display methods of reality, processor 102 are stored in memory 104 by operation
Software program and module realize the aobvious of above-mentioned mixed reality thereby executing various function application and data processing
Show method.Memory 104 may include high speed random access memory, may also include nonvolatile memory, such as one or more magnetic
Property storage device, flash memory or other non-volatile solid state memories.In some instances, memory 104 can further comprise
The memory remotely located relative to processor 102, these remote memories can pass through network connection to terminal 10.
The example of above-mentioned network includes but is not limited to internet, intranet, local area network, mobile radio communication and combinations thereof.
Transmission module 106 is used to that data to be received or sent via a network.Above-mentioned network specific example may include
The wireless network that the communication providers of terminal 10 provide.In an example, transmission module 106 includes that a network is suitable
Orchestration (Network Interface Controller, NIC), can be connected by base station with other network equipments so as to
Internet is communicated.In an example, transmission module 106 can be radio frequency (Radio Frequency, RF) module,
For wirelessly being communicated with internet.
Display can such as touch-screen type liquid crystal display (LCD), the liquid crystal display aloow user with
The user interface of terminal 10 (or mobile device) interacts.
Herein it should be noted that in some optional embodiments, above-mentioned computer equipment shown in FIG. 1 (or movement is set
It is standby) it may include hardware element (including circuit), software element (including the computer generation that may be stored on the computer-readable medium
Code) or both hardware element and software element combination.It should be pointed out that Fig. 1 is only a reality of particular embodiment
Example, and it is intended to show that the type for the component that may be present in above-mentioned computer equipment (or mobile device).
According to embodiments of the present invention, a kind of display methods embodiment of mixed reality is provided, it should be noted that attached
The step of process of figure illustrates can execute in a computer system such as a set of computer executable instructions, though also,
So logical order is shown in flow charts, but in some cases, it can be to be different from shown by sequence execution herein
Or the step of description.
Fig. 2 is a kind of flow chart of the display methods of mixed reality according to an embodiment of the present invention, as shown in Fig. 2, the party
Method includes the following steps:
Step S202 receives the virtual scene information that mixed reality equipment is passed back；
Step S204 receives reality scene image and the first posture information that image capture device is passed back, wherein first
Appearance information is used to indicate position and the posture of image capture device；
Step S206 adjusts Softcam in virtual scene information according to the first posture information of image capture device
Position and posture obtain virtual scene image；
Step S208, display reality scene image and the mixed reality image after virtual scene image superposition.
Through the above steps, computer can receive virtual scene information that mixed reality equipment is passed back and image is adopted
The reality scene image and the first posture information that collection equipment is passed back, further according to the first location information of image capture device, adjustment
It is corresponding virtual in virtual scene information to obtain Softcam for the position of Softcam and posture in virtual scene information
Scene image obtains mixed reality image then by reality scene image and virtual scene image superposition, and shows that the mixing is existing
Real image has achieved the purpose that through Computer display mixed reality image, to realize the mixing to mixed reality equipment
The technical effect that real world images are shown by computer, and then solve the corresponding mixed reality image of mixed reality equipment
The technical issues of can not being shown by third party device.
In the scheme that step S202 is provided, virtual scene information can indicate being seen using object for mixed reality equipment
The threedimensional model of the virtual scene measured.
For example, mixed reality equipment can observe a virtual " side using object by mixed display equipment
Block ", then virtual scene information can be the threedimensional model of virtual " square ", and then computer is obtaining virtual " square "
After threedimensional model, the observation angle and observation position of the adjustable threedimensional model, to be somebody's turn to do the virtual field of " square "
Scape image.
Optionally, computer can receive the threedimensional model of the virtual scene of mixed display equipment real-time Transmission, namely mixed
Close the complete threedimensional model that the virtual scene information that real world devices are passed back is virtual scene.
Optionally, the threedimensional model of multiple virtual scenes in mixed reality equipment, computer can be stored in advance in computer
The virtual scene information passed back according to mixed reality equipment can determine being observed using object for current mixed reality equipment
The threedimensional model of virtual scene.
For example, the threedimensional model of virtual scene A, the threedimensional model of virtual scene B and virtual can be stored in advance in computer
The threedimensional model of scene C uses object in the case where observing virtual scene A in mixed reality equipment, can be by virtual
Scene information instruction is observed virtual scene A, and then computer can transfer virtual scene A according to the virtual scene information
Threedimensional model.
It should be noted that computer equipment and mixed reality equipment and image capture device are established and communicated to connect, make to count
Calculate the available image capture device of machine equipment pass back reality scene information, the first posture information of image capture device, with
And mixed reality equipment is the virtual scene information that user provides.
In the scheme that step S204 is provided, image capture device can be camera, such as high definition network head.
Optionally, USB2.0 cable, USB3.0 cable or HDMI wire can be passed through between image capture device and computer
The wired communication modes such as cable establish communication connection, can also be established and be communicated to connect by communications such as bluetooth, WiFi.
Optionally, it can be established by communications such as bluetooth or WiFi between mixed reality equipment and computer logical
Letter connection；It can also be established and be communicated to connect by wired communication modes such as client cables.
The above embodiment of the present invention only needs transfer of virtual scene information, nothing between mixed reality equipment and computer
Mixed reality equipment simultaneous transmission virtual scene information and reality scene image are needed, mixed reality equipment is reduced and is set with computer
The information content of transmitted information between standby, and then computer can receive the virtual scene information of mixed reality equipment offer, and
The reality scene image that image capture device provides improves the efficiency that computer obtains mixed reality image, makes computer can
To obtain more virtual scene information and reality scene image within the unit time, and mixed reality image is based on void
What quasi- scene information and reality scene image determined, since computer can obtain more virtual scene letters within the unit time
Breath and reality scene image keep mixing existing so as to improve the fluency and clarity of Computer display mixed reality image
The picture of real image is more smooth, lowers camera lens spinning sensation.
Optionally, computer can determine virtual scene image according to virtual scene information, and then by virtual scene image
With reality scene image superposition, mixed reality image is generated, and mixed reality image transmitting to the display of generation is shown
Show.
It should be noted that the mixed reality image generated can be the high-definition image of 2K resolution ratio, mixed reality is shown
The display of image can support the display of 2K resolution ratio, and then computer equipment sends the mixed reality image of 2K resolution ratio
To display, the mixed reality image of 2K resolution ratio is shown by display.
Optionally, by image capture device, real field of the mixed reality equipment under first person can be acquired
Scape image, i.e. mixed reality equipment arrive real picture using what object was seen；Mixed reality equipment can also be acquired in the third party
Claim the reality scene image under visual angle, i.e. what the user of the third person was seen when seeing to mixed reality equipment using object arrives reality
Picture.
As a kind of optional embodiment, image capture device includes at least one of: the first image capture device, is used
Reality scene image is acquired in the first person based on mixed reality equipment, obtains the first reality scene image；Second figure
As acquisition equipment, reality scene image is acquired for the third person based on mixed reality equipment, obtains the second real field
Scape image.
The above embodiment of the present invention can be generated and be based on based on the first reality scene image of first person acquisition
The mixed reality image of first person generates the mixed reality figure of mixed reality equipment observed using object
Picture；Based on the second reality scene image of third person acquisition, the mixed reality based on third person can be generated
Image, the i.e. user of the generation third person observe when using object of mixed reality equipment, the mixed reality figure observed
Picture generates that is, using object including mixed reality equipment in the second reality scene image according to the second reality scene image
Mixed reality image in also including mixed reality equipment use object.
As a kind of optional example, needing to obtain the first person based on mixed reality equipment collected the
It, can be by the first image capture device and mixed reality apparatus bound, such as by the first figure in the case where one reality scene image
It is arranged in mixed reality equipment as acquiring equipment, then the first image capture device the first reality scene image collected, i.e.,
The reality scene observed by the wearing object of mixed display equipment, also, the posture information of the first image capture device is (i.e.
The position of first image capture device and posture), the as position of mixed display equipment and posture.
It is collected needing to obtain the first person based on mixed reality equipment as another optional example
In the case where first reality scene image, the third party based on mixed reality equipment can be obtained by the second image capture device
Claim the acquired reality scene image in visual angle, then further according to the pose pair between the second acquisition vision facilities and mixed reality equipment
It should be related to, reality scene image acquired to third person is handled, and based on mixed reality equipment can be obtained
One people claims visual angle reality scene image collected.
It should be noted that can be digitized to reality scene image in mixed reality technology (i.e. MR technology)
Digitized reality scene image, is then combined by processing with virtual scene image again.
As a kind of optional example, the second image capture device can be multiple, be set according to multiple second Image Acquisition
The reality scene image of standby acquisition, several available reality scene images, and then can be obtained according to several reality scene images
To the digital three-dimemsional model of reality scene, thus according to the digital three-dimemsional model of reality scene and the second acquisition image
Pose corresponding relationship between equipment and mixed reality equipment can determine the first person institute based on mixed reality equipment
Acquire reality scene image.
For example, the second image capture device includes camera A and camera B, camera A acquires reality scene image A, takes the photograph
As head B acquisition reality scene image B, and then it can be based on camera A and camera B component Binocular Stereo Vision System, according to
Camera A and calibration relationship and reality scene image A and reality scene figure of the camera B under Binocular Stereo Vision System
As B, the digital three-dimemsional model of reality scene is generated, and then according to the digital three-dimemsional model of reality scene, and from reality
Posture information (i.e. the mixed reality equipment and camera A of the mixed display equipment of scene image A and reality scene image B identification
Between pose corresponding relationship and mixed reality equipment and camera B between pose corresponding relationship), can determine based on mixed
Close first person the first reality scene image collected of real world devices.
It should be noted that since image capture device includes: the first image capture device and the second image capture device；
Therefore reality scene image includes: that the first reality scene image of the first image capture device acquisition and the second Image Acquisition are set
Second reality scene image of standby acquisition, and then the first posture information of image capture device includes: the first image capture device
The first posture information and the second image capture device the first posture information, and the first image capture device the first pose letter
Breath is the first posture information based on the first reality scene image, and the first posture information of the second image capture device is base
In the first posture information of the second reality scene image.
Figure is received in the case where image capture device is the first image capture device as a kind of optional embodiment
The reality scene image and the first posture information passed back as acquisition equipment include: receive the first image capture device and pass back first
Scene image, and the first posture information based on the first scene image, wherein the first posture information based on the first scene image
For indicating position and the posture of the first image capture device；According to the first posture information of image capture device, adjustment is virtual
It includes: according to the first image capture device that the position of Softcam and posture, which obtain virtual scene image, in scene information
One posture information adjusts the position of Softcam and posture in virtual scene information and obtains the first virtual scene image, wherein
Virtual scene image includes the first virtual scene image；Mixing after showing reality scene image and virtual scene image superposition is existing
Real image includes: the first mixed reality image shown after the first reality scene image and the first virtual scene image superposition,
In, mixed reality image includes the first mixed reality image, and the first mixed reality image is based on mixed reality equipment for indicating
The picture (the mixed reality image i.e. under first person) that is observed of first person.
It is received as an alternative embodiment in the case where image capture device is the second image capture device
The reality scene image and the first posture information that image capture device is passed back include: for receiving the second image capture device and passing back
Two scene images, and the first posture information based on the second scene image, wherein the first pose letter based on the second scene image
Breath is for indicating position and the posture of the second image capture device；According to the first posture information of image capture device, adjustment is empty
It includes: according to the second image capture device that the position of Softcam and posture, which obtain virtual scene image, in quasi- scene information
First posture information adjusts the position of Softcam and posture in virtual scene information and obtains the second virtual scene image,
In, virtual scene image includes the second virtual scene image；Show reality scene image and mixing after virtual scene image superposition
Closing real world images includes: the second mixed reality figure shown after the second reality scene image and the second virtual scene image superposition
Picture, wherein mixed reality image includes the second mixed reality image, and the second mixed reality image is based on mixed reality for indicating
The picture (the mixed reality image i.e. under third person) that the third person of equipment is observed.
It optionally, can be according to second image in the case where image capture device is the second image capture device
The second reality scene image that acquisition equipment is passed back and the first posture information based on the second reality scene image, display first are existing
The first mixed reality image after real scene image and the first virtual scene image superposition, namely according under third person
Reality scene image (i.e. the second reality scene image) shows mixed reality image (the i.e. first mixing under first person
Real world images) the case where.
As a kind of optional embodiment, in the case where image capture device is the second image capture device；Receive figure
The first posture information as acquiring reality scene image and image capture device that equipment is passed back includes: to receive the second Image Acquisition
The second reality scene image that equipment is passed back and the first posture information based on the second reality scene image；According to the second real field
The position of scape image recognition mixed reality equipment and posture, and the first posture information based on the second reality scene image, obtain
The third posture information of mixed reality equipment；It is the first reality by the second reality scene Image Adjusting according to third posture information
Scene image；Determine that third posture information is the first posture information based on the first reality scene image.
The above embodiment of the present invention can be by second in the case where image capture device is the second image capture device
(i.e. the first pose of the second image capture device is believed for reality scene image and the first posture information based on the second scene image
Breath), be converted to the first reality scene image and the first posture information (i.e. the first image capture device based on the first scene image
The first posture information), so as to show that the mixed reality image based on the first scene image (shows first person
Under mixed reality image).
As a kind of optional embodiment, according to the position of the second reality scene image recognition mixed reality equipment and appearance
State, the third posture information for obtaining mixed reality equipment include: to identify mixed reality equipment and the according to the second reality scene figure
Pose corresponding relationship between two image capture devices, wherein pose corresponding relationship is for indicating mixed reality equipment and second
Position corresponding relationship and posture corresponding relationship between image capture device；According to pose corresponding relationship and based on the second real field
First posture information of scape image, determines third posture information.
It is received as another optional embodiment in the case where image capture device is the second image capture device
The reality scene image and the first posture information that image capture device is passed back include: for receiving the second image capture device and passing back
Two scene images, and the first posture information based on the second scene image, wherein the first pose letter based on the second scene image
Breath is for indicating position and the posture of the second image capture device；And by the second scene image and based on the second scene image
One posture information is converted to the first scene image and the first posture information based on the first scene image, wherein is based on first
First posture information of scape image is used to indicate position and the posture of the first image capture device；According to the of image capture device
One posture information, adjusting the position of Softcam and posture in virtual scene information, to obtain virtual scene image include: basis
First posture information of the first image capture device adjusts the position of Softcam and posture in virtual scene information and obtains the
One virtual scene image, wherein virtual scene image includes the first virtual scene image；Show reality scene image and virtual field
Mixed reality image after scape image superposition includes: after showing the first reality scene image and the first virtual scene image superposition
First mixed reality image, wherein mixed reality image includes the first mixed reality image, and the first mixed reality image is used for table
Show picture (the mixed reality figure i.e. under first person that the first person based on mixed reality equipment is observed
Picture).
It optionally, can be according to the first image in the case where image capture device is the first image capture device
The first reality scene image that acquisition equipment is passed back and the first posture information based on the first reality scene image, display second are existing
The second mixed reality image after real scene image and the second virtual scene image superposition, namely according under first person
Reality scene image (i.e. the first reality scene image) shows mixed reality image (the i.e. second mixing under third person
Real world images) the case where.
Wherein, the first reality scene image is converted into the second reality scene image, by the of the first reality scene image
One posture information is converted to by the specific implementation of the first posture information of the second reality scene image, in above-described embodiment
" the second reality scene image is converted into the first reality scene image, the first posture information of the second reality scene image is turned
It is changed to the specific implementation of the first posture information of the first reality scene image " it is identical, details are not described herein again.
It is received as another optional embodiment in the case where image capture device is the first image capture device
The reality scene image and the first posture information that image capture device is passed back include: for receiving the first image capture device and passing back
One scene image, and the first posture information based on the first scene image, wherein the first pose letter based on the first scene image
Breath is for indicating position and the posture of the first image capture device；And by the first scene image and based on the first scene image
One posture information is converted to the second scene image and the first posture information based on the second scene image, wherein is based on second
First posture information of scape image is used to indicate position and the posture of the second image capture device；According to the of image capture device
One posture information, adjusting the position of Softcam and posture in virtual scene information, to obtain virtual scene image include: basis
First posture information of the second image capture device adjusts the position of Softcam and posture in virtual scene information and obtains the
Two virtual scene images, wherein virtual scene image includes the second virtual scene image；Show reality scene image and virtual field
Mixed reality image after scape image superposition includes: after showing the second reality scene image and the second virtual scene image superposition
Second mixed reality image, wherein mixed reality image includes the second mixed reality image, and the second mixed reality image is used for table
Show picture (the mixed reality figure i.e. under third person that the third person based on mixed reality equipment is observed
Picture).
In the scheme that step S206 is provided, Softcam is position and the angle for observing virtual scene information, for example,
" square " virtual in observation using object of mixed reality equipment, then this uses the eyes of object to be based on where virtual " square "
Position and posture under the coordinate system of Virtual Space.
It should be noted that mixed reality image is obtained by reality scene image and virtual scene image superposition, it is described folded
Adding reality scene image and virtual scene image should be the image based on same observation position and posture, therefore receive image
It, can be according to the first pose after the reality scene image that acquisition equipment is passed back and the first posture information based on reality scene image
Information adjusts position and posture of the Softcam in virtual scene information, obtains for indicating Softcam in virtual field
Second posture information of position and posture in scape information, and then Softcam is based on the second posture information and believes in virtual scene
Acquired image in breath, virtual scene image as identical as the observation position of reality scene image and posture, thus according to phase
With observation position and the reality scene image of posture and virtual scene image, that is, produce mixed reality image.
Virtual scene information is adjusted according to the first posture information of image capture device as a kind of optional embodiment
It includes: to determine Softcam according to the first posture information that the position of middle Softcam and posture, which obtain virtual scene image,
Corresponding position and posture in virtual scene information, obtain the second posture information；Based on the second posture information, virtual camera shooting is determined
The virtual scene image of head acquisition.
In the scheme that step S208 is provided, mixed reality image can be by reality scene image and virtual scene figure
As being overlapped and obtaining, computer can be shown by display through obtained mixed reality image after superposition.
A kind of mixed reality figure as optional embodiment, after showing reality scene image and virtual scene image superposition
As including:, by virtual scene image superposition on the upper layer of virtual scene image, to obtain institute using reality scene image as Background From Layer
State mixed reality image；Show mixed reality image.
As a kind of optional embodiment, by virtual scene image superposition before the upper layer of virtual scene image, side
Method further include: calculate the transparent channel of virtual scene image.
Optionally, mixed reality equipment is multiple, and the second reality scene image of the second image capture device acquisition can be with
Object is used including multiple mixed reality equipment, and then can be with according to the mixed reality image that the second reality scene image generates
Show the movement using object of multiple mixed reality equipment.
Optionally, mixed reality equipment is multiple, if multiple mixed reality equipment are sent out using any object in object
Out to the operation of virtual scene information, then the operating result of the object can be synchronized in mixed reality image, i.e., it is right based on this
As the operating result to virtual scene information, virtual scene image is acquired, regenerates mixed display image.For example, multiple users
Same virtual " square " is observed jointly by mixed reality equipment, if one of user depicts figure in virtual block
Case A can also show the pattern A that the user draws in the mixed display image then shown on " square ".
Optionally, mixed reality equipment is multiple, if multiple mixed reality equipment are sent out using any object in object
It, then can be by showing the object to virtual scene information institute in multiple mixed reality equipment out to the operation of virtual scene information
The operation of execution.For example, multiple users observe same virtual " square " by mixed reality equipment jointly, if one of those
User depicts pattern A in virtual block, then can also show on " square " that other users are observed by mixed reality equipment
Show the pattern A that the user draws.
Optionally, mixed reality equipment is multiple, and is connected with same computer, if multiple mixed reality equipment make
The operation to virtual scene information is issued with any object in object, then mixing used in the available object of computer
The operation information of real world devices transmission, and send the operation information to other mixed reality equipment being connected with the computer
In, make to realize that operation is synchronous with multiple mixed reality equipment that same computer is connected.
Fig. 3 is the schematic diagram of the display methods of mixed reality according to an embodiment of the present invention, as shown in figure 3, user wears
Mixed reality equipment observes virtual " square ", then can acquire the user's by camera (i.e. the second image capture device)
Reality scene image (as shown in the image 41 in Fig. 4), computer can receive reality scene image (such as Fig. 4 that camera is passed back
In image 41 shown in) and the virtual scene image (as shown in the image 43 in Fig. 4) passed back of mixed reality equipment, it is then right
Reality scene image (as shown in the image 41 in Fig. 4) and virtual scene image (as shown in the image 43 in Fig. 4) are overlapped,
Obtain mixed reality image (as shown in the image 45 in Fig. 4).
Fig. 4 is the schematic diagram according to an embodiment of the present invention for generating mixed reality image, as shown in figure 4, by reality scene
Image (as shown in the image 41 in Fig. 4) is used as Background From Layer, and virtual scene image (as shown in the image 43 in Fig. 4) is superimposed
On the upper layer of Background From Layer, obtain mixed reality image (as shown in the image 45 in Fig. 4).
According to still another embodiment of the invention, a kind of storage medium is additionally provided, storage medium includes the program of storage,
Wherein, the method for executing any of the above-described when program is run.
According to still another embodiment of the invention, a kind of processor is additionally provided, processor is for running program, wherein
The method of any of the above-described is executed when program is run.
According to embodiments of the present invention, a kind of display device embodiment of mixed reality is additionally provided, it should be noted that should
The display device of mixed reality can be used for executing the display methods of the mixed reality in the embodiment of the present invention, the embodiment of the present invention
In the display methods of mixed reality can be executed in the display device of the mixed reality.
Fig. 5 is the schematic diagram of the display device of mixed reality according to an embodiment of the present invention, as shown in figure 5, the device can
To include: the first receiving unit 51, the virtual scene information passed back for receiving mixed reality equipment；Second receiving unit 53,
For receiving the reality scene image and the first posture information that image capture device is passed back, wherein the first posture information is used for table
Show position and the posture of image capture device；Adjustment unit 55, for the first posture information according to image capture device, adjustment
The position of Softcam and posture obtain virtual scene image in virtual scene information；Display unit 57, for showing reality
Mixed reality image after scene image and virtual scene image superposition.
It should be noted that the first receiving unit 512 in the embodiment can be used for executing in the embodiment of the present application
Step S202, the second receiving unit 53 in the embodiment can be used for executing the step S204 in the embodiment of the present application, the reality
Applying the adjustment unit 55 in example can be used for executing step S206 in the embodiment of the present application, the display unit 57 in the embodiment
It can be used for executing the step S208 in the embodiment of the present application.The example and applied field that said units are realized with corresponding step
Scape is identical, but is not limited to the above embodiments disclosure of that.
Embodiment in the present invention, computer can receive the virtual scene information and image that mixed reality equipment is passed back
The reality scene image and the first posture information that acquisition equipment is passed back are adjusted further according to the first location information of image capture device
The position of Softcam and posture in whole virtual scene information obtain Softcam corresponding void in virtual scene information
Quasi- scene image obtains mixed reality image then by reality scene image and virtual scene image superposition, and shows the mixing
Real world images have achieved the purpose that mix mixed reality equipment to realize by Computer display mixed reality image
The technical effect that real world images are shown by computer is closed, and then solves the corresponding mixed reality figure of mixed reality equipment
As the technical issues of can not be shown by third party device.
As a kind of optional embodiment, adjustment unit includes: the first determining module, is used for according to the first posture information,
It determines Softcam corresponding position and posture in virtual scene information, obtains the second posture information；Second determining module is used
In being based on the second posture information, the virtual scene image of Softcam acquisition is determined.
As a kind of optional embodiment, display unit includes: laminating module, for using reality scene image as Background
Layer, by virtual scene image superposition on the upper layer of virtual scene image, obtains mixed reality image；Display module, for showing
Mixed reality image.
As a kind of optional embodiment, device further include: computing unit, for by virtual scene image superposition in void
Before the upper layer of quasi- scene image, the transparent channel of virtual scene image is calculated.
As a kind of optional embodiment, image capture device includes at least one of: the first image capture device, is used
Reality scene image is acquired in the first person based on mixed reality equipment, obtains the first reality scene image；Second figure
As acquisition equipment, reality scene image is acquired for the third person based on mixed reality equipment, obtains the second real field
Scape image.
As a kind of optional embodiment, in the case where image capture device is the second image capture device；Second connects
Receiving unit includes: receiving module, for receiving the second reality scene image that the second image capture device is passed back and being based on second
First posture information of reality scene image；Identification module, for according to the second reality scene image recognition mixed reality equipment
Position and posture, and the first posture information based on the second reality scene image, obtain the third pose of mixed reality equipment
Information；Module is adjusted, for being the first reality scene image by the second reality scene Image Adjusting according to third posture information；
Determining module, for determining that third posture information is the first posture information based on the first reality scene image.
As a kind of optional embodiment, identification module includes: identification submodule, is used for according to the second reality scene figure,
Identify the pose corresponding relationship between mixed reality equipment and the second image capture device, wherein pose corresponding relationship is used for table
Show the position corresponding relationship and posture corresponding relationship between mixed reality equipment and the second image capture device；Determine submodule,
For determining third posture information according to pose corresponding relationship and the first posture information based on the second reality scene image.
Fig. 6 is the schematic diagram of the display system of mixed reality according to an embodiment of the present invention, as shown in fig. 6, the system can
To include: image capture device 61, for acquiring reality scene image；Computer 63 is passed for receiving mixed reality equipment 67
The reality scene image and the first posture information that the virtual scene information and image capture device returned are passed back, are adopted according to image
Collect the first posture information of equipment, adjusts the position of Softcam and posture in virtual scene information and obtain virtual scene figure
Picture, and by reality scene image and virtual scene image superposition, obtained mixed reality image, wherein the first posture information is used
In the position and posture that indicate image capture device；Display 65, for showing mixed reality image.
Embodiment in the present invention, computer can receive the virtual scene information and image that mixed reality equipment is passed back
The reality scene image and the first posture information that acquisition equipment is passed back are adjusted further according to the first location information of image capture device
The position of Softcam and posture in whole virtual scene information obtain Softcam corresponding void in virtual scene information
Quasi- scene image obtains mixed reality image then by reality scene image and virtual scene image superposition, and shows the mixing
Real world images have achieved the purpose that mix mixed reality equipment to realize by Computer display mixed reality image
The technical effect that real world images are shown by computer is closed, and then solves the corresponding mixed reality figure of mixed reality equipment
As the technical issues of can not be shown by third party device.
Optionally, USB2.0 cable, USB3.0 cable or HDMI wire can be passed through between image capture device and computer
The wired communication modes such as cable establish communication connection, can also be established and be communicated to connect by communications such as bluetooth, WiFi.
Optionally, it can be established by communications such as bluetooth or WiFi between mixed reality equipment and computer logical
Letter connection；It can also be established and be communicated to connect by wired communication modes such as client cables.
Optionally, image capture device includes at least one of: the first image capture device, for being based on mixed reality
The first person of equipment acquires reality scene image, obtains the first reality scene image；Second image capture device, is used for
Third person based on mixed reality equipment acquires reality scene image, obtains the second reality scene image.
Optionally, in the case where image capture device is the first image capture device, the first image capture device and mixed
Close real world devices binding.
Optionally, mixed reality equipment is multiple, and multiple mixed reality equipment are connected with same computer, multiple mixed
The operation for closing real world devices is synchronous, i.e., can after any one in multiple mixed reality equipment receives the operation information of user
It is sent to computer to synchronize the operation information, and is connected to other on same computer with the mixed reality equipment
Mixed reality equipment realizes that operation synchronizes.
Optionally, display can support the display of 2K resolution ratio.The mixed reality image of generation can be 2K resolution ratio
High-definition image, and then the mixed reality image of 2K resolution ratio is sent to display by computer equipment, shows 2K by display
The mixed reality image of resolution ratio.
The serial number of the above embodiments of the invention is only for description, does not represent the advantages or disadvantages of the embodiments.
In the above embodiment of the invention, it all emphasizes particularly on different fields to the description of each embodiment, does not have in some embodiment
The part of detailed description, reference can be made to the related descriptions of other embodiments.
In several embodiments provided herein, it should be understood that disclosed technology contents can pass through others
Mode is realized.Wherein, the apparatus embodiments described above are merely exemplary, such as the division of the unit, Ke Yiwei
A kind of logical function partition, there may be another division manner in actual implementation, for example, multiple units or components can combine or
Person is desirably integrated into another system, or some features can be ignored or not executed.Another point, shown or discussed is mutual
Between coupling, direct-coupling or communication connection can be through some interfaces, the INDIRECT COUPLING or communication link of unit or module
It connects, can be electrical or other forms.
The unit as illustrated by the separation member may or may not be physically separated, aobvious as unit
The component shown may or may not be physical unit, it can and it is in one place, or may be distributed over multiple
On unit.It can some or all of the units may be selected to achieve the purpose of the solution of this embodiment according to the actual needs.
It, can also be in addition, the functional units in various embodiments of the present invention may be integrated into one processing unit
It is that each unit physically exists alone, can also be integrated in one unit with two or more units.Above-mentioned integrated list
Member both can take the form of hardware realization, can also realize in the form of software functional units.
If the integrated unit is realized in the form of SFU software functional unit and sells or use as independent product
When, it can store in a computer readable storage medium.Based on this understanding, technical solution of the present invention is substantially
The all or part of the part that contributes to existing technology or the technical solution can be in the form of software products in other words
It embodies, which is stored in a storage medium, including some instructions are used so that a computer
Equipment (can for personal computer, server or network equipment etc.) execute each embodiment the method for the present invention whole or
Part steps.And storage medium above-mentioned includes: that USB flash disk, read-only memory (ROM, Read-Only Memory), arbitrary access are deposited
Reservoir (RAM, Random Access Memory), mobile hard disk, magnetic or disk etc. be various to can store program code
Medium.
The above is only a preferred embodiment of the present invention, it is noted that for the ordinary skill people of the art
For member, various improvements and modifications may be made without departing from the principle of the present invention, these improvements and modifications are also answered
It is considered as protection scope of the present invention.
Claims (10)
1. a kind of display methods of mixed reality characterized by comprising
Receive the virtual scene information that mixed reality equipment is passed back；
Receive the reality scene image and the first posture information that image capture device is passed back, wherein first posture information is used
In the position and posture that indicate described image acquisition equipment；
The first posture information that equipment is acquired according to described image, adjusts the position of Softcam in the virtual scene information
Virtual scene image is obtained with posture；
Mixed reality image after showing the reality scene image and the virtual scene image superposition.
2. the method according to claim 1, wherein according to described image acquire equipment the first posture information,
It adjusts the position of Softcam and posture in the virtual scene information and obtains virtual scene image and include:
According to first posture information, determines Softcam corresponding position and posture in the virtual scene information, obtain
To the second posture information；
Based on second posture information, the virtual scene image of the Softcam acquisition is determined.
3. the method according to claim 1, wherein showing the reality scene image and the virtual scene figure
As superimposed mixed reality image includes:
Using the reality scene image as Background From Layer, by the virtual scene image superposition in the upper of the virtual scene image
Layer, obtains the mixed reality image；
Show the mixed reality image.
4. method described in any one of -3 according to claim 1, which is characterized in that it includes following that described image, which acquires equipment,
At least one:
First image capture device acquires the reality scene figure for the first person based on the mixed reality equipment
Picture obtains the first reality scene image；
Second image capture device acquires the reality scene figure for the third person based on the mixed reality equipment
Picture obtains the second reality scene image.
5. the method according to any one of claim 4, which is characterized in that in described image acquisition equipment be described the
In the case where two image capture devices；The reality scene image and described image that reception image capture device is passed back acquire equipment
First posture information includes:
Receive the second reality scene image and be based on second reality scene that second image capture device is passed back
First posture information of image；
It is existing according to the position of mixed reality equipment described in the second reality scene image recognition and posture, and based on described second
First posture information of real scene image obtains the third posture information of the mixed reality equipment；
It is the first reality scene image by the second reality scene Image Adjusting according to the third posture information；
Determine that the third posture information is the first posture information based on the first reality scene image.
6. the method according to any one of claim 5, which is characterized in that known according to the second reality scene image
The position of the not described mixed reality equipment and posture, the third posture information for obtaining the mixed reality equipment include:
According to the second reality scene figure, the position between the mixed reality equipment and second image capture device is identified
Appearance corresponding relationship, wherein the pose corresponding relationship is for indicating that the mixed reality equipment is set with second Image Acquisition
Position corresponding relationship and posture corresponding relationship between standby；
According to the pose corresponding relationship and the first posture information based on the second reality scene image, the third is determined
Posture information.
7. a kind of display device of mixed reality characterized by comprising
First receiving unit, the virtual scene information passed back for receiving mixed reality equipment；
Second receiving unit, for receiving the reality scene image and the first posture information that image capture device is passed back, wherein institute
The first posture information is stated for indicating position and the posture of described image acquisition equipment；
Adjustment unit adjusts empty in the virtual scene information for acquiring the first posture information of equipment according to described image
The position of quasi- camera and posture obtain virtual scene image；
Display unit, for showing the reality scene image and the mixed reality image after the virtual scene image superposition.
8. a kind of display system of mixed reality characterized by comprising
Image capture device, for acquiring reality scene image；
Computer, for receiving the virtual scene information that mixed reality equipment is passed back and the reality that image capture device is passed back
Scene image and the first posture information acquire the first posture information of equipment according to described image, adjust the virtual scene letter
The position of Softcam and posture obtain virtual scene image in breath, and by the reality scene image and the virtual scene
Image superposition, obtained mixed reality image, wherein first posture information is used to indicate the position of described image acquisition equipment
It sets and posture；
Display, for showing the mixed reality image.
9. a kind of storage medium, which is characterized in that the storage medium includes the program of storage, wherein described program right of execution
Benefit require any one of 1 to 6 described in mixed reality display methods.
10. a kind of processor, which is characterized in that the processor is for running program, wherein right of execution when described program is run
Benefit require any one of 1 to 6 described in mixed reality display methods.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910419915.9A CN110211238A (en) | 2019-05-20 | 2019-05-20 | Display methods, device, system, storage medium and the processor of mixed reality |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910419915.9A CN110211238A (en) | 2019-05-20 | 2019-05-20 | Display methods, device, system, storage medium and the processor of mixed reality |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110211238A true CN110211238A (en) | 2019-09-06 |
Family
ID=67787854
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910419915.9A Pending CN110211238A (en) | 2019-05-20 | 2019-05-20 | Display methods, device, system, storage medium and the processor of mixed reality |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110211238A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110751728A (en) * | 2019-09-30 | 2020-02-04 | 万翼科技有限公司 | Virtual reality equipment and method with BIM building model mixed reality function |
-
2019
- 2019-05-20 CN CN201910419915.9A patent/CN110211238A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110751728A (en) * | 2019-09-30 | 2020-02-04 | 万翼科技有限公司 | Virtual reality equipment and method with BIM building model mixed reality function |
| CN110751728B (en) * | 2019-09-30 | 2023-05-23 | 万翼科技有限公司 | Virtual reality equipment with BIM building model mixed reality function and method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3494695B1 (en) | Single depth tracked accommodation-vergence solutions | |
| JP4228646B2 (en) | Stereoscopic image generation method and stereoscopic image generation apparatus | |
| CN106302427B (en) | Sharing method and device in reality environment | |
| PH12018502093A1 (en) | Information display method, device and system | |
| US20130300739A1 (en) | Stereoscopic apparel try-on method and device | |
| CN107479699A (en) | Virtual reality exchange method, apparatus and system | |
| WO2014091519A1 (en) | Medical assistance system and method for same | |
| CN109671141A (en) | The rendering method and device of image, storage medium, electronic device | |
| CN104866261B (en) | A kind of information processing method and device | |
| CN112381003B (en) | Motion capture method, motion capture device, motion capture equipment and storage medium | |
| CN109126121A (en) | AR terminal interconnected method, system, device and computer readable storage medium | |
| CN104731338B (en) | One kind is based on enclosed enhancing virtual reality system and method | |
| CN111984114A (en) | Multi-person interaction system based on virtual space and multi-person interaction method thereof | |
| CN108829868A (en) | data display method and device, storage medium and electronic device | |
| CN109523615B (en) | Data processing method and device for virtual animation character actions | |
| CN106887033A (en) | The rendering intent and device of scene | |
| CN107272882A (en) | The holographic long-range presentation implementation method of one species | |
| CN107885334B (en) | Information processing method and virtual equipment | |
| CN106530408A (en) | Museum temporary exhibition planning and design system | |
| CN107133981B (en) | Image processing method and device | |
| CN205610838U (en) | Virtual stereoscopic display's device | |
| CN109360277B (en) | Virtual simulation display control method and device, storage medium and electronic device | |
| CN110211238A (en) | Display methods, device, system, storage medium and the processor of mixed reality | |
| CN110290192A (en) | The storage of data distribution formula and data capture method, device based on block chain | |
| CN108833737A (en) | For the synchronous photo taking control method of polyphaser array |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |