CN110310328A - Mixed reality operates method for registering and device - Google Patents
Mixed reality operates method for registering and device Download PDFInfo
- Publication number
- CN110310328A CN110310328A CN201910661022.5A CN201910661022A CN110310328A CN 110310328 A CN110310328 A CN 110310328A CN 201910661022 A CN201910661022 A CN 201910661022A CN 110310328 A CN110310328 A CN 110310328A
- Authority
- CN
- China
- Prior art keywords
- visual
- operating area
- coordinate system
- visual indicia
- registering
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/003—Navigation within 3D models or images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/006—Mixed reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Software Systems (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Graphics (AREA)
- Computer Hardware Design (AREA)
- General Engineering & Computer Science (AREA)
- Geometry (AREA)
- Remote Sensing (AREA)
- Radar, Positioning & Navigation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Processing Or Creating Images (AREA)
Abstract
The embodiment of the present invention provides a kind of mixed reality operation method for registering and device, this method comprises: obtaining the space coordinate of visual indicia under visual coordinate system, the predetermined position of operating area is arranged in the visual indicia；According to the space coordinate of the visual indicia and the relative pose relationship of the visual indicia and the operating area, pose of the operating area in visual coordinate system is obtained；The virtual three-dimensional model including courses of action and components placement position is transformed into visual coordinate system and is imaged according to pose of the operating area in visual coordinate system, to realize operation registration.The coordinate that visual indicia need to be only obtained in implementation process, without obtaining whole operation area three-dimensional model, so that calculation amount is smaller.It after obtaining the relative position of operating area and visual coordinate system, is rebuild by three-dimensional data, determines courses of action and components placement position, so as to accurately determine region to be operated, and then be advantageously implemented accurately operation registration.
Description
Technical field
The present invention relates to mixed reality application field more particularly to a kind of mixed reality operation method for registering and device.
Background technique
With the development of science and technology, AR (Augmented Reality, augmented reality) and VR (Virtual
Reality, virtual reality technology) it has been dissolved into public life, MR (Mixed Reality, the mixing then risen
Reality technology) etc. new digitals technology also shown great potential value.Unlike AR and VR, MR is by virtual generation
The timely acquisition that real-time, interactive and information are carried out while boundary is merged with real world, enhances the sense of reality of user experience, has
Have authenticity, real-time interactive with the features such as.
In MR application field, MR technology can be also used for guidance one in addition to the Three-dimensional Display that can be used for image is other than interacting
The higher operation of a little precision requirements, such as the installation of precision instrument part.Usual way is the image number for first obtaining operating area
According to operation planning is carried out, whole operation planning is then exported into three-dimensional scene models in three-dimensional Autocad, is uploaded
Onto MR holographic glasses, it is presented in actual environment and carries out overlapping fusion with operating position.User matches in operation
It wears holographic glasses to be operated, so that it may observe directly the structure of operating area, so that installation be instructed to operate, improve operation peace
Full property and accuracy.
In order to which dummy model to be mapped in true environment, need to be registrated it.The accuracy of registration operation can shadow
Ring the result of operation.Conventional method carries out manual registration using the tool that MR holographic glasses provide, i.e., by can be manually rotated, translate
Dummy model several characteristic points corresponding in true environment are coincided together Deng operation and realize Rigid Registration, locking registration
As a result.This method needs the object under real scene to remain stationary, otherwise with regard to needing to reregister.And in MR glasses
Middle progress manual registration error is larger, if operation precision requirement is very high, there are hidden danger for this method.
The current method for registering that MR technology is applied to when guidance accurately operates is not accurate enough, it is difficult to will be virtual
Model accurately matches together with real scene, to influence operation precision.
Summary of the invention
To solve the above-mentioned problems, the embodiment of the present invention provides a kind of mixed reality operation method for registering and device.
In a first aspect, the embodiment of the present invention provides a kind of mixed reality operation method for registering, comprising: obtain visual coordinate system
The predetermined position of operating area is arranged in the space coordinate of lower visual indicia, the visual indicia；According to the visual indicia
Space coordinate and the visual indicia and the operating area relative pose relationship, obtain operating area vision sit
Pose in mark system；It will include courses of action and components placement position according to pose of the operating area in visual coordinate system
Virtual three-dimensional model is transformed into visual coordinate system and is imaged, to realize operation registration.
Further, described according to the space coordinate of the visual indicia and the visual indicia and the operating space
The relative pose relationship in domain obtains operating area before the pose in visual coordinate system, further includes: by calibration, obtains institute
State the relative pose of operating area and the visual indicia.
Further, described by the virtual three-dimensional model including courses of action and components placement position, it is transformed into vision seat
Before mark system is imaged, further includes: according to the 3 d scan data of operating area, carry out three-dimensional reconstruction；After reconstruction
Threedimensional model obtains the courses of action and the components placement position, to generate the virtual three-dimensional model.
Further, the visual indicia is arranged on guide plate matrix, and the guide plate is arranged in a manner of being bonded to be operated
At region.
Further, described to obtain under visual coordinate system before the space coordinate of visual indicia, further includes: according to the behaviour
Make path and the components placement position, the guide plate including visual indicia is generated by 3D printing.
Further, the guide plate waits for that observation window and/or pilot hole are arranged at operation；The observation window, for observing behaviour
Make the position of instrument；The pilot hole, for guiding operation instrument.
Further, the space coordinate for obtaining visual indicia under visual coordinate system, the acquisition operating area are regarding
Feel the pose and described by the virtual three-dimensional model including courses of action and components placement position in coordinate system, is transformed into view
Feel that coordinate system is imaged, is servo-actuated in real time carry out.
Second aspect, the embodiment of the present invention provide a kind of mixed reality operation registration apparatus, comprising: coordinate obtaining module,
For obtaining the space coordinate of visual indicia under visual coordinate system, the predeterminated position of operating area is arranged in the visual indicia
Place；Coordinate transferring, according to the space coordinate of the visual indicia and the phase of the visual indicia and the operating area
To position orientation relation, pose of the operating area in visual coordinate system is obtained；Imaging module, for being regarded according to operating area
Feel that the virtual three-dimensional model including courses of action and components placement position is transformed into visual coordinate system by the pose in coordinate system
It is imaged, to realize operation registration.
The third aspect, the embodiment of the present invention provides a kind of electronic equipment, including memory, processor and is stored in memory
Computer program that is upper and can running on a processor, processor realize first aspect present invention mixed reality behaviour when executing program
The step of making method for registering.
Fourth aspect, the embodiment of the present invention provide a kind of non-transient computer readable storage medium, are stored thereon with calculating
Machine program realizes the step of first aspect present invention mixed reality operation method for registering when the computer program is executed by processor
Suddenly.
Mixed reality operation method for registering and device provided in an embodiment of the present invention, obtain visual indicia under visual coordinate system
Space coordinate, the coordinate of visual indicia need to be only obtained in implementation process, without obtaining the threedimensional model in whole operation region, from
And calculation amount is smaller.By the virtual three-dimensional model including courses of action and components placement position, it is transformed into the progress of visual coordinate system
Imaging is rebuild by three-dimensional data after obtaining the relative position of operating area and visual coordinate system, determines courses of action and portion
Part placement location so as to accurately determine region to be operated, and then is advantageously implemented accurately operation registration.
Detailed description of the invention
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below
There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is this hair
Bright some embodiments for those of ordinary skill in the art without creative efforts, can be with root
Other attached drawings are obtained according to these attached drawings.
Fig. 1 is that mixed reality provided in an embodiment of the present invention operates method for registering flow chart；
Fig. 2 is that mixed reality provided in an embodiment of the present invention operates method for registering application scenario diagram；
Fig. 3 is the guide plate structure schematic diagram provided in an embodiment of the present invention with visual indicia；
Fig. 4 is that mixed reality provided in an embodiment of the present invention operates registration apparatus structure chart；
Fig. 5 is the entity structure schematic diagram of a kind of electronic equipment provided in an embodiment of the present invention.
Specific embodiment
In order to make the object, technical scheme and advantages of the embodiment of the invention clearer, below in conjunction with the embodiment of the present invention
In attached drawing, technical scheme in the embodiment of the invention is clearly and completely described, it is clear that described embodiment is
A part of the embodiment of the present invention, instead of all the embodiments.Based on the embodiments of the present invention, those of ordinary skill in the art
Every other embodiment obtained without creative efforts, shall fall within the protection scope of the present invention.
Fig. 1 is that mixed reality provided in an embodiment of the present invention operates method for registering flow chart, and Fig. 2 mentions for the embodiment of the present invention
The mixed reality of confession operates method for registering application scenario diagram, and as shown in Figure 1, 2, the embodiment of the present invention provides a kind of mixed reality behaviour
Make method for registering, comprising:
101, the space coordinate of visual indicia under visual coordinate system is obtained, the pre- of operating area is arranged in the visual indicia
If at position.
The embodiment of the present invention realized by the wearable MR glasses such as mixed reality glasses, for example, HoloLens or
MagicLeap mixed reality glasses.MR glasses 203 perceive the movement of user's sight by eyeball tracking sensor, and then automatic
Virtual three-dimensional model is adjusted, and is projected in user's eye by light.In addition to this depth camera is also equipped on MR glasses 203
It perceives user to carry out SLAM (positioning and map structuring immediately) calculating with inertial navigation sensors such as gyroscopes and is closing
Position and posture in scene, and then determine position of the dummy model relative to user.
In 101, visual coordinate system is the coordinate system of the depth camera in MR glasses 203, in the default position of operating area
The place of setting is provided with visual indicia 201.Visual indicia 201 can be used QR code (one kind of two-dimensional bar code) or have height right by a series of
Specific pattern than degree forms, and the angle point that chequered with black and white region is constituted in pattern can be used as characteristic point convenient for image procossing and vision
Detection.Visual indicia constitutes a space coordinates.Visual indicia is obtained in vision by the depth camera module of MR glasses 203
The space coordinate of coordinate system.
102, according to the space coordinate of the visual indicia and the opposite position of the visual indicia and the operating area
Appearance relationship obtains pose of the operating area in visual coordinate system.
In 102, pose (position and posture) relationship of visual indicia 201 and operating area has obtained in advance, according to
The position of visual indicia 201 can calculate position and posture of the operating area in 203 visual coordinate system of MR glasses indirectly.
103, according to pose of the operating area in visual coordinate system, by the void including courses of action and components placement position
Quasi-3-dimensional model is transformed into visual coordinate system and is imaged, to realize operation registration.
In 103, virtual three-dimensional model is the threedimensional model for including courses of action, components placement position and operating area,
Component is the object for needing to be mounted on operating area, before operation and in operating process, need to be projected the threedimensional model by light
Into user's eye.The threedimensional model image of operating area and the true picture of operating area are overlapped in user's eye after projection, then real
The registration of operation is showed.At this point, the three-dimensional model diagram of courses of action, components placement position seem it is accurate and with actual position one
It causes.Operating area is after the pose in visual coordinate system determines, then pose of the threedimensional model in visual coordinate system is accordingly true
It is fixed, the predetermined virtual three-dimensional model including courses of action and components placement position, it is transformed into the progress of visual coordinate system
It is imaged and carries out imaging projection, i.e. completion dummy model 205 and actual scene 204 is registrated, to realize the registration of operation.
It can implement corresponding operation by operating instrument 202.
Mixed reality provided in this embodiment operates method for registering, and the space for obtaining visual indicia under visual coordinate system is sat
It marks, the coordinate of visual indicia need to be only obtained in implementation process, without obtaining the threedimensional model in whole operation region, thus calculation amount
It is smaller.By the virtual three-dimensional model including courses of action and components placement position, it is transformed into visual coordinate system and is imaged, obtained
It behind the relative position for obtaining operating area and visual coordinate system, is rebuild by three-dimensional data, determines that courses of action and component place position
It sets, so as to accurately determine region to be operated, and then is advantageously implemented accurately operation registration.
Content based on the above embodiment, as a kind of alternative embodiment, according to the space coordinate of the visual indicia, with
And the relative pose relationship of the visual indicia and the operating area, obtain pose of the operating area in visual coordinate system it
Before, further includes: by calibration, obtain the relative pose of the operating area and the visual indicia.I.e. visual indicia relative to
The relative pose relationship of operating area can be obtained by calibration in advance before the use.As alternative embodiment, the relative pose
Relationship can also be measured in Three-dimensional Design Software and be obtained.By calibration, the phase of the operating area and the visual indicia is obtained
To pose, be conducive to the calculating speed for improving subsequent coordinates conversion.
Content based on the above embodiment will include courses of action and components placement position as a kind of alternative embodiment
Virtual three-dimensional model, be transformed into before visual coordinate system is imaged, further includes: according to the 3-D scanning number of operating area
According to progress three-dimensional reconstruction；According to the threedimensional model after reconstruction, the courses of action and the components placement position are obtained, with life
At the virtual three-dimensional model.
Before implementing operation, three-dimensional reconstruction is carried out according to the 3 d scan data of operating area, obtains operating area after reconstruction
Threedimensional model.For example, scanning three-dimensional data by 3D point cloud, CT and CBCT (conical beam CT) etc. carries out three-dimensional reconstruction.In weight
Operation planning, design operation path and ideal components placement position are carried out in threedimensional model after building, such as pass through design software
The planning in path and placement location is carried out, components placement position may also include the angle of component simultaneously and place the information such as depth.
Threedimensional model after being determined according to courses of action and the components placement position, virtual three-dimensional model in generation.
Method provided in an embodiment of the present invention, is rebuild by 3 d scan data, ultimately generates the virtual three-dimensional model,
Region and courses of action to be operated can be accurately determined, to be advantageously implemented accurate operation before operation is implemented
Registration.
Content based on the above embodiment, as a kind of alternative embodiment, visual indicia is arranged on guide plate matrix, guide plate
It is arranged at operating area by the way of fitting.
Fig. 3 is the guide plate structure schematic diagram provided in an embodiment of the present invention with visual indicia, as shown in figure 3, in order to guarantee
The relative position of visual indicia and operating area is not susceptible to change, and in the embodiment of the present invention, visual indicia 303 is arranged in guide plate
Matrix 301 on.Guide plate is arranged on operating area by the way of fitting, i.e., is set according to the contoured surface shape of operating area
Family planning allows to tightly be fitted in operating area face, provides good immobilizing foundation at the inner surface of guide plate.
As an alternative embodiment, visual indicia 303 can be connect by connecting rod 302 with the matrix 301 of guide plate, thus
Outside extended operating area, consequently facilitating depth camera obtains visual indicia, it will not be because of the screening of operating area in operating process
It keeps off and is affected.
Content based on the above embodiment, as a kind of alternative embodiment, visual indicia under the acquisition visual coordinate system
Space coordinate before, further includes: according to the courses of action and the components placement position, by 3D printing generate described in lead
Plate.
In order to guarantee that the relative position of visual indicia and operating area is not susceptible to change in operation, the present invention is real
It applies example visual indicia is arranged on guide plate, before implementing operation, integrally be printed by rapid shaping techniques such as 3D printings
The above-mentioned guide plate equipped with visual indicia.In view of guide plate is for implementing operation, convenient for placing component, the specific shape of guide plate
Shape need to specifically be determined according to courses of action and components placement position.As alternative embodiment, courses of action and components placement position
It can be obtained according to virtual three-dimensional model in above-described embodiment.The embodiment of the present invention is put according to the courses of action and the component
Seated position generates the plantation guide plate by 3D printing, so as to reduce the Production Time of guide plate, and can generate and be conducive to
Operate the guide plate implemented.
Content based on the above embodiment, as a kind of alternative embodiment, the guide plate wait at operation be arranged observation window and/
Or pilot hole；The observation window, for observing the position of operation instrument；The pilot hole, for guiding operation instrument.Such as Fig. 3,
Guide plate waits for that pilot hole 304 is arranged at operation.In order in operation, further increase the precision of registration, can wait grasping in guide plate
At work, that is, the key position of operation is embodied, observation window is set, the observation of operating process is easy to implement.Meanwhile it may also set up
Pilot hole, for guiding the position of operation instrument mobile.
Content based on the above embodiment, as a kind of alternative embodiment, visual indicia under the acquisition visual coordinate system
Space coordinate, it is described to obtain pose of the operating area in visual coordinate system, and will include that courses of action and component are placed
The virtual three-dimensional model of position is transformed into visual coordinate system and is imaged, and is servo-actuated in real time carry out.
Due to operator or a certain range of movement can occur, operating area also due to the reason of application scenarios generate
It is mobile.In the embodiment of the present invention, to the spatial position transformation of the acquisition of the coordinate of visual indicia, three dimensional virtual models and virtual three
The imaging of dimension module is servo-actuated in real time carries out.Real-time perfoming refers to the acquisition of coordinate and conversion is to execute in real time, non-delay.With
When change in location occurs for dynamic finger operating area or operator, the coordinate of visual indicia is obtained again accordingly, carry out coordinate conversion
And imaging generates.In this situation, three dimensional virtual models can follow in real scene operating area or operator together with
It is dynamic, to give operator's accurately visual feedback, avoid the generation of operating accident.
Fig. 4 is that mixed reality provided in an embodiment of the present invention operates registration apparatus structure chart, mixed reality behaviour as shown in Figure 4
It include: coordinate obtaining module 401, coordinate transferring 402 and imaging module 403 as registration apparatus.Wherein, coordinate obtains
Module 401 is used to obtain the space coordinate of visual indicia under visual coordinate system, and the pre- of operating area is arranged in the visual indicia
If at position；Coordinate transferring 402 is according to the space coordinate of the visual indicia and the visual indicia and the operation
The relative pose relationship in region obtains pose of the operating area in visual coordinate system；Imaging module 403 is used for according to behaviour
Make pose of the region in visual coordinate system to be transformed into the virtual three-dimensional model including courses of action and components placement position
Visual coordinate system is imaged, to realize operation registration.
The predetermined position of operating area is provided with visual indicia, visual indicia constitutes a space coordinates.Pass through
Coordinate obtaining module 401 obtains visual indicia in the space coordinate of visual coordinate system.
The position orientation relation of visual indicia and operating area has obtained in advance, and coordinate transferring 402 is according to visual indicia
Position can calculate position and posture of the operating area in visual coordinate system under true environment indirectly.
Virtual three-dimensional model is the threedimensional model for including courses of action, components placement position and operating area, operation before and
In operating process, which need to be projected in user's eye by light.After projection user eye in operating area three-dimensional
The true picture of model image and operating area is overlapped, then realizes the registration of operation.At this point, courses of action, component place position
The three-dimensional model diagram set seems accurate and consistent with actual position.Pose of the operating area in visual coordinate system determines
Afterwards, pose of the threedimensional model in visual coordinate system accordingly determines, imaging module 403 is predetermined including operation road
The virtual three-dimensional model of diameter and components placement position is transformed into visual coordinate system and is imaged and is carried out imaging projection, that is, completes
Dummy model is registrated with actual scene, to realize the registration of operation.
Installation practice provided in an embodiment of the present invention is the detailed process and in detail in order to realize above-mentioned each method embodiment
Thin content please refers to above method embodiment, and details are not described herein again.
Mixed reality provided in an embodiment of the present invention operates registration apparatus, and coordinate obtaining module obtains to be regarded under visual coordinate system
Feel the space coordinate of label, the coordinate of visual indicia need to be only obtained in implementation process, without obtaining the three-dimensional in whole operation region
Model, so that calculation amount is smaller.Imaging module turns the virtual three-dimensional model including courses of action and components placement position
It changes to visual coordinate system to be imaged, after obtaining the relative position of operating area and visual coordinate system, passes through three-dimensional data weight
It builds, determines courses of action and components placement position, so as to accurately determine region to be operated, and then be advantageously implemented standard
True operation registration.
Fig. 5 is the entity structure schematic diagram of a kind of electronic equipment provided in an embodiment of the present invention, as shown in figure 5, the electronics
Equipment may include: processor (processor) 501, communication interface (Communications Interface) 502, storage
Device (memory) 503 and bus 504, wherein processor 501, communication interface 502, memory 503 complete phase by bus 504
Communication between mutually.Communication interface 502 can be used for the information transmission of electronic equipment.Processor 501 can call in memory 503
Logical order, with execute include following method: obtain visual coordinate system under visual indicia space coordinate, the vision mark
The predetermined position of operating area is arranged in note；According to the space coordinate of the visual indicia and the visual indicia and institute
The relative pose relationship of operating area is stated, pose of the operating area in visual coordinate system is obtained；According to operating area in vision
Pose in coordinate system, by the virtual three-dimensional model including courses of action and components placement position, be transformed into visual coordinate system into
Row imaging, to realize operation registration.
In addition, the logical order in above-mentioned memory 503 can be realized by way of SFU software functional unit and conduct
Independent product when selling or using, can store in a computer readable storage medium.Based on this understanding, originally
Substantially the part of the part that contributes to existing technology or the technical solution can be in other words for the technical solution of invention
The form of software product embodies, which is stored in a storage medium, including some instructions to
So that a computer equipment (can be personal computer, server or the network equipment etc.) executes the above-mentioned each side of the present invention
The all or part of the steps of method embodiment.And storage medium above-mentioned include: USB flash disk, mobile hard disk, read-only memory (ROM,
Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic or disk etc. it is various
It can store the medium of program code.
On the other hand, the embodiment of the present invention also provides a kind of non-transient computer readable storage medium, is stored thereon with meter
Calculation machine program, the computer program are implemented to carry out the transmission method of the various embodiments described above offer when being executed by processor, such as
It include: the space coordinate for obtaining visual indicia under visual coordinate system, the predeterminated position of operating area is arranged in the visual indicia
Place；According to the space coordinate of the visual indicia and the relative pose relationship of the visual indicia and the operating area, obtain
Pose of the extract operation region in visual coordinate system；It will include operation road according to pose of the operating area in visual coordinate system
The virtual three-dimensional model of diameter and components placement position is transformed into visual coordinate system and is imaged, to realize operation registration.
The apparatus embodiments described above are merely exemplary, wherein described, unit can as illustrated by the separation member
It is physically separated with being or may not be, component shown as a unit may or may not be physics list
Member, it can it is in one place, or may be distributed over multiple network units.It can be selected according to the actual needs
In some or all of the modules achieve the purpose of the solution of this embodiment.Those of ordinary skill in the art are not paying creativeness
Labour in the case where, it can understand and implement.
Through the above description of the embodiments, those skilled in the art can be understood that each embodiment can
It realizes by means of software and necessary general hardware platform, naturally it is also possible to pass through hardware.Based on this understanding, on
Stating technical solution, substantially the part that contributes to existing technology can be embodied in the form of software products in other words, should
Computer software product may be stored in a computer readable storage medium, such as ROM/RAM, magnetic disk, CD, including several fingers
It enables and using so that a computer equipment (can be personal computer, server or the network equipment etc.) executes each implementation
Method described in certain parts of example or embodiment.
Finally, it should be noted that the above embodiments are merely illustrative of the technical solutions of the present invention, rather than its limitations；Although
Present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that: it still may be used
To modify the technical solutions described in the foregoing embodiments or equivalent replacement of some of the technical features；
And these are modified or replaceed, technical solution of various embodiments of the present invention that it does not separate the essence of the corresponding technical solution spirit and
Range.
Claims (10)
1. a kind of mixed reality operates method for registering characterized by comprising
The space coordinate of visual indicia under visual coordinate system is obtained, the predeterminated position of operating area is arranged in the visual indicia
Place；
According to the space coordinate of the visual indicia and the relative pose relationship of the visual indicia and the operating area,
Obtain pose of the operating area in visual coordinate system；
According to pose of the operating area in visual coordinate system, by the virtual three-dimensional mould including courses of action and components placement position
Type is transformed into visual coordinate system and is imaged, to realize operation registration.
2. mixed reality according to claim 1 operates method for registering, which is characterized in that described according to the visual indicia
Space coordinate and the visual indicia and the operating area relative pose relationship, obtain operating area vision sit
Before pose in mark system, further includes:
By calibration, the relative pose of the operating area and the visual indicia is obtained.
3. mixed reality according to claim 1 operates method for registering, which is characterized in that it is described will include courses of action and
The virtual three-dimensional model of components placement position is transformed into before visual coordinate system is imaged, further includes:
According to the 3 d scan data of operating area, three-dimensional reconstruction is carried out；
According to the threedimensional model after reconstruction, the courses of action and the components placement position are obtained, to generate described virtual three
Dimension module.
4. mixed reality according to claim 1 operates method for registering, which is characterized in that the visual indicia setting is being led
On plate matrix, the guide plate is arranged at operating area in a manner of being bonded.
5. mixed reality according to claim 4 operates method for registering, which is characterized in that under the acquisition visual coordinate system
Before the space coordinate of visual indicia, further includes:
According to the courses of action and the components placement position, the guide plate including visual indicia is generated by 3D printing.
6. mixed reality according to claim 4 operates method for registering, which is characterized in that the guide plate waits for being arranged at operation
Observation window and/or pilot hole；
The observation window, for observing the position of operation instrument；
The pilot hole, for guiding operation instrument.
7. mixed reality according to claim 1 operates method for registering, which is characterized in that under the acquisition visual coordinate system
The space coordinate of visual indicia, it is described obtain pose of the operating area in visual coordinate system and it is described will include operation road
The virtual three-dimensional model of diameter and components placement position is transformed into visual coordinate system and is imaged, and is servo-actuated in real time carry out.
8. a kind of mixed reality operates registration apparatus characterized by comprising
Coordinate obtaining module, for obtaining the space coordinate of visual indicia under visual coordinate system, the visual indicia setting is being grasped
Make the predetermined position in region；
Coordinate transferring, according to the space coordinate of the visual indicia and the visual indicia and the operating area
Relative pose relationship obtains pose of the operating area in visual coordinate system；
Imaging module will include that courses of action and component are put for the pose according to operating area in visual coordinate system
The virtual three-dimensional model of seated position is transformed into visual coordinate system and is imaged, to realize operation registration.
9. a kind of electronic equipment including memory, processor and stores the calculating that can be run on a memory and on a processor
Machine program, which is characterized in that the processor is realized when executing described program to be mixed now as described in any one of claim 1 to 7
The step of method for registering, is made in practical operation.
10. a kind of non-transient computer readable storage medium, is stored thereon with computer program, which is characterized in that the computer
The step of mixed reality operates method for registering as described in any one of claim 1 to 7 is realized when program is executed by processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910661022.5A CN110310328B (en) | 2019-07-22 | 2019-07-22 | Mixed reality operation registration method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910661022.5A CN110310328B (en) | 2019-07-22 | 2019-07-22 | Mixed reality operation registration method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110310328A true CN110310328A (en) | 2019-10-08 |
| CN110310328B CN110310328B (en) | 2021-04-30 |
Family
ID=68080443
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910661022.5A Active CN110310328B (en) | 2019-07-22 | 2019-07-22 | Mixed reality operation registration method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110310328B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110711030A (en) * | 2019-10-21 | 2020-01-21 | 北京国润健康医学投资有限公司 | Femoral head necrosis minimally invasive surgery navigation system and surgery method based on AR technology |
| CN111476833A (en) * | 2020-04-02 | 2020-07-31 | 北京触幻科技有限公司 | Method for registering model based on CT/MRI (computed tomography/magnetic resonance imaging) with real object in mixed reality |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150256815A1 (en) * | 2012-10-01 | 2015-09-10 | Telefonaktiebolaget L M Ericsson (Publ) | 3d camera calibration |
| US20180247456A1 (en) * | 2017-02-27 | 2018-08-30 | Hiscene (Shanghai) Information Technology Co., Ltd. | Method and device for augmented reality display of real physical model |
| CN108961326A (en) * | 2018-07-03 | 2018-12-07 | 雅客智慧（北京）科技有限公司 | An a kind of method for registering and electronic equipment for kind of tooth operation vision guided navigation |
| CN109498156A (en) * | 2017-09-14 | 2019-03-22 | 北京大华旺达科技有限公司 | A kind of head operation air navigation aid based on 3-D scanning |
-
2019
- 2019-07-22 CN CN201910661022.5A patent/CN110310328B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150256815A1 (en) * | 2012-10-01 | 2015-09-10 | Telefonaktiebolaget L M Ericsson (Publ) | 3d camera calibration |
| US20180247456A1 (en) * | 2017-02-27 | 2018-08-30 | Hiscene (Shanghai) Information Technology Co., Ltd. | Method and device for augmented reality display of real physical model |
| CN109498156A (en) * | 2017-09-14 | 2019-03-22 | 北京大华旺达科技有限公司 | A kind of head operation air navigation aid based on 3-D scanning |
| CN108961326A (en) * | 2018-07-03 | 2018-12-07 | 雅客智慧（北京）科技有限公司 | An a kind of method for registering and electronic equipment for kind of tooth operation vision guided navigation |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110711030A (en) * | 2019-10-21 | 2020-01-21 | 北京国润健康医学投资有限公司 | Femoral head necrosis minimally invasive surgery navigation system and surgery method based on AR technology |
| CN111476833A (en) * | 2020-04-02 | 2020-07-31 | 北京触幻科技有限公司 | Method for registering model based on CT/MRI (computed tomography/magnetic resonance imaging) with real object in mixed reality |
| CN111476833B (en) * | 2020-04-02 | 2020-11-13 | 北京触幻科技有限公司 | Method for registering model based on CT/MRI (computed tomography/magnetic resonance imaging) with real object in mixed reality |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110310328B (en) | 2021-04-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105354820B (en) | Adjust the method and device of virtual reality image | |
| KR101229283B1 (en) | Method and system for visualising virtual three-dimensional objects | |
| CN103365416B (en) | System and method for virtual engineering | |
| TWI521469B (en) | Two - dimensional Roles Representation of Three - dimensional Action System and Method | |
| CN108401461A (en) | Three-dimensional mapping method, device and system, cloud platform, electronic equipment and computer program product | |
| CN109767452A (en) | A kind of object localization method and device, unmanned plane | |
| CN109003325A (en) | A kind of method of three-dimensional reconstruction, medium, device and calculate equipment | |
| CN109700550A (en) | A kind of augmented reality method and device for dental operation | |
| CN104916182A (en) | Immersion type virtual reality maintenance and training simulation system | |
| CN113421346B (en) | Design method of AR-HUD head-up display interface for enhancing driving feeling | |
| JP2007042055A (en) | Image processing method and image processor | |
| KR101997298B1 (en) | Selection of objects in a three-dimensional virtual scene | |
| CN106454311A (en) | LED three-dimensional imaging system and method | |
| CN109313821A (en) | Three dimensional object scanning feedback | |
| CN114022565A (en) | Alignment method and alignment device for display equipment and vehicle-mounted display system | |
| CN110310328A (en) | Mixed reality operates method for registering and device | |
| Whitaker et al. | Object calibration for augmented reality | |
| EP3990302B1 (en) | Method and control device for operating a virtual reality headset in a vehicle | |
| Moser et al. | Evaluation of user-centric optical see-through head-mounted display calibration using a leap motion controller | |
| CN109345581A (en) | Augmented reality method, apparatus and system based on more mesh cameras | |
| CN109920000A (en) | A kind of augmented reality method without dead angle based on polyphaser collaboration | |
| CN116110270A (en) | Multi-degree-of-freedom driving simulator based on mixed reality | |
| CN110322539A (en) | Threedimensional model cutting process display methods, device and the electronic equipment of three-dimension object | |
| CN113238556A (en) | Water surface unmanned ship control system and method based on virtual reality | |
| CN112530022A (en) | Method for computer-implemented simulation of LIDAR sensors in a virtual environment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |