CN107153369A - It is a kind of to position the localization method of object, system and wear display device - Google Patents
It is a kind of to position the localization method of object, system and wear display device Download PDFInfo
- Publication number
- CN107153369A CN107153369A CN201710370301.7A CN201710370301A CN107153369A CN 107153369 A CN107153369 A CN 107153369A CN 201710370301 A CN201710370301 A CN 201710370301A CN 107153369 A CN107153369 A CN 107153369A
- Authority
- CN
- China
- Prior art keywords
- illuminator
- display device
- dimensional coordinate
- imu
- handle
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B19/00—Programme-control systems
- G05B19/02—Programme-control systems electric
- G05B19/04—Programme control other than numerical control, i.e. in sequence controllers or logic controllers
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/10—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration
- G01C21/12—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning
- G01C21/16—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation
- G01C21/165—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation combined with non-inertial navigation instruments
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/20—Instruments for performing navigational calculations
- G01C21/206—Instruments for performing navigational calculations specially adapted for indoor navigation
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/26—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network
- G01C21/28—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network with correlation of data from several navigational instruments
- G01C21/30—Map- or contour-matching
- G01C21/32—Structuring or formatting of map data
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
Landscapes
- Engineering & Computer Science (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Optics & Photonics (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
Position the localization method of object, system the invention discloses a kind of and wear display device.This method includes：The camera set by wearing on display device obtains the spatial image for wearing space residing for display device；The first attitude information of display device is worn in the first IMU collections by wearing display device, according to spatial image and the first attitude information, obtains the six-freedom degree information for wearing display device；Wear historical track and the 2nd IMU historical track that display device obtains the illuminator on handle；Wear the attitude information that display device receives the handle of the 2nd IMU collections；According to the positional information of luminous point in historical track and spatial image, the three-dimensional coordinate of illuminator is determined；According to three-dimensional coordinate and the attitude information of handle, the six-freedom degree information of handle is determined.It can be seen that, the location equipment in the present invention is provided on positioning object, it is not necessary to pre-set external location equipment, using facility, and orientation range is wide, improves Consumer's Experience.
Description
Technical field
The present invention relates to field of locating technology, more particularly to a kind of localization method for positioning object, system and display is worn
Equipment.
Background technology
Location technology is widely used in the fields such as virtual reality, augmented reality, motion capture, is the important of man-machine interaction
Part.Wherein, optical alignment so that its is high-precision, inexpensive, high refresh rate the advantages of turn into indoor positioning field by most
Wide variety of technology.
Optical alignment of the prior art carries out the positioning of object generally by the location equipment for being placed on object.But
It is that the orientation range of this scheme relies on the visual angle of location equipment and recognizable distance, and orientation range is limited, is particularly wearing
In the positioning application of display device and handle, once display device or handle are worn not in orientation range, it is impossible to realization is worn
Display device or handle are accurately positioned；And need to pre-set external location equipment, using not convenient, influence Consumer's Experience.
The content of the invention
In view of the orientation range of optical alignment scheme of the prior art is limited, using not convenient, Consumer's Experience is influenceed
Problem, it is proposed that the localization method and system of a kind of positioning object of the invention, to solve or to solve at least in part above-mentioned
Problem.
According to an aspect of the invention, there is provided a kind of localization method for positioning object, methods described includes：
The spatial image in space residing for display device is worn described in the camera acquisition set by wearing on display device；
Pass through the first Inertial Measurement Unit (the Inertial measurement unit, abbreviation for wearing display device
IMU the first attitude information of display device) is worn described in collection, according to the spatial image and first attitude information, is obtained
Take the six-freedom degree information for wearing display device；
It is described to wear the historical track that display device obtains the historical track and the 2nd IMU of illuminator；Wherein, it is described luminous
Body and the 2nd IMU are arranged to be worn on the corresponding handle of display device with described；
It is described to wear the attitude information that display device receives the handle of the 2nd IMU collections；
According to the positional information of luminous point in the historical track and the spatial image, the three-dimensional of the illuminator is determined
Coordinate, wherein, the luminous point is corresponding with the illuminator；
According to the three-dimensional coordinate and the attitude information of the handle, the six-freedom degree information of the handle is determined.
Display device is worn there is provided one kind according to another aspect of the present invention, the display device of wearing includes taking the photograph
As head, the first Inertial Measurement Unit IMU；The display device of wearing also includes：
Image acquisition units, for the space diagram by wearing space residing for display device described in camera acquisition
Picture；
Positioning unit, is believed by the first posture that display device is worn described in the first Inertial Measurement Unit IMU collections
Breath, according to the spatial image and first attitude information, wears the six-freedom degree information of display device described in acquisition；
Acquiring unit, for obtaining the historical track of illuminator and the 2nd IMU historical track；Wherein, the illuminator
It is arranged at and is worn with described on the corresponding handle of display device with the 2nd IMU；
Receiving unit, the attitude information of the handle for receiving the 2nd IMU collections；
Three-dimensional coordinate determining unit, for the position letter according to luminous point in the historical track and the spatial image
Breath, determines the three-dimensional coordinate of the illuminator, wherein, the luminous point is corresponding with the illuminator；
The positioning unit, is additionally operable to the attitude information according to the three-dimensional coordinate and the handle, determines the handle
Six-freedom degree information.
According to a further aspect of the invention there is provided a kind of alignment system for positioning object, the system includes：As before
It is described to wear display device and wear the corresponding handle of display device with described；
Display device is worn described in the wired or wireless connection of handle.
In summary, the camera that the present invention is set by wearing on display device, which is obtained, wears space residing for display device
Spatial image；The first posture that display device is worn in the first Inertial Measurement Unit IMU collections by wearing display device is believed
Breath, according to spatial image and the first attitude information, obtains the six-freedom degree information for wearing display device, realization is worn display and set
Standby positioning；Also, wear display device and obtain the historical track of illuminator and the 2nd IMU historical track；Wherein, illuminator
It is arranged at the 2nd IMU on handle corresponding with wearing display device；Receive the attitude information of the handle of the 2nd IMU collections；
According to the positional information of luminous point in historical track and spatial image, the three-dimensional coordinate of illuminator is determined, wherein, luminous point and hair
Body of light is corresponding；According to three-dimensional coordinate and the attitude information of handle, the six-freedom degree information of handle is determined, determining for handle is realized
Position.It can be seen that, technical scheme is different from camera external in the prior art, is by wearing taking the photograph on display device
Realized as head and wear the positioning of display device and handle corresponding with wearing display device, camera is with wearing display device
Move and move, orientation range is wide.That is, the location equipment in technical scheme is provided in positioning object
On, it is not necessary to external location equipment is pre-set, using facility, orientation range is wide, can improve Consumer's Experience.
Brief description of the drawings
A kind of schematic flow sheet of the localization method for positioning object that Fig. 1 provides for one embodiment of the invention；
A kind of structural representation for wearing display device that Fig. 2 provides for one embodiment of the invention；
A kind of structural representation of the alignment system for positioning object that Fig. 3 provides for one embodiment of the invention.
Embodiment
The present invention mentality of designing be：By being arranged on the spatial image that the camera worn on display device is gathered, knot
The first attitude information for wearing display device for being arranged on the first IMU collections worn on display device is closed, display is worn in realization
The positioning of equipment；And, the historical track for the illuminator being arranged on by determination on handle corresponding with wearing display device, with
And the 2nd IMU collection handle attitude information, realize the positioning of handle.To make the object, technical solutions and advantages of the present invention
Clearer, below in conjunction with accompanying drawing, embodiment of the present invention is described further in detail.
A kind of schematic flow sheet of the localization method for positioning object that Fig. 1 provides for one embodiment of the invention.Such as Fig. 1 institutes
Show, this method includes：
Step S110, the camera set by wearing on display device obtains the space for wearing space residing for display device
Image.
In the present embodiment, it can be wide-angle camera to be arranged on the camera worn on display device so that the present embodiment
In be applicable orientation range it is wider.Also, in order to be obtained in being positioned in handle by spatial image and lighting in image
The three-dimensional coordinate of the corresponding illuminator of point, the number for wearing the camera on display device is at least 2, when display is worn in collection
During spatial image residing for equipment, at least two camera gathers spatial image simultaneously, and each camera obtains a space
Image.Because in the present embodiment being at least two wide-angle camera, spatial image is also at least two.
Step S120, the first of display device is worn by the first Inertial Measurement Unit IMU collections for wearing display device
Attitude information, according to spatial image and the first attitude information, obtains the six-freedom degree information for wearing display device.
Here Inertial Measurement Unit is Inertial measurement unit, hereinafter referred to as IMU.
In the present embodiment, according to the spatial structural form in spatial image, using instant positioning and map structuring
(simultaneous localization and mapping, abbreviation slam) method is calculated, can obtain and wear display
The one-movement-freedom-degree (i.e. three-dimensional coordinate) of equipment；Wear display device first gathered according to the first Inertial Measurement Unit IMU
Attitude information (for example, quaternary number information), can obtain wear display device rotational freedom (i.e. yaw angle, the angle of pitch and
Roll angle)；The six-freedom degree letter for wearing display device is assured that using above-mentioned one-movement-freedom-degree and rotational freedom
Breath, realizes the positioning for wearing display device.
Step S130, wears display device and obtains the historical track of illuminator and the 2nd IMU historical track；Wherein, send out
Body of light and the 2nd IMU are arranged on handle corresponding with wearing display device.
In the present embodiment, the positioning of itself will not only be realized by wearing display device, also realize and wear display device
The positioning of corresponding handle.To realize the positioning of handle, it is necessary to obtain the historical track and the 2nd IMU of the illuminator on handle
Historical track.
In the present embodiment, an illuminator is set on handle, multiple illuminators can also be set.Multiple illuminators are set
Purpose be, when carrying out the positioning of handle, to prevent from occurring in spatial image other interference luminous points, or, in no handle
Historical track in the case of, it is possible to use the regularity of distribution of the luminous point corresponding with multiple illuminators in spatial image is carried out
The positioning of handle.
Step S140, wears the attitude information that display device receives the handle of the 2nd IMU collections.
For example, the quaternary number information of the handle of the 2nd IMU collections.
Step S150, according to the positional information of luminous point in historical track and spatial image, determines the three-dimensional seat of illuminator
Mark, wherein, luminous point is corresponding with illuminator, so, according to the positional information of luminous point in image, it may be determined that with luminous point
The three-dimensional coordinate of corresponding illuminator.
For example, in the above description, it is indicated that the number of the camera in the present embodiment is at least two, then pair of acquisition
Answering the image of same illuminator just has at least two, it is possible to utilize binocular imaging principle, according to correspondence at least two image
The positional information of the luminous point of same illuminator, obtains the three-dimensional coordinate of the illuminator.Here binocular imaging principle, is base
In principle of parallax and using two images of the binocular camera from different position acquisition objects, by calculating corresponding points in image
Between position deviation, the method to obtain object dimensional aggregate information.The advantage of this method is efficiency high, precision height, structure letter
Single, low cost and other advantages.
Step S160, according to three-dimensional coordinate and the attitude information of handle, determines the six-freedom degree information of handle.
Here three-dimensional coordinate can determine the one-movement-freedom-degree information of handle；Attitude information (quaternary number) can determine hand
The rotational freedom information of handle, is the six-freedom degree information that can determine that handle according to one-movement-freedom-degree and rotational freedom.
It can be seen that, technical scheme is different from camera external in the prior art, is by wearing display device
On camera is realized and wears the positioning of display device and handle corresponding with wearing display device, camera is with wearing display
The movement of equipment and move, orientation range is wide.That is, the location equipment in technical scheme is provided in positioning
On object, it is not necessary to pre-set external location equipment, using facility, orientation range is wide, can improve Consumer's Experience.
In one embodiment of the invention, it is above-mentioned wear display device for virtual reality (Virtual Reality,
Abbreviation VR) equipment, above-mentioned handle is handle corresponding with VR equipment.
In the technical scheme of we, when determining the three-dimensional coordinate of illuminator on handle, different situations are might have,
Such as, the illuminator on handle is not blocked, and has the historical track of illuminator；Illuminator on handle is not blocked, but
It is the historical track without illuminator；Illuminator on handle is blocked, and has the historical track of illuminator.So, it is determined that
During the three-dimensional coordinate of illuminator, the method for use should be different.Illustrated below for above-mentioned situation.
(1) illuminator on handle is not blocked, and has the historical track of illuminator.
In one embodiment of the invention, in step S150 foundation historical track and spatial image luminous point position
Information, determining the three-dimensional coordinate of illuminator includes：
1. the linear velocity and linear acceleration of illuminator are obtained according to the historical track of illuminator；2. add according to linear velocity and line
Speed determines the three-dimensional coordinate of illuminator, is estimated during the three-dimensional coordinate of illuminator here according to historical track, is one and estimates
Calculation value；3. by the three-dimensional coordinate back projection of illuminator to spatial image, the two-dimensional coordinate of illuminator in spatial image is obtained, because
It is an estimated value for the three-dimensional coordinate of illuminator, the two-dimensional coordinate of the illuminator obtained here is also estimated value；4. calculate empty
Between luminous point in image two-dimensional coordinate；Wherein, the luminous point in spatial image is corresponding with illuminator, that is to say, that hair
Luminous point is corresponding to be the actual position of illuminator, so the two-dimensional coordinate of the luminous point calculated in this step should be illuminator
True two-dimensional coordinate, still, now the corresponding relation between luminous point and illuminator has not determined, i.e., the luminous point tool in image
Which illuminator body, which corresponds to, has not determined；5. the two-dimensional coordinate value of the two-dimensional coordinate value of illuminator and luminous point is compared
Compared with, that is, the two-dimensional coordinate of the luminous point of calculating and the two-dimensional coordinate estimated value of illuminator calculated be compared；Will hair
The difference value of the two-dimensional coordinate value of luminous point and the two-dimensional coordinate value of illuminator is defined as and sent out in the luminous point of preset threshold range
Body of light has the luminous point of corresponding relation；6. foundation has the two-dimensional coordinate of the luminous point of corresponding relation with illuminator, it is determined that hair
The three-dimensional coordinate of body of light.
For example, in the above description, it is indicated that the number of the camera in the present embodiment is at least two, then according to it is same
One illuminator has the two-dimensional coordinate of the luminous point of corresponding relation, it is possible to utilizes binocular imaging principle, determines illuminator
Three-dimensional coordinate, it is determined that three-dimensional coordinate be exactly illuminator three-dimensional coordinate actual value.
In described above, an illuminator can be set on handle, multiple illuminators can also be set.The above method is outstanding
It is applied to the situation of multiple illuminators.When there are multiple illuminators, and wear display device and include at least two camera
When, for an illuminator on handle, accelerated according to the linear velocity and line that the historical track of the illuminator obtains the illuminator
Degree；The three-dimensional coordinate of the illuminator is determined according to linear velocity and linear acceleration；The three-dimensional coordinate of the illuminator is distinguished into back projection
Onto each spatial image, the two-dimensional coordinate of the illuminator in each spatial image is obtained respectively；Each space diagram is calculated respectively
The two-dimensional coordinate of each luminous point as in；For a spatial image, by the two-dimensional coordinate of the illuminator of this in spatial image and
The two-dimensional coordinate of each luminous point compares, by the two-dimensional coordinate difference value of the two-dimensional coordinate of luminous point and the illuminator in default threshold
It is worth the luminous point of scope, is defined as the luminous point that there is corresponding relation with the illuminator, by that analogy, it is determined that each spatial image
In the luminous point with the illuminator with corresponding relation；Using to should illuminator at least two luminous point two-dimensional coordinate,
It is determined that the three-dimensional coordinate with the illuminator.The three-dimensional coordinate for each illuminator being assured that by this method on handle.
, it is necessary to ensure in spatial image before the three-dimensional coordinate of illuminator to be distinguished to back projection to each spatial image
Luminous point occur at least two spatial image, exclude do not appear in simultaneously it is luminous at least two spatial image
Point.For example, there are 3 cameras, 3 spatial images are acquired, are spatial image 1, spatial image 2, spatial image 3 respectively, it is right
Luminous point 1 in spatial image 1, does not occur in spatial image 2 and spatial image 3, then just exclude the luminous point 1.
When it is determined that each illuminator three-dimensional coordinate after, using RPnP algorithms or other related algorithms of the prior art,
With reference to the attitude information of the 2nd IMU handles gathered, the six-freedom degree information of handle is determined, the positioning of handle is realized.
(2) illuminator on handle is not blocked, but the historical track or historical track without illuminator are imperfect.This
Embodiment is applied to be provided with the situation of multiple illuminators on handle.
In one embodiment of the invention, in step S150 foundation historical track and spatial image luminous point position
Information, determining the three-dimensional coordinate of illuminator includes：
1. the two-dimensional coordinate of each luminous point in spatial image is calculated；2. the distribution of the luminous point in spatial image
Rule, determines the corresponding relation of the illuminator on the luminous point and handle in spatial image；3. using luminous on correspondence handle
The two-dimensional coordinate of the luminous point of body, determines the three-dimensional coordinate of the illuminator on corresponding handle.Because this method is existing
There is relevant programme in technology, no longer describe in detail herein.
For example, being provided with four illuminators on handle, tetrahedroid is arranged as, the regularity of distribution in spatial image is four
Side shape, then, in spatial image, there are four luminous points for being arranged in similar quadrangle just to correspond to handle, and can be according to four
The relative position information between luminous point in the shape of side, determines the corresponding relation between the illuminator on luminous point and handle.
The present embodiment, it is adaptable to the situation of multiple illuminators is provided with handle, and in order to ensure the accuracy of positioning, this
One or more of embodiment illuminator can not be in same plane.
(3) illuminator on handle is blocked, and has the historical track of illuminator.
When handle is because of noise jamming, or when the reason such as blocking and can not be positioned, the temporary transient interruption of positioning can be caused, in order to
Prevent the appearance of above-mentioned situation from having influence on the usage experience of user, in one embodiment of the invention, step S150 foundation
The positional information of luminous point in historical track and spatial image, determining the three-dimensional coordinate of illuminator includes：
1. determined according to the historical track of illuminator the three-dimensional position at the first moment of illuminator, the 2nd IMU first when
The speed at quarter and the unit direction vector at the 2nd first moment of IMU, the first moment here may be considered in illuminator quilt
At the time of last before blocking；2. according to the 2nd IMU gather attitude information obtain the 2nd second moment of IMU unit direction to
Amount；3. according to the unit direction vector at the second moment and the unit direction vector at the first moment, the 2nd IMU direction change is calculated
Unit vector；4. the three-dimensional coordinate at the moment of illuminator second is calculated according to three-dimensional coordinate formula.
Specifically, the first moment in the present embodiment may be considered the previous moment at the second moment.Above-mentioned three-dimensional seat
Marking formula is：Pt=Pt-1+Vt × l；Wherein, Pt is the three-dimensional coordinate at the moment of illuminator second；Pt-1 is the first of illuminator
The three-dimensional coordinate at moment；Vt is the 2nd IMU direction change unit vector；L is the speed at the 2nd IMU the first moment.
In the present embodiment, when handle can not be positioned, it is possible to use the history three-dimensional track of each illuminator in handle
And the 2nd the methods that are combined of IMU realize the positioning of handle.
Because the three-dimensional coordinate at the second moment of each illuminator is three-dimensional, then the l in above-mentioned formula specifically should be the
The component velocity on three-dimensional at two IMU the first moment.
In order that the positioning for wearing display device and handle is more accurate, in one embodiment of the invention, in step
Acquisition in S120 is worn after the six-freedom degree information of display device, and the method shown in Fig. 1 further comprises：With reference to first
First attitude information of IMU collections, carries out smooth interpolation filtering, by filtered head by the six-freedom degree for wearing display device
Wear the six-freedom degree of the display device six-freedom degree final as display device is worn.
After the six-freedom degree information of step S160 determination handle, the method shown in Fig. 1 further comprises：With reference to
The attitude information of the handle of 2nd IMU collections, carries out smooth interpolation filtering, by filtered handle by the six-freedom degree of handle
Six-freedom degree be used as the final six-freedom degree of handle.
In the present embodiment, the six-freedom degree information of display device will be worn and the six-freedom degree information of handle is carried out
Smooth interpolation filtering can use kalman filter method.Wherein, Kalman filtering is that one kind utilizes linear system state equation,
Data are observed by system input and output, the algorithm of optimal estimation is carried out to system mode, because observation data include system
In noise and interference influence, so optimal estimation is also considered as filtering.In the present embodiment, Kalman filtering can
New measurement error is obtained to go to look into merge past measurement estimation, by after the six-freedom degree filtering for positioning object, can be obtained
More accurate data are obtained, refreshing frequency is effectively improved.
In addition, in the present embodiment, the attitude information gathered with reference to IMU carries out smooth interpolation filtering, also for entering one
Step obtains the six-freedom degree for more accurately positioning object.
It should also be noted that, when carrying out smooth interpolation filtering, what is utilized is not only the attitude information of IMU collections, also
Need the acceleration information gathered using IMU.Because the data of IMU collections not only include above-mentioned attitude information, while can also
Acceleration information is exported, so when carrying out smooth interpolation filtering, as long as being from the IMU extracting data acceleration informations gathered
Can.
In the present invention, it is determined that the three-dimensional coordinate of the illuminator on handle needs the history using illuminator and the 2nd IMU
Track.In one embodiment of the invention, after the six-freedom degree information of the determination handle in step S160, Fig. 1 institutes
The method shown also includes：The six-freedom degree information of handle is preserved, six-freedom degree information is to record illuminator and the 2nd IMU
Historical track.
A kind of structural representation for wearing display device that Fig. 2 provides for one embodiment of the invention.As shown in Fig. 2 should
Wearing display device 200 includes camera 210, the first Inertial Measurement Unit IMU220；Wearing display device 200 also includes：
Image acquisition units 230, the spatial image in space residing for display device is worn for being obtained by camera；
Positioning unit 240, the first attitude information for wearing display device is gathered by the first Inertial Measurement Unit IMU, according to
According to spatial image and the first attitude information, the six-freedom degree information for wearing display device is obtained；
Acquiring unit 250, for obtaining the historical track of illuminator and the 2nd IMU historical track；Wherein, illuminator and
2nd IMU is arranged on handle corresponding with wearing display device；
Receiving unit 260, the attitude information of the handle for receiving the 2nd IMU collections；
Three-dimensional coordinate determining unit 270, for the positional information according to luminous point in historical track and spatial image, it is determined that
The three-dimensional coordinate of illuminator, wherein, luminous point is corresponding with illuminator；
Positioning unit 240, is additionally operable to the attitude information according to three-dimensional coordinate and handle, determines the six-freedom degree letter of handle
Breath.
In one embodiment of the invention, three-dimensional coordinate determining unit 270, specifically for the history rail according to illuminator
Mark obtains the linear velocity and linear acceleration of illuminator；The three-dimensional coordinate of illuminator is determined according to linear velocity and linear acceleration；Will hair
The three-dimensional coordinate back projection of body of light obtains the two-dimensional coordinate of illuminator in spatial image on spatial image, wherein, spatial image
In luminous point it is corresponding with illuminator；Calculate the two-dimensional coordinate of the luminous point in spatial image；Wherein, the hair in spatial image
Luminous point is corresponding with illuminator；The two-dimensional coordinate value of the two-dimensional coordinate value of illuminator and luminous point is compared, by luminous point
Two-dimensional coordinate value and illuminator two-dimensional coordinate value difference value in the luminous point of preset threshold range, be defined as and illuminator
Luminous point with corresponding relation；According to the two-dimensional coordinate of the luminous point with illuminator with corresponding relation, illuminator is determined
Three-dimensional coordinate.
In one embodiment of the invention, three-dimensional coordinate determining unit 270, specifically for being determined to send out according to historical track
Three-dimensional position, the speed at the 2nd IMU the first moment and the unit side at the 2nd first moment of IMU at the first moment of body of light
To vector；The attitude information gathered according to the 2nd IMU obtains the unit direction vector at the 2nd second moment of IMU；During according to second
The unit direction vector at quarter and the unit direction vector at the first moment, calculate the 2nd IMU direction change unit vector；According to three
Dimension coordinate formula calculates the three-dimensional coordinate at the moment of illuminator second.
In one embodiment of the invention, display device 200 is worn to further comprise：Filter unit, for combining
First attitude information of one IMU collections, smooth interpolation filtering is carried out by the six-freedom degree for wearing display device, will be filtered
Wear the six-freedom degree of the display device six-freedom degree final as display device is worn；And, with reference to the 2nd IMU collections
Handle attitude information, the six-freedom degree of handle is subjected to smooth interpolation filtering, by six freedom of filtered handle
Degree is used as the final six-freedom degree of handle.
Historical track recording unit, the six-freedom degree information for preserving handle, six-freedom degree information record lights
The historical track of body and the 2nd IMU.
A kind of structural representation of the alignment system for positioning object that Fig. 3 provides for one embodiment of the invention.Such as Fig. 3 institutes
Show, the alignment system 300 of the positioning object includes：As shown in Figure 2 wears display device 200 and with wearing display device 200
Corresponding handle 320.
Display device 200 is worn in the wired or wireless connection of handle 320, to ensure that wearing display device 200 can get
Corresponding data.
It should be noted that each embodiment for wearing display device and the system shown in Fig. 3 shown in Fig. 2 with shown in Fig. 1
Method each embodiment correspondence it is identical, be described in detail, will not be repeated here above.
In summary, the camera that the present invention is set by wearing on display device, which is obtained, wears space residing for display device
Spatial image；The first attitude information of display device is worn in the first IMU collections by wearing display device, according to space diagram
Picture and the first attitude information, obtain the six-freedom degree information for wearing display device, realize the positioning for wearing display device；And
And, wear display device and obtain the historical track of illuminator and the 2nd IMU historical track；Wherein, illuminator and the 2nd IMU are equal
It is arranged on handle corresponding with wearing display device；Receive the attitude information of the handle of the 2nd IMU collections；According to historical track
With the positional information of luminous point in spatial image, the three-dimensional coordinate of illuminator is determined, wherein, luminous point is corresponding with illuminator；
According to three-dimensional coordinate and the attitude information of handle, the six-freedom degree information of handle is determined, the positioning of handle is realized.It can be seen that, this
The technical scheme of invention is different from camera external in the prior art, is to realize head by the camera worn on display device
The positioning of display device and handle corresponding with wearing display device is worn, camera is moved with the movement for wearing display device
Dynamic, orientation range is wide.That is, the location equipment in technical scheme is provided on positioning object, it is not necessary to
External location equipment is pre-set, using facility, orientation range is wide, can improve Consumer's Experience.
The foregoing is only a specific embodiment of the invention, under the above-mentioned teaching of the present invention, those skilled in the art
Other improvement or deformation can be carried out on the basis of above-described embodiment.It will be understood by those skilled in the art that above-mentioned tool
The purpose of the present invention is simply preferably explained in body description, and protection scope of the present invention should be defined by scope of the claims.
Claims (10)
1. a kind of localization method for positioning object, it is characterised in that methods described includes：
The spatial image in space residing for display device is worn described in the camera acquisition set by wearing on display device；
Believed by the first posture that display device is worn described in the first Inertial Measurement Unit IMU collections for wearing display device
Breath, according to the spatial image and first attitude information, wears the six-freedom degree information of display device described in acquisition；
It is described to wear the historical track that display device obtains the historical track and the 2nd IMU of illuminator；Wherein, the illuminator and
2nd IMU is arranged to be worn on the corresponding handle of display device with described；
It is described to wear the attitude information that display device receives the handle of the 2nd IMU collections；
According to the positional information of luminous point in the historical track and the spatial image, the three-dimensional seat of the illuminator is determined
Mark, wherein, the luminous point is corresponding with the illuminator；
According to the three-dimensional coordinate and the attitude information of the handle, the six-freedom degree information of the handle is determined.
2. the method as described in claim 1, it is characterised in that described to be sent out according in the historical track and the spatial image
The positional information of luminous point, determining the three-dimensional coordinate of the illuminator includes：
The linear velocity and linear acceleration of the illuminator are obtained according to the historical track of the illuminator；
The three-dimensional coordinate of the illuminator is determined according to the linear velocity and the linear acceleration；
By in the three-dimensional coordinate back projection of the illuminator to the spatial image, illuminator described in the spatial image is obtained
Two-dimensional coordinate；
Calculate the two-dimensional coordinate of the luminous point in the spatial image；Wherein, the luminous point in the spatial image and the hair
Body of light is corresponding；The two-dimensional coordinate value of the two-dimensional coordinate value of the illuminator and the luminous point is compared, by the hair
The difference value of the two-dimensional coordinate value of the two-dimensional coordinate value of luminous point and the illuminator preset threshold range the luminous point, really
It is set to the luminous point that there is corresponding relation with the illuminator；
According to the two-dimensional coordinate of the luminous point with the illuminator with corresponding relation, the three-dimensional seat of the illuminator is determined
Mark.
3. the method as described in claim 1, it is characterised in that described to be sent out according in the historical track and the spatial image
The positional information of luminous point, determining the three-dimensional coordinate of the illuminator includes：
Three-dimensional position, first moment of the 2nd IMU at the first moment of the illuminator are determined according to the historical track
Speed and the unit direction vector at the first moment of the 2nd IMU；
The attitude information gathered according to the 2nd IMU obtains the unit direction vector at the second moment of the 2nd IMU；
According to the unit direction vector at second moment and the unit direction vector at first moment, described second is calculated
IMU direction change unit vector；
The three-dimensional coordinate at the moment of illuminator second is calculated according to three-dimensional coordinate formula.
4. the method as described in claim 1, it is characterised in that
After the six-freedom degree information of display device is worn described in the acquisition, methods described further comprises：
The first attitude information gathered with reference to the first IMU, the six-freedom degree for wearing display device is carried out smooth
Filtering interpolation, regard the filtered six-freedom degree for wearing display device as wear display device finally six
The free degree；
After the six-freedom degree information for determining the handle, methods described further comprises：
With reference to the attitude information of the 2nd IMU handles gathered, the six-freedom degree of the handle is smoothly inserted
Value filtering, regard the six-freedom degree of the filtered handle as the final six-freedom degree of the handle.
5. the method as described in claim 1, it is characterised in that the six-freedom degree information for determining the handle it
Afterwards, methods described also includes：
The six-freedom degree information of the handle is preserved, the six-freedom degree information is to record the illuminator and described
Two IMU historical track.
6. one kind wears display device, it is characterised in that the display device of wearing includes camera, the first Inertial Measurement Unit
IMU；The display device of wearing also includes：
Image acquisition units, for the spatial image by wearing space residing for display device described in camera acquisition；
Positioning unit, by wearing the first attitude information of display device described in the first Inertial Measurement Unit IMU collections, according to
According to the spatial image and first attitude information, the six-freedom degree information of display device is worn described in acquisition；
Acquiring unit, for obtaining the historical track of illuminator and the 2nd IMU historical track；Wherein, the illuminator and institute
State the 2nd IMU and be arranged at and worn with described on the corresponding handle of display device；
Receiving unit, the attitude information of the handle for receiving the 2nd IMU collections；
Three-dimensional coordinate determining unit, for the positional information according to luminous point in the historical track and the spatial image, really
The three-dimensional coordinate of the fixed illuminator, wherein, the luminous point is corresponding with the illuminator；
The positioning unit, is additionally operable to the attitude information according to the three-dimensional coordinate and the handle, determines the six of the handle
Individual free degree information.
7. equipment as claimed in claim 6, it is characterised in that
The three-dimensional coordinate determining unit, the linear speed specifically for obtaining the illuminator according to the historical track of the illuminator
Degree and linear acceleration；The three-dimensional coordinate of the illuminator is determined according to the linear velocity and the linear acceleration；Will be described luminous
The three-dimensional coordinate back projection of body obtains the two-dimensional coordinate of illuminator described in the spatial image on the spatial image, its
In, the luminous point in the spatial image is corresponding with the illuminator；Calculate the two dimension of the luminous point in the spatial image
Coordinate；Wherein, the luminous point in the spatial image is corresponding with the illuminator；By the two-dimensional coordinate value of the illuminator and
The two-dimensional coordinate value of the luminous point is compared, by the two-dimensional coordinate value of the luminous point and the two-dimensional coordinate of the illuminator
The difference value of value is defined as the luminous point for having corresponding relation with the illuminator in the luminous point of preset threshold range；
According to the two-dimensional coordinate of the luminous point with the illuminator with corresponding relation, the three-dimensional coordinate of the illuminator is determined.
8. equipment as claimed in claim 6, it is characterised in that
The three-dimensional coordinate determining unit, three of the first moment specifically for determining the illuminator according to the historical track
Tie up position, the speed at the first moment of the 2nd IMU and the unit direction vector at the first moment of the 2nd IMU；According to
The attitude information of the 2nd IMU collections obtains the unit direction vector at the second moment of the 2nd IMU；During according to described second
The unit direction vector at quarter and the unit direction vector at first moment, calculate the direction change unit of the 2nd IMU to
Amount；The three-dimensional coordinate at the moment of illuminator second is calculated according to three-dimensional coordinate formula.
9. equipment as claimed in claim 6, it is characterised in that the equipment further comprises：
Filter unit, for the first attitude information gathered with reference to the first IMU, by wear display device six from
Smooth interpolation filtering is carried out by degree, the filtered six-freedom degree for wearing display device is worn display and set as described
Standby final six-freedom degree；And, with reference to the attitude information of the 2nd IMU handles gathered, by the handle
Six-freedom degree carries out smooth interpolation filtering, using the six-freedom degree of the filtered handle as the handle it is final six
The individual free degree；
Historical track recording unit, the six-freedom degree information for preserving the handle, the six-freedom degree information record
The historical track of the illuminator and the 2nd IMU.
10. a kind of alignment system for positioning object, it is characterised in that the system includes：As described in claim any one of 6-9
Wear display device and wear the corresponding handle of display device with described；
Display device is worn described in the wired or wireless connection of handle.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710370301.7A CN107153369B (en) | 2017-05-23 | 2017-05-23 | It is a kind of to position the localization method of object, system and wear display equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710370301.7A CN107153369B (en) | 2017-05-23 | 2017-05-23 | It is a kind of to position the localization method of object, system and wear display equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107153369A true CN107153369A (en) | 2017-09-12 |
| CN107153369B CN107153369B (en) | 2019-08-20 |
Family
ID=59793315
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710370301.7A Active CN107153369B (en) | 2017-05-23 | 2017-05-23 | It is a kind of to position the localization method of object, system and wear display equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107153369B (en) |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107490866A (en) * | 2017-09-20 | 2017-12-19 | 中国航空工业集团公司洛阳电光设备研究所 | A kind of method of the wearable display system of pilot and its positioning head posture |
| CN108022302A (en) * | 2017-12-01 | 2018-05-11 | 深圳市天界幻境科技有限公司 | A kind of sterically defined AR 3 d display devices of Inside-Out |
| CN108762527A (en) * | 2018-05-23 | 2018-11-06 | 重庆创通联达智能技术有限公司 | A kind of recognition positioning method and device |
| CN109991740A (en) * | 2017-12-30 | 2019-07-09 | 深圳多哚新技术有限责任公司 | One kind is frivolous to wear display equipment |
| CN110378339A (en) * | 2019-07-22 | 2019-10-25 | 青岛海信电器股份有限公司 | The location determining method and device of illuminator |
| CN110572635A (en) * | 2019-08-28 | 2019-12-13 | 重庆爱奇艺智能科技有限公司 | Method, equipment and system for tracking and positioning handheld control equipment |
| CN111061363A (en) * | 2019-11-21 | 2020-04-24 | 青岛小鸟看看科技有限公司 | Virtual reality system |
| CN111174683A (en) * | 2020-01-07 | 2020-05-19 | 青岛小鸟看看科技有限公司 | Handle positioning method, head-mounted display device and storage medium |
| CN111420391A (en) * | 2020-03-04 | 2020-07-17 | 青岛小鸟看看科技有限公司 | Head-mounted display system and space positioning method thereof |
| CN112286343A (en) * | 2020-09-16 | 2021-01-29 | 青岛小鸟看看科技有限公司 | Positioning tracking method, platform and head-mounted display system |
| CN112822480A (en) * | 2020-12-31 | 2021-05-18 | 青岛小鸟看看科技有限公司 | VR system and positioning tracking method thereof |
| CN113012232A (en) * | 2021-03-01 | 2021-06-22 | 重庆工程职业技术学院 | Positioning system and method used in mine |
| CN116257134A (en) * | 2023-02-09 | 2023-06-13 | 上海鱼微阿科技有限公司 | Method, device, equipment and medium for tracking handle helmet under non-inertial reference system |
| WO2024041202A1 (en) * | 2022-08-22 | 2024-02-29 | 华为技术有限公司 | Image processing method, calibration system, and related equipment |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6474159B1 (en) * | 2000-04-21 | 2002-11-05 | Intersense, Inc. | Motion-tracking |
| CN105487670A (en) * | 2015-12-18 | 2016-04-13 | 北京诺亦腾科技有限公司 | Method for positioning physical prop of virtual reality system and virtual reality system |
| CN105528079A (en) * | 2015-12-18 | 2016-04-27 | 北京诺亦腾科技有限公司 | Method for locating physical prop of virtual reality system and virtual reality system |
| CN105607737A (en) * | 2015-12-18 | 2016-05-25 | 北京诺亦腾科技有限公司 | Positioning method for physical prop of virtual reality system, and virtual reality system |
| CN206096621U (en) * | 2016-07-30 | 2017-04-12 | 广州数娱信息科技有限公司 | Enhancement mode virtual reality perception equipment |
-
2017
- 2017-05-23 CN CN201710370301.7A patent/CN107153369B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6474159B1 (en) * | 2000-04-21 | 2002-11-05 | Intersense, Inc. | Motion-tracking |
| CN105487670A (en) * | 2015-12-18 | 2016-04-13 | 北京诺亦腾科技有限公司 | Method for positioning physical prop of virtual reality system and virtual reality system |
| CN105528079A (en) * | 2015-12-18 | 2016-04-27 | 北京诺亦腾科技有限公司 | Method for locating physical prop of virtual reality system and virtual reality system |
| CN105607737A (en) * | 2015-12-18 | 2016-05-25 | 北京诺亦腾科技有限公司 | Positioning method for physical prop of virtual reality system, and virtual reality system |
| CN206096621U (en) * | 2016-07-30 | 2017-04-12 | 广州数娱信息科技有限公司 | Enhancement mode virtual reality perception equipment |
Cited By (24)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107490866A (en) * | 2017-09-20 | 2017-12-19 | 中国航空工业集团公司洛阳电光设备研究所 | A kind of method of the wearable display system of pilot and its positioning head posture |
| CN107490866B (en) * | 2017-09-20 | 2019-11-29 | 中国航空工业集团公司洛阳电光设备研究所 | A kind of method of the wearable display system of pilot and its positioning head posture |
| CN108022302A (en) * | 2017-12-01 | 2018-05-11 | 深圳市天界幻境科技有限公司 | A kind of sterically defined AR 3 d display devices of Inside-Out |
| CN108022302B (en) * | 2017-12-01 | 2021-06-29 | 深圳市天界幻境科技有限公司 | Stereo display device of Inside-Out space orientation's AR |
| CN109991740A (en) * | 2017-12-30 | 2019-07-09 | 深圳多哚新技术有限责任公司 | One kind is frivolous to wear display equipment |
| CN108762527A (en) * | 2018-05-23 | 2018-11-06 | 重庆创通联达智能技术有限公司 | A kind of recognition positioning method and device |
| CN110378339A (en) * | 2019-07-22 | 2019-10-25 | 青岛海信电器股份有限公司 | The location determining method and device of illuminator |
| CN110378339B (en) * | 2019-07-22 | 2021-08-17 | 海信视像科技股份有限公司 | Method and device for determining position of luminous body |
| CN110572635A (en) * | 2019-08-28 | 2019-12-13 | 重庆爱奇艺智能科技有限公司 | Method, equipment and system for tracking and positioning handheld control equipment |
| CN111061363A (en) * | 2019-11-21 | 2020-04-24 | 青岛小鸟看看科技有限公司 | Virtual reality system |
| CN111174683A (en) * | 2020-01-07 | 2020-05-19 | 青岛小鸟看看科技有限公司 | Handle positioning method, head-mounted display device and storage medium |
| CN111174683B (en) * | 2020-01-07 | 2021-11-30 | 青岛小鸟看看科技有限公司 | Handle positioning method, head-mounted display device and storage medium |
| CN111420391A (en) * | 2020-03-04 | 2020-07-17 | 青岛小鸟看看科技有限公司 | Head-mounted display system and space positioning method thereof |
| US11625841B2 (en) | 2020-09-16 | 2023-04-11 | Qingdao Pico Technology Co., Ltd. | Localization and tracking method and platform, head-mounted display system, and computer-readable storage medium |
| WO2022057579A1 (en) * | 2020-09-16 | 2022-03-24 | 青岛小鸟看看科技有限公司 | Positioning and tracking method and platform, head-mounted display system, and computer-readable storage medium |
| CN112286343A (en) * | 2020-09-16 | 2021-01-29 | 青岛小鸟看看科技有限公司 | Positioning tracking method, platform and head-mounted display system |
| CN112822480A (en) * | 2020-12-31 | 2021-05-18 | 青岛小鸟看看科技有限公司 | VR system and positioning tracking method thereof |
| CN112822480B (en) * | 2020-12-31 | 2022-05-17 | 青岛小鸟看看科技有限公司 | VR system and positioning tracking method thereof |
| US11819755B2 (en) | 2020-12-31 | 2023-11-21 | Qingdao Pico Technology Co., Ltd. | VR system and positioning and tracking method of VR system |
| CN113012232A (en) * | 2021-03-01 | 2021-06-22 | 重庆工程职业技术学院 | Positioning system and method used in mine |
| CN113012232B (en) * | 2021-03-01 | 2023-10-27 | 重庆工程职业技术学院 | Positioning system and method for mine |
| WO2024041202A1 (en) * | 2022-08-22 | 2024-02-29 | 华为技术有限公司 | Image processing method, calibration system, and related equipment |
| CN116257134A (en) * | 2023-02-09 | 2023-06-13 | 上海鱼微阿科技有限公司 | Method, device, equipment and medium for tracking handle helmet under non-inertial reference system |
| CN116257134B (en) * | 2023-02-09 | 2024-04-09 | 玩出梦想(上海)科技有限公司 | Method, device, equipment and medium for tracking handle helmet under non-inertial reference system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107153369B (en) | 2019-08-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107153369B (en) | It is a kind of to position the localization method of object, system and wear display equipment | |
| US11481024B2 (en) | Six degree of freedom tracking with scale recovery and obstacle avoidance | |
| US10068344B2 (en) | Method and system for 3D capture based on structure from motion with simplified pose detection | |
| JP4708752B2 (en) | Information processing method and apparatus | |
| CN102448681B (en) | Operating space presentation device, operating space presentation method, and program | |
| CN106525003B (en) | A kind of attitude measurement method based on binocular vision | |
| CN109166149A (en) | A kind of positioning and three-dimensional wire-frame method for reconstructing and system of fusion binocular camera and IMU | |
| CN108733039A (en) | The method and apparatus of navigator fix in a kind of robot chamber | |
| Dorfmüller | Robust tracking for augmented reality using retroreflective markers | |
| TWI520576B (en) | Method and system for converting 2d images to 3d images and computer-readable medium | |
| CN104596502A (en) | Object posture measuring method based on CAD model and monocular vision | |
| US10110822B2 (en) | Method for tracking at least one object and method for replacing at least one object by a virtual object in a moving image signal recorded by a camera | |
| JP2017532695A (en) | Method and system for scanning an object using an RGB-D sensor | |
| CN108074251A (en) | Mobile Robotics Navigation control method based on monocular vision | |
| CN110533719A (en) | Augmented reality localization method and device based on environmental visual Feature point recognition technology | |
| CN112907633B (en) | Dynamic feature point identification method and application thereof | |
| JP6487545B2 (en) | Recognition calculation device, recognition calculation method, and recognition calculation program | |
| Davidson et al. | Monocular vision-based range estimation supported by proprioceptive motion | |
| CA2375990A1 (en) | Three-dimensional model forming apparatus | |
| CN208314856U (en) | A kind of system for the detection of monocular airborne target | |
| Hui et al. | Visual SLAM based on EKF filtering algorithm from omnidirectional camera | |
| Kundra et al. | Improving orientation estimation in mobiles with built-in camera | |
| Moemeni et al. | Inertial-visual pose tracking using optical flow-aided particle filtering | |
| Benndorf et al. | NAND-measure: An Android App for Marker-based Spatial Measurement. | |
| Branca et al. | Mobile robot navigation using egomotion estimates |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |