CN111935396A - 6DoF data processing method and device of VR (virtual reality) all-in-one machine - Google Patents

6DoF data processing method and device of VR (virtual reality) all-in-one machine Download PDF

Info

Publication number
CN111935396A
CN111935396A CN202010625632.2A CN202010625632A CN111935396A CN 111935396 A CN111935396 A CN 111935396A CN 202010625632 A CN202010625632 A CN 202010625632A CN 111935396 A CN111935396 A CN 111935396A
Authority
CN
China
Prior art keywords
6dof
data
time
smoothing
machine
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010625632.2A
Other languages
Chinese (zh)
Inventor
吴涛
李汉振
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Qingdao Xiaoniao Kankan Technology Co Ltd
Original Assignee
Qingdao Xiaoniao Kankan Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Qingdao Xiaoniao Kankan Technology Co Ltd filed Critical Qingdao Xiaoniao Kankan Technology Co Ltd
Priority to CN202010625632.2A priority Critical patent/CN111935396A/en
Publication of CN111935396A publication Critical patent/CN111935396A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/695—Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/04—Synchronising

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Length Measuring Devices With Unspecified Measuring Means (AREA)

Abstract

The invention discloses a 6DoF data processing method and a device of a VR (virtual reality) all-in-one machine, wherein the method comprises the following steps: acquiring original 6DoF data of the VR all-in-one machine in real time; performing first smoothing filtering processing on the original 6DoF data to obtain 6DoF smoothing filtering data; constructing an extended Kalman filter; and performing secondary smoothing filtering processing on the 6DoF smoothing filtering data by using the extended Kalman filter to obtain 6DoF tracking data of the VR all-in-one machine. The invention improves the accuracy of positioning and tracking through 6DoF data information and further improves the stability of positioning and tracking of the VR integrated machine.

Description

6DoF data processing method and device of VR (virtual reality) all-in-one machine
Technical Field
The invention relates to the technical field of VR (virtual reality), in particular to a 6DoF (DoF) data processing method and device of a VR (virtual reality) all-in-one machine.
Background
At present, in the technical field of VR, the problem of 6dof (hierarchy of free) of VR headsets is generally solved through an inside-out tracking technology. Compare from the beginning to the end tracking technology, do not need the user to set up a plurality of tracking cameras or VR all-in-one machine in the scene that it used and track perception sensor, only need through the built-in tracking camera of VR all-in-one machine just can solve head-mounted end 6DoF information, need not the user additionally to add too much sensor equipment and build the work load of environment, only need the user wear the VR head just can a relative real environment space, freely think freely in virtual scene has experienced.
However, the actual usage scenario is complex and changeable, for example, there are many lights in some user environment scenarios, there are many reflective objects in some user environment scenarios, or there are not enough abundant object texture information in some user environment scenarios, which may result in low accuracy of the head-mounted end 6DoF data of the VR all-in-one machine, which may result in the occurrence of phenomena of inaccurate and unstable positioning tracking, which may result in the phenomenon of shaking or instantaneous shaking of the VR virtual scenario, especially when the user is walking or rotating at a slow speed, the shaking or instantaneous shaking of the VR virtual scenario may be obvious, the user may have phenomena of vertigo, etc., which may easily affect the immersion of the user.
Disclosure of Invention
In view of the above problems, an object of the present invention is to provide a method and an apparatus for processing 6DoF data of a VR all-in-one machine, so as to solve the problem that location tracking is not accurate and unstable enough according to 6DoF data information.
In order to achieve the purpose, the invention adopts the following technical scheme:
one aspect of the present invention provides a 6DoF data processing method for a VR all-in-one machine, including:
acquiring original 6DoF data of the VR all-in-one machine in real time;
performing first smoothing filtering processing on the original 6DoF data to obtain 6DoF smoothing filtering data;
constructing an extended Kalman filter;
performing second smoothing filtering processing on the 6DoF smoothing filtering data by using the extended Kalman filter to obtain 6DoF tracking data of the VR all-in-one machine;
wherein, the state transition model of the extended Kalman filter is as follows:
Figure BDA0002564624540000021
wherein the content of the first and second substances,
Figure BDA0002564624540000022
and
Figure BDA0002564624540000023
the displacement components in the x-axis direction after the optimization iteration of the 6DoF smooth filtering data at the k-th time and the k-1 time respectively,
Figure BDA0002564624540000024
and
Figure BDA0002564624540000025
respectively optimizing the displacement component in the y-axis direction after iteration on the 6DoF smooth filtering data at the k-th time and the k-1 time,
Figure BDA0002564624540000026
and
Figure BDA0002564624540000027
respectively optimizing and iterating the 6DoF smooth filtering data at the k moment and the k-1 moment to obtain displacement components in the z-axis direction;
Figure BDA0002564624540000028
and
Figure BDA0002564624540000029
the velocity displacement component in the x-axis direction in the 6DoF smoothed filtered data at the k-th time and k-1 time respectively,
Figure BDA00025646245400000210
and
Figure BDA00025646245400000211
the velocity displacement component in the y-axis direction in the 6DoF smoothed filtered data at the k-th time and k-1 time respectively,
Figure BDA00025646245400000212
and
Figure BDA00025646245400000213
velocity displacement components in the z-axis direction in 6DoF smooth filtering data at the k-th moment and the k-1 moment respectively;
Figure BDA00025646245400000214
respectively representing the motion data of the gravity acceleration sensor in the x axis, the y axis and the z axis after the gravity direction is removed at the kth moment;
Δ T represents a time difference between the time k and the time k-1;
the observation model of the extended kalman filter is shown as follows:
Figure BDA00025646245400000215
wherein the content of the first and second substances,
Figure BDA00025646245400000216
respectively generating position data in the x direction, the y direction and the z direction according to the iterative optimization of the extended Kalman filter;
Figure BDA00025646245400000217
the data are position shift data in the x, y, z three-axis directions at the time k in the 6DoF smoothing filtered data, respectively.
Preferably, the step of constructing the extended kalman filter further comprises:
setting a process noise variance matrix as shown in the following formula:
Figure BDA0002564624540000031
wherein pnoiseov represents a process noise variance matrix, p _ error represents a displacement noise error, and v _ error represents a velocity noise error.
Preferably, the displacement noise error and the velocity noise error in the process noise are adaptively adjusted by the following model:
p_error＝(1-x)4*10，x∈[0.01,0.1]
v_error＝(1-x)5*10，x∈[0.01,0.1]
wherein x represents the smooth confidence of the position movement data in the x, y and z axis directions of the k time in the 6DoF smooth filtering data of the frame VR all-in-one machine at the current time.
Preferably, the step of constructing the extended kalman filter further comprises:
setting a measurement noise variance matrix as shown in the following formula:
Figure BDA0002564624540000032
where MNoiseCov represents the measurement noise variance matrix and M _ error represents the measurement noise error.
Preferably, the original 6DoF data is subjected to a smoothing filtering process according to the following formula:
St＝(Ft+1+S1 t)/2
wherein S istRepresents the 6DoF smoothing filter data obtained by the first smoothing filter process, S1 t6DoF smoothing filter data representing the current time frame, Ft+1Represents the smoothed value of the original 6DoF data for the next time instant frame.
Preferably, the smoothed value of the original 6DoF data of the future time instant frame is obtained by the following formula:
Ft+m＝bt+ct*m+dt*m2/2
wherein, bt＝3*S1 t-3*S2 t+S3 t；
ct＝a*((6–5*a)*S1 t-(10–8*a)*S2 t+(4–3*a)*S3 t)/(2*(1-a)2)；
dt＝a2*(S1 t-2*S2 t+S3 t)/(1-a)2；
S1 t＝a*Dt+(1-a)*S1 t-1；
S2 t＝a*S1 t+(1-a)*S2 t-1；
S3 t＝a*S2 t+(1-a)*S3 t-1；
Ft+mA smoothed value of the original 6DoF data representing a frame at a future time, t representing the current time, S1 t、S2 t、S3 t6DoF smoothing data representing the current time frame, S1 t-1、S2 t-1、S3 t-16DoF smooth filtering data representing a previous time frame; a represents a coefficient, DtRepresenting the original 6DoF data.
Another aspect of the present invention provides a 6DoF data processing apparatus of a VR all-in-one machine, including:
the data acquisition module is used for acquiring original 6DoF data of the VR all-in-one machine in real time;
a first smoothing module, configured to perform first smoothing filtering on the original 6DoF data to obtain 6DoF smoothing filtering data;
the filter construction module is used for constructing an extended Kalman filter;
the secondary smoothing module is used for performing secondary smoothing filtering processing on the 6DoF smoothing filtering data by using the extended Kalman filter to obtain 6DoF tracking data of the VR all-in-one machine;
wherein, the state transition model of the extended Kalman filter is as follows:
Figure BDA0002564624540000041
wherein，
Figure BDA0002564624540000042
And
Figure BDA0002564624540000043
the displacement components in the x-axis direction after the optimization iteration of the 6DoF smooth filtering data at the k-th time and the k-1 time respectively,
Figure BDA0002564624540000044
and
Figure BDA0002564624540000045
respectively optimizing the displacement component in the y-axis direction after iteration on the 6DoF smooth filtering data at the k-th time and the k-1 time,
Figure BDA0002564624540000046
and
Figure BDA0002564624540000047
respectively optimizing and iterating the 6DoF smooth filtering data at the k moment and the k-1 moment to obtain displacement components in the z-axis direction;
Figure BDA0002564624540000048
and
Figure BDA0002564624540000049
the velocity displacement component in the x-axis direction in the 6DoF smoothed filtered data at the k-th time and k-1 time respectively,
Figure BDA00025646245400000410
and
Figure BDA00025646245400000411
the velocity displacement component in the y-axis direction in the 6DoF smoothed filtered data at the k-th time and k-1 time respectively,
Figure BDA00025646245400000412
and
Figure BDA00025646245400000413
velocity displacement components in the z-axis direction in 6DoF smooth filtering data at the k-th moment and the k-1 moment respectively;
Figure BDA00025646245400000414
respectively representing the motion data of the gravity acceleration sensor in the x axis, the y axis and the z axis after the gravity direction is removed at the kth moment;
Δ T represents a time difference between the time k and the time k-1;
the observation model of the extended kalman filter is shown as follows:
Figure BDA0002564624540000051
wherein the content of the first and second substances,
Figure BDA0002564624540000052
respectively generating position data in the x direction, the y direction and the z direction according to the iterative optimization of the extended Kalman filter;
Figure BDA0002564624540000053
the data are position shift data in the x, y, z three-axis directions at the time k in the 6DoF smoothing filtered data, respectively.
Preferably, when the filter construction module constructs the extended kalman filter, a process noise variance matrix is set as shown in the following formula:
Figure BDA0002564624540000054
wherein pnoiseov represents a process noise variance matrix, p _ error represents a displacement noise error, and v _ error represents a velocity noise error.
Preferably, when the filter construction module constructs the extended kalman filter, a measurement noise variance matrix is set as shown in the following formula:
Figure BDA0002564624540000055
where MNoiseCov represents the measurement noise variance matrix and M _ error represents the measurement noise error.
Compared with the prior art, the invention has the following advantages and beneficial effects:
according to the invention, the 6DoF data of the VR all-in-one machine are optimized through the extended Kalman filter, so that the accuracy of positioning and tracking through the 6DoF data information is improved, and the stability of positioning and tracking of the VR all-in-one machine is further improved.
Drawings
Fig. 1 is a schematic flow diagram of a 6DoF data processing method of a VR all-in-one machine according to the present invention.
Detailed Description
The embodiments of the present invention will be described below with reference to the accompanying drawings. Those of ordinary skill in the art will recognize that the described embodiments can be modified in various different ways, or combinations thereof, without departing from the spirit and scope of the present invention. Accordingly, the drawings and description are illustrative in nature and not intended to limit the scope of the claims. Furthermore, in the present description, the drawings are not to scale and like reference numerals refer to like parts.
Fig. 1 is a schematic flow diagram of a 6DoF data processing method of a VR all-in-one machine according to the present invention, and as shown in fig. 1, the 6DoF data processing method of the VR all-in-one machine according to the present invention includes:
step S1, acquiring original 6DoF data of the VR all-in-one machine in real time, wherein the original 6DoF data comprise 6DoF data of each time frame; when a user carries out VR scene interaction, namely the user wears a VR head-mounted all-in-one machine and freely moves and rotates in a relative space, the space environment information of the user at the moment can be captured in real time through one or two or more tracking cameras built in the VR head-mounted all-in-one machine, and the 6DoF information (the rotation information and the position movement information of the user) of the VR head-mounted all-in-one machine of the user relative to a three-dimensional environment space is calculated in real time through a computer vision technology, a multi-sensor fusion technology and an image processing technology and is used as original 6DoF data;
step S2, performing first smoothing filtering processing on the original 6DoF data to obtain 6DoF smoothing filtering data;
step S3, constructing an extended Kalman filter;
and step S4, performing secondary smoothing filtering processing on the 6DoF smoothing filtering data by using the extended Kalman filter to obtain 6DoF tracking data of the VR all-in-one machine.
When the first smoothing filtering is carried out, the original 6DoF data of each frame needs to be processed, and the track information delay of the 6DoF data caused by the smoothing filtering is not considered. Preferably, the smoothed value of the original 6DoF data of the future time instant frame is obtained by the following formula:
Ft+m＝bt+ct*m+dt*m2/2
wherein, bt＝3*S1 t-3*S2 t+S3 t；
ct＝a*((6–5*a)*S1 t-(10–8*a)*S2 t+(4–3*a)*S3 t)/(2*(1-a)2)；
dt＝a2*(S1 t-2*S2 t+S3 t)/(1-a)2；
S1 t＝a*Dt+(1-a)*S1 t-1；
S2 t＝a*S1 t+(1-a)*S2 t-1；
S3 t＝a*S2 t+(1-a)*S3 t-1；
Ft+mA smoothed value of the original 6DoF data representing a frame at a future time, t representing the current time, S1 t、S2 t、S3 t6DoF smoothing data representing the current time frame, S1 t-1、S2 t-1、S3 t-16DoF smooth filtering data representing a previous time frame; the superscripts 1,2,3 do not denote any physical meaning, but merely denote the order of processing iterations for 6DoF smooth-filtered data, a denotes the coefficient, DtRepresenting the original 6DoF data.
Further, taking m as 1, i.e. F, in consideration of the influence of the localization tracking precision and the smooth delayt+mRepresents the smoothed value of the original 6DoF data for the next time instant frame. Performing first smoothing filtering processing on the original 6DoF data according to the formula:
St＝(Ft+1+S1 t)/2
wherein S istRepresents the 6DoF smoothing filter data obtained by the first smoothing filter process, S1 t6DoF smoothing filter data representing the current time frame, Ft+1Represents the smoothed value of the original 6DoF data for the next time instant frame.
It should be noted that, the first time frame of the VR headset operation does not perform any processing, that is, the data S after the first smoothing filtering processing is performed on the original 6DoF data of the first time framet＝Dt(ii) a Through the data stability observation to the VR head-mounted all-in-one machine 6DoF localization tracking system, take a to be 0.372.
As can be seen from the inside-out position tracking system, the jitter of the position movement of the three axes X, Y and Z in the 6DoF data is large relative to the jitter of the rotation angle. In one embodiment, an extended kalman filter is further constructed for position movement data in the directions of three axes of X, Y, and Z of 6DoF data of the VR headset, and the 6DoF smoothing filter data is further subjected to second smoothing filter processing to obtain final 6DoF tracking data.
The state transition model of the extended kalman filter is shown as follows:
Figure BDA0002564624540000071
wherein the content of the first and second substances,
Figure BDA0002564624540000081
and
Figure BDA0002564624540000082
the displacement components in the x-axis direction after the optimization iteration of the 6DoF smooth filtering data at the k-th time and the k-1 time respectively,
Figure BDA0002564624540000083
and
Figure BDA0002564624540000084
respectively optimizing the displacement component in the y-axis direction after iteration on the 6DoF smooth filtering data at the k-th time and the k-1 time,
Figure BDA0002564624540000085
and
Figure BDA0002564624540000086
respectively optimizing and iterating the 6DoF smooth filtering data at the k moment and the k-1 moment to obtain displacement components in the z-axis direction; the displacement component in the direction of the three axes x, y, z at the first moment is equal to S1 t；
Figure BDA0002564624540000087
And
Figure BDA0002564624540000088
the velocity displacement component in the x-axis direction in the 6DoF smoothed filtered data at the k-th time and k-1 time respectively,
Figure BDA0002564624540000089
and
Figure BDA00025646245400000810
the velocity displacement component in the y-axis direction in the 6DoF smoothed filtered data at the k-th time and k-1 time respectively,
Figure BDA00025646245400000811
and
Figure BDA00025646245400000812
velocity displacement components in the z-axis direction in 6DoF smooth filtering data at the k-th moment and the k-1 moment respectively; the velocity displacement component in the direction of the three axes of x, y and z at the first moment is 0;
Figure BDA00025646245400000813
respectively representing the motion data of the gravity acceleration sensor in the x axis, the y axis and the z axis after the gravity direction is removed at the kth moment;
Δ T represents a time difference between the time k and the time k-1; preferably, 6DoF smooth filtered data S is obtainedtThe frame rate is 500Hz, i.e. the time difference Δ T between the k-th instant and the k-1 st instant is 2 ms.
The observation model of the extended kalman filter is shown as follows:
Figure BDA00025646245400000814
wherein the content of the first and second substances,
Figure BDA00025646245400000815
respectively generating position data in the x direction, the y direction and the z direction according to the iterative optimization of the extended Kalman filter;
Figure BDA00025646245400000816
the data are position shift data in the x, y, z three-axis directions at the time k in the 6DoF smoothing filtered data, respectively.
Further, the step of constructing the extended kalman filter further comprises setting a process noise variance matrix as shown in the following formula:
Figure BDA00025646245400000817
wherein pnoiseov represents a process noise variance matrix, p _ error represents a displacement noise error, and v _ error represents a velocity noise error.
Setting the process noise variance matrix as a diagonal matrix, wherein according to the noise attribute, displacement noise errors in the directions of the x axis, the y axis and the z axis of the VR head-mounted all-in-one machine are equal, and speed noise errors in the directions of the x axis, the y axis and the z axis are equal; known by a 6DoF positioning and tracking system of the VR head-mounted all-in-one machine, the smaller the movement speed of the VR head-mounted all-in-one machine is, the larger the process noise of the position translation and speed in the directions of the three axes of x, y and z is, the larger the movement speed of the VR head-mounted all-in-one machine is, and the smaller the process noise of the position translation and speed in the directions of the three axes of x, y and z is. Therefore, the displacement noise error and the speed noise error in the process noise are adaptively adjusted through the following model:
p_error＝(1-x)4*10，x∈[0.01,0.1]
v_error＝(1-x)5*10，x∈[0.01,0.1]
wherein x represents the smooth confidence of the position movement data in the x, y and z axes directions of the k moment in the 6DoF smooth filtering data of the VR all-in-one machine at the current moment, and x is in inverse proportion to the movement speed of the VR all-in-one machine.
In one embodiment, the step of constructing the extended kalman filter further comprises setting a measurement noise variance matrix as shown by:
Figure BDA0002564624540000091
where MNoiseCov represents the measurement noise variance matrix and M _ error represents the measurement noise error.
The measurement noise variance matrix is also a diagonal matrix, and the values in the directions of the three axes x, y and z can be considered to be equal. Preferably, the value M _ error is 2 in consideration of the overall noise evaluation of the VR headset 6DoF localization tracking system. Of course, M _ error may be other values for different position tracking systems.
The invention also provides a 6DoF data processing device of the VR machine, which comprises:
the data acquisition module is used for acquiring original 6DoF data of the VR all-in-one machine in real time;
a first smoothing module, configured to perform first smoothing filtering on the original 6DoF data to obtain 6DoF smoothing filtering data;
the filter construction module is used for constructing an extended Kalman filter;
the secondary smoothing module is used for performing secondary smoothing filtering processing on the 6DoF smoothing filtering data by using the extended Kalman filter to obtain 6DoF tracking data of the VR all-in-one machine;
wherein, the state transition model of the extended Kalman filter is as follows:
Figure BDA0002564624540000101
wherein the content of the first and second substances,
Figure BDA0002564624540000102
and
Figure BDA0002564624540000103
the displacement components in the x-axis direction after the optimization iteration of the 6DoF smooth filtering data at the k-th time and the k-1 time respectively,
Figure BDA0002564624540000104
and
Figure BDA0002564624540000105
respectively optimizing the displacement component in the y-axis direction after iteration on the 6DoF smooth filtering data at the k-th time and the k-1 time,
Figure BDA0002564624540000106
and
Figure BDA0002564624540000107
are respectively the firstOptimizing the displacement component in the z-axis direction after iteration on the 6DoF smooth filtering data at the k moment and the k-1 moment;
Figure BDA0002564624540000108
and
Figure BDA0002564624540000109
the velocity displacement component in the x-axis direction in the 6DoF smoothed filtered data at the k-th time and k-1 time respectively,
Figure BDA00025646245400001010
and
Figure BDA00025646245400001011
the velocity displacement component in the y-axis direction in the 6DoF smoothed filtered data at the k-th time and k-1 time respectively,
Figure BDA00025646245400001012
and
Figure BDA00025646245400001013
velocity displacement components in the z-axis direction in 6DoF smooth filtering data at the k-th moment and the k-1 moment respectively;
Figure BDA00025646245400001014
respectively representing the motion data of the gravity acceleration sensor in the x axis, the y axis and the z axis after the gravity direction is removed at the kth moment;
Δ T represents a time difference between the time k and the time k-1;
the observation model of the extended kalman filter is shown as follows:
Figure BDA00025646245400001015
wherein the content of the first and second substances,
Figure BDA00025646245400001016
respectively generating position data in the x direction, the y direction and the z direction according to the iterative optimization of the extended Kalman filter;
Figure BDA00025646245400001017
the data are position shift data in the x, y, z three-axis directions at the time k in the 6DoF smoothing filtered data, respectively.
In one embodiment, the filter construction module, when constructing the extended kalman filter, sets a process noise variance matrix as shown in the following equation:
Figure BDA00025646245400001018
wherein pnoiseov represents a process noise variance matrix, p _ error represents a displacement noise error, and v _ error represents a velocity noise error.
In one embodiment, the filter construction module, when constructing the extended kalman filter, sets a measurement noise variance matrix as shown in the following equation:
Figure BDA0002564624540000111
where MNoiseCov represents the measurement noise variance matrix and M _ error represents the measurement noise error.
It should be noted that the specific embodiment of the 6DoF data processing apparatus of the VR all-in-one machine of the present invention is substantially the same as the specific embodiment of the 6DoF data processing method of the VR all-in-one machine, and details are not repeated here.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (9)

1. A6 DoF data processing method of a VR (virtual reality) all-in-one machine is characterized by comprising the following steps:
acquiring original 6DoF data of the VR all-in-one machine in real time;
performing first smoothing filtering processing on the original 6DoF data to obtain 6DoF smoothing filtering data;
constructing an extended Kalman filter;
performing second smoothing filtering processing on the 6DoF smoothing filtering data by using the extended Kalman filter to obtain 6DoF tracking data of the VR all-in-one machine;
wherein, the state transition model of the extended Kalman filter is as follows:
Figure FDA0002564624530000011
wherein the content of the first and second substances,
Figure FDA0002564624530000012
and
Figure FDA0002564624530000013
the displacement components in the x-axis direction after the optimization iteration of the 6DoF smooth filtering data at the k-th time and the k-1 time respectively,
Figure FDA0002564624530000014
and
Figure FDA0002564624530000015
respectively optimizing the displacement component in the y-axis direction after iteration on the 6DoF smooth filtering data at the k-th time and the k-1 time,
Figure FDA0002564624530000016
and
Figure FDA0002564624530000017
after optimization iteration of 6DoF smooth filtering data at the k-th time and the k-1 time respectivelyA displacement component in the z-axis direction;
Figure FDA0002564624530000018
and
Figure FDA0002564624530000019
the velocity displacement component in the x-axis direction in the 6DoF smoothed filtered data at the k-th time and k-1 time respectively,
Figure FDA00025646245300000110
and
Figure FDA00025646245300000111
the velocity displacement component in the y-axis direction in the 6DoF smoothed filtered data at the k-th time and k-1 time respectively,
Figure FDA00025646245300000112
and
Figure FDA00025646245300000113
velocity displacement components in the z-axis direction in 6DoF smooth filtering data at the k-th moment and the k-1 moment respectively;
Figure FDA00025646245300000114
respectively representing the motion data of the gravity acceleration sensor in the x axis, the y axis and the z axis after the gravity direction is removed at the kth moment;
Δ T represents a time difference between the time k and the time k-1;
the observation model of the extended kalman filter is shown as follows:
Figure FDA00025646245300000115
wherein the content of the first and second substances,
Figure FDA00025646245300000116
respectively generating position data in the x direction, the y direction and the z direction according to the iterative optimization of the extended Kalman filter;
Figure FDA0002564624530000021
the data are position shift data in the x, y, z three-axis directions at the time k in the 6DoF smoothing filtered data, respectively.
2. The 6DoF data processing method for the VR all-in-one machine of claim 1, wherein the step of constructing the extended kalman filter further comprises:
setting a process noise variance matrix as shown in the following formula:
Figure FDA0002564624530000022
wherein pnoiseov represents a process noise variance matrix, p _ error represents a displacement noise error, and v _ error represents a velocity noise error.
3. The 6DoF data processing method of the VR machine of claim 2, wherein the process-in-noise displacement noise error and the speed noise error are adaptively adjusted through the following models:
p_error＝(1-x)4*10，x∈[0.01，0.1]
v_error＝(1-x)5*10，x∈[0.01，0.1]
wherein x represents the smooth confidence of the position movement data in the x, y and z axis directions of the k time in the 6DoF smooth filtering data of the frame VR all-in-one machine at the current time.
4. The 6DoF data processing method for the VR all-in-one machine of claim 1, wherein the step of constructing the extended kalman filter further comprises:
setting a measurement noise variance matrix as shown in the following formula:
Figure FDA0002564624530000023
where MNoiseCov represents the measurement noise variance matrix and M _ error represents the measurement noise error.
5. The 6DoF data processing method of the VR machine of claim 1, wherein the first smoothing filtering is performed on the original 6DoF data according to the following formula:
St＝(Ft+1+S1 t)/2
wherein S istRepresents the 6DoF smoothing filter data obtained by the first smoothing filter process, S1 t6DoF smoothing filter data representing the current time frame, Ft+1Represents the smoothed value of the original 6DoF data for the next time instant frame.
6. The method of claim 1, wherein the smoothed value of the original 6DoF data for the future time frame is obtained by:
Ft+m＝bt+ct*m+dt*m2/2
wherein, bt＝3*S1 t-3*S2 t+S3 t；
ct＝a*((6–5*a)*S1 t-(10–8*a)*S2 t+(4–3*a)*S3 t)/(2*(1-a)2)；
dt＝a2*(S1 t-2*S2 t+S3 t)/(1-a)2；
S1 t＝a*Dt+(1-a)*S1 t-1；
S2 t＝a*S1 t+(1-a)*S2 t-1；
S3 t＝a*S2 t+(1-a)*S3 t-1；
Ft+mA smoothed value of the original 6DoF data representing a frame at a future time, t representing the current time, S1 t、S2 t、S3 t6DoF smoothing data representing the current time frame, S1 t-1、S2 t-1、S3 t-16DoF smooth filtering data representing a previous time frame; a represents a coefficient, DtRepresenting the original 6DoF data.
7. The utility model provides a 6DoF data processing apparatus of VR all-in-one which characterized in that includes:
the data acquisition module is used for acquiring original 6DoF data of the VR all-in-one machine in real time;
a first smoothing module, configured to perform first smoothing filtering on the original 6DoF data to obtain 6DoF smoothing filtering data;
the filter construction module is used for constructing an extended Kalman filter;
the secondary smoothing module is used for performing secondary smoothing filtering processing on the 6DoF smoothing filtering data by using the extended Kalman filter to obtain 6DoF tracking data of the VR all-in-one machine;
wherein, the state transition model of the extended Kalman filter is as follows:
Figure FDA0002564624530000031
wherein the content of the first and second substances,
Figure FDA0002564624530000032
and
Figure FDA0002564624530000033
the displacement components in the x-axis direction after the optimization iteration of the 6DoF smooth filtering data at the k-th time and the k-1 time respectively,
Figure FDA0002564624530000034
and
Figure FDA0002564624530000035
respectively optimizing the displacement component in the y-axis direction after iteration on the 6DoF smooth filtering data at the k-th time and the k-1 time,
Figure FDA0002564624530000036
and
Figure FDA0002564624530000037
respectively optimizing and iterating the 6DoF smooth filtering data at the k moment and the k-1 moment to obtain displacement components in the z-axis direction;
Figure FDA0002564624530000041
and
Figure FDA0002564624530000042
the velocity displacement component in the x-axis direction in the 6DoF smoothed filtered data at the k-th time and k-1 time respectively,
Figure FDA0002564624530000043
and
Figure FDA0002564624530000044
the velocity displacement component in the y-axis direction in the 6DoF smoothed filtered data at the k-th time and k-1 time respectively,
Figure FDA0002564624530000045
and
Figure FDA0002564624530000046
velocity displacement components in the z-axis direction in 6DoF smooth filtering data at the k-th moment and the k-1 moment respectively;
Figure FDA0002564624530000047
respectively representing the motion data of the gravity acceleration sensor in the x axis, the y axis and the z axis after the gravity direction is removed at the kth moment;
Δ T represents a time difference between the time k and the time k-1;
the observation model of the extended kalman filter is shown as follows:
Figure FDA0002564624530000048
wherein the content of the first and second substances,
Figure FDA0002564624530000049
respectively generating position data in the x direction, the y direction and the z direction according to the iterative optimization of the extended Kalman filter;
Figure FDA00025646245300000410
the data are position shift data in the x, y, z three-axis directions at the time k in the 6DoF smoothing filtered data, respectively.
8. The 6DoF data processing device of the VR machine of claim 7,
when the filter construction module constructs the extended Kalman filter, a process noise variance matrix shown as the following formula is set:
Figure FDA00025646245300000411
wherein pnoiseov represents a process noise variance matrix, p _ error represents a displacement noise error, and v _ error represents a velocity noise error.
9. The 6DoF data processing device of the VR machine of claim 7,
when the filter construction module constructs the extended Kalman filter, a measurement noise variance matrix shown as the following formula is set:
Figure FDA00025646245300000412
where MNoiseCov represents the measurement noise variance matrix and M _ error represents the measurement noise error.
CN202010625632.2A 2020-07-01 2020-07-01 6DoF data processing method and device of VR (virtual reality) all-in-one machine Pending CN111935396A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010625632.2A CN111935396A (en) 2020-07-01 2020-07-01 6DoF data processing method and device of VR (virtual reality) all-in-one machine

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010625632.2A CN111935396A (en) 2020-07-01 2020-07-01 6DoF data processing method and device of VR (virtual reality) all-in-one machine

Publications (1)

Publication Number Publication Date
CN111935396A true CN111935396A (en) 2020-11-13

Family

ID=73317393

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010625632.2A Pending CN111935396A (en) 2020-07-01 2020-07-01 6DoF data processing method and device of VR (virtual reality) all-in-one machine

Country Status (1)

Country Link
CN (1) CN111935396A (en)

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6067340A (en) * 1998-07-06 2000-05-23 Eppstein; Margaret J. Three-dimensional stochastic tomography with upscaling
CN101116101A (en) * 2005-02-04 2008-01-30 佳能株式会社 Position posture measuring method and device
CN105806338A (en) * 2016-03-17 2016-07-27 孙红星 GNSS/INS integrated positioning and directioning algorithm based on three-way Kalman filtering smoother
US20180053056A1 (en) * 2016-08-22 2018-02-22 Magic Leap, Inc. Augmented reality display device with deep learning sensors
CN108170297A (en) * 2017-09-11 2018-06-15 南京睿悦信息技术有限公司 Real-time six degree of freedom VR/AR/MR equipment localization methods
CN108267715A (en) * 2017-12-26 2018-07-10 青岛小鸟看看科技有限公司 Localization method and device, the virtual reality device and system of external equipment
CN108346432A (en) * 2017-01-25 2018-07-31 北京三星通信技术研究有限公司 The processing method and relevant device of Virtual Reality audio
CN109358745A (en) * 2018-08-31 2019-02-19 青岛小鸟看看科技有限公司 The position filtering method, apparatus and computer storage medium of interactive handle
CN109561254A (en) * 2018-12-18 2019-04-02 深圳岚锋创视网络科技有限公司 A kind of method, apparatus and portable terminal of panoramic video stabilization

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6067340A (en) * 1998-07-06 2000-05-23 Eppstein; Margaret J. Three-dimensional stochastic tomography with upscaling
CN101116101A (en) * 2005-02-04 2008-01-30 佳能株式会社 Position posture measuring method and device
CN105806338A (en) * 2016-03-17 2016-07-27 孙红星 GNSS/INS integrated positioning and directioning algorithm based on three-way Kalman filtering smoother
US20180053056A1 (en) * 2016-08-22 2018-02-22 Magic Leap, Inc. Augmented reality display device with deep learning sensors
CN108346432A (en) * 2017-01-25 2018-07-31 北京三星通信技术研究有限公司 The processing method and relevant device of Virtual Reality audio
CN108170297A (en) * 2017-09-11 2018-06-15 南京睿悦信息技术有限公司 Real-time six degree of freedom VR/AR/MR equipment localization methods
CN108267715A (en) * 2017-12-26 2018-07-10 青岛小鸟看看科技有限公司 Localization method and device, the virtual reality device and system of external equipment
CN109358745A (en) * 2018-08-31 2019-02-19 青岛小鸟看看科技有限公司 The position filtering method, apparatus and computer storage medium of interactive handle
CN109561254A (en) * 2018-12-18 2019-04-02 深圳岚锋创视网络科技有限公司 A kind of method, apparatus and portable terminal of panoramic video stabilization

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
H. HAJIMOLAHOSEINI等: "《Real-time pose estimation and tracking of rigid objects in 3D space using extended Kalman filter》", 《THE 22ND IRANIAN CONFERENCE ON ELECTRICAL ENGINEERING (ICEE 2014)》 *
高祎轩: "《基于扩展卡尔曼滤波的10-DOF惯性测量单元》", 《中国优秀硕士学位论文全文数据库 信息科技辑》 *

Similar Documents

Publication Publication Date Title
US20210233275A1 (en) Monocular vision tracking method, apparatus and non-transitory computer-readable storage medium
CN107255476B (en) Indoor positioning method and device based on inertial data and visual features
CN108627153B (en) Rigid body motion tracking system based on inertial sensor and working method thereof
Klein et al. Parallel tracking and mapping on a camera phone
CN104658012B (en) Motion capture method based on inertia and optical measurement fusion
CN103440624B (en) A kind of image deblurring method based on motion detection and device
CN110880189B (en) Combined calibration method and combined calibration device thereof and electronic equipment
EP3753685A1 (en) Control system and control method
US20180096259A1 (en) Deep-learning motion priors for full-body performance capture in real-time
CN111949123B (en) Multi-sensor handle controller hybrid tracking method and device
CN109029448B (en) Monocular vision inertial positioning's IMU aided tracking model
CN102848389A (en) Realization method for mechanical arm calibrating and tracking system based on visual motion capture
CN103136912A (en) Moving posture capture system
CN109029425B (en) Fuzzy star map restoration method adopting regional filtering
CN111678521B (en) Method and system for evaluating positioning accuracy of mobile robot
CN111899276A (en) SLAM method and system based on binocular event camera
CN111609868A (en) Visual inertial odometer method based on improved optical flow method
WO2024045632A1 (en) Binocular vision and imu-based underwater scene three-dimensional reconstruction method, and device
CN111623773B (en) Target positioning method and device based on fisheye vision and inertial measurement
CN110211151A (en) A kind of method for tracing and device of moving object
CN110533719A (en) Augmented reality localization method and device based on environmental visual Feature point recognition technology
CN114926547A (en) Calibration method of camera and IMU, electronic device and system
CN112907633B (en) Dynamic feature point identification method and application thereof
CN112985450B (en) Binocular vision inertial odometer method with synchronous time error estimation
CN107145706B (en) Evaluation method and device for performance parameters of virtual reality VR equipment fusion algorithm

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20201113

RJ01 Rejection of invention patent application after publication