CN105446472A - 360-degree visible direct interaction type true three-dimensional virtual display system - Google Patents

Abstract

The invention relates to a 360-degree visible direct interaction type true three-dimensional virtual display system. A Kinect device is arranged at an upper suspender of a bracket; two parabolic reflectors are located right below the Kinect device; the opening of one parabolic reflector is upward, the other parabolic reflector is arranged at the upper end of the parabolic reflector, and the center of each parabolic reflector is located at the focal point of the other parabolic reflector; a visual limit divergence device forms a 45-degree angle with the horizontal plane, and a target object is placed below the visual limit divergence device; the lower parabolic reflector is fixed on a hollow rotating table, and the rotating table is further connected with an optical rotary synchronous monitor; an infrared thermal reflector is placed below the rotating table, and a 150mm convex lens is placed below the infrared thermal reflector; and the lens center of a DMD high speed projector is aligned to the centers of the two parabolic reflector having the same size. The invention further provides a gesture recognition method. The 360-degree visible direct interaction type true three-dimensional virtual display system provided by the invention realizes the direct interaction of a user and an imaging object and enhances the user experience in true three-dimensional display and application.

Description

360 degree of visual direct interaction formula true three-dimensional virtual display systems

Technical field

The present invention relates to a kind of 360 degree of visual direct interaction formula true three-dimensional virtual display systems, belong to three dimensions human-computer interaction technique field.

Background technology

True three-dimension display technlolgies (True3DVolumetricDisplayTechnique) is research direction up-to-date in bore hole stereo display.Observer can at any angle views to the stereo-picture with physical depth without the need to wearing any special device, and it meets the sense of reality that people observe the world, meets the depth cueing of all physiology and psychology.True three-dimension display technlolgies can provide 360 degree nakedly to look three-dimensional imaging, and human-computer interaction technology on this basis also result in the concern of researchist.Interactive true three-dimension display technlolgies has broad application prospects and Major Strategic value in various fields such as computer-aided design (CAD), multimedia show, teaching deduction and medical images.

Both at home and abroad carried out a lot of research for 360 degree of visual true 3-D display, this kind of display system generally all comprises these features: 1 provides view-point correction; 2 can show stereoscopic three-dimensional image in real time to user; 3 visual ranges are 360 degree 4 not to be needed to wear extra utility appliance.And these display techniques just progressively trend towards the parallax and the tangential movement parallax that provide depth cue and correct eyes adjustment.Facts have proved, 360 degree of visual true 3-D display can provide three-dimensional sensation more true to nature to beholder compared to those displays only containing stereoeffect.

But current most display system all cannot allow user and display object carry out direct interaction, in general these systems all can separate user and display object with cloche or other blister pack, this is mainly because some is the state being in motion in display object, or some device is to the dangerous property of user, therefore their research direction mainly concentrates on the mutual of display object periphery, cannot allow user directly with display object alternately, user will be hindered like this to the sense of reality of three dimensional object.

Summary of the invention

In view of above prior art Problems existing and deficiency, the invention provides a kind of 360 degree of visual direct interaction formula real three-dimensional display systems, user can be allowed directly to touch display object.

For achieving the above object, the present invention designs a kind of 360 degree of visual direct interaction formula true three-dimensional virtual display systems, comprises support, two identical parabolic mirrors, optics rotate Simultaneous Monitoring device, DMD high speed projector, visual constraints emanator, motor transmission belt, turntable, infra-red heat catoptron, motor, 150mm convex lens, target object and Kinect device; Wherein:

Described support comprises a frame base and a top suspension rod;

Described Kinect device be placed in described support top suspension rod and aim at immediately below;

Described two identical parabolic mirrors are positioned at immediately below described Kinect device, are placed on the upper end of described frame base; The opening upwards of one of them parabolic mirror, another parabolic mirror end placed on it, namely in the involutory placement of clam shell, and makes the center of each parabolic mirror be in the focus of another one catoptron;

Described visual constraints emanator, target object are placed on the lower recess of the parabolic mirror of below, and wherein visual constraints emanator is horizontal by degree angle, and target object is placed on below described visual constraints emanator;

The lower end of the parabolic mirror of below is fixed on the turntable of hollow by support, and motor drives turntable by motor transmission belt, and described turntable also rotates Simultaneous Monitoring device with optics and is connected;

Described infra-red heat catoptron is placed on the below of turntable, and described 150mm convex lens are placed on below described infra-red heat catoptron; Described DMD high speed projector is positioned at the bottom of frame base, and makes the optical center of DMD high speed projector aim at the center of the parabolic mirror of described two formed objects.

The present invention also provides a kind of gesture identification method based on said apparatus, it is characterized in that comprising following steps:

(1) predefine is carried out to gesture motion;

(2) depth data conversion: launch laser light by Kinect device, projected equably on target object by the grating before its infrared transmitter camera lens, target object reflects to form random speckle, again by each speckle in infrared transmitter shot record space, just obtain 3D depth image by calculating;

(3) foreground extraction: the depth data obtained by Kinect device, is divided into prospect and background data; Then direct prospect of the application data, namely extract the hand of user from background;

(4) profile is described: by peak-and-valley algorithm, sketch out the profile of finger from foreground data;

(5) finger detects: perform Kalman filtering algorithm by circulation, identifies the concrete action that finger runs.

The present invention compared with prior art, the parabolic mirror of two equal sizes is utilized to be the involutory placement of clam shell, and when the center of each catoptron is in the focus of another one catoptron, now any one target object is placed on the central area of the 150mm convex lens being in lower, just the three-dimensional imaging seeing this object above the parabolic mirror of eminence can be in, thus achieve the direct interaction of user and imaging object, enhance real tri-dimension display and the Consumer's Experience sense in application.The present invention is reliable, compact conformation, and stability is high.

Accompanying drawing explanation

Fig. 1 is structural representation of the present invention

Fig. 2 is the principle key diagram of embodiment.

Embodiment

One embodiment of the present of invention accompanying drawings is as follows.See Fig. 1, comprise support 1, two identical parabolic mirrors 2, optics rotates Simultaneous Monitoring device 3, DMD high speed projector 4, visual constraints emanator 5, motor transmission belt 6, turntable 7, infra-red heat catoptron 8, motor 9,150mm convex lens 10, target object 11 and Kinect device 12; Wherein: described support 1 comprises a frame base and a top suspension rod; Described Kinect device 12 be placed in described support 1 top suspension rod and aim at immediately below; Described two identical parabolic mirrors 2 are positioned at immediately below described Kinect device 12, are placed on the upper end of described frame base; The opening upwards of one of them parabolic mirror 2, another parabolic mirror 2 end placed on it, namely in the involutory placement of clam shell, and makes the center of each parabolic mirror 2 be in the focus of another one catoptron; Described visual constraints emanator 5, target object 11 are placed on the lower recess of the parabolic mirror 2 of below, and wherein visual constraints emanator 5 is horizontal by miter angle, and target object 11 is placed on below described visual constraints emanator 5; The lower end of the parabolic mirror 2 of below is fixed on the turntable 7 of hollow by support, and motor 9 drives turntable 7 by motor transmission belt 6, and described turntable 7 also rotates Simultaneous Monitoring device 3 with optics and is connected; Described infra-red heat catoptron 8 is placed on the below of turntable 7, and described 150mm convex lens 10 are placed on below described infra-red heat catoptron 8; Described DMD high speed projector 4 be positioned at frame base bottom, and make the optical center of DMD high speed projector 4 aim at the center of parabolic mirror 2 of described two formed objects.

As preferably, parabolic mirror 2 adopts No. 22 Mirage mold products of Opti-Gone company, the diameter of parabolic mirror 2 is 56cm, the opening of its lower recess is for circle and diameter is 15cm, motor 9 adopts model to be the intelligent electric machine of the SM2316D of Animacs company, and DMD high speed projector 4 adopts the DMDDiscovery0.7 of TI company " XGAD4100 high speed projector.

See Fig. 2, the displaying principle of above-described embodiment is when the parabolic mirror 2 of two equal sizes is the involutory placement of clam shell and makes the center of each catoptron be in the focus of another one catoptron, when now any one target object 11 being placed on the central area of the mirror being in lower, just can see the three-dimensional imaging of this target object 11 at the overthe openings of the mirror being in eminence.

During use, motor 9 drives turntable 7 by motor transmission belt 6, and the parabolic mirror 2 that drive two is identical then horizontally rotates; Visual constraints emanator 5 is used for simulating the narrow visual angle of micro-louver (-vre) also needed for generation, and visual constraints emanator 5 often rotates a circle and will produce a lot of viewpoint.DMD high speed projector 4 shows the viewpoint that visual constraints emanator 5 produces with very high frame per second, thus realizes playing up 3D scene more.DMD high speed projector 4 keeps synchronous with motor 9, can guarantee like this at the correct image of correct time showing.Utilize the depth data that the video camera of Kinect device 12 provides, when finger touches 3D image, we can be split from background by the profile of depth data handle, then carry out the profile of track user hand with peak-and-valley image processing algorithm and detect finger, once find finger, we are just inferred by the depth value of equalization finger block of pixels around its 3D position allows 3D image change according to predefined action then.

Wherein, optics rotation Simultaneous Monitoring device 3 adopts the target object 11 on optical principle Simultaneous Monitoring turntable 7; Infra-red heat catoptron 8 is for reflecting the thermal light source processed through 150mm convex lens 10; 150mm convex lens 10 are for focusing on the projection ray of DMD high speed projector injection.

The present invention can also implement a kind of gesture identification method based on said apparatus, is specially and comprises following steps: (one) carries out predefine to gesture motion; (2) depth data conversion: launch laser light by Kinect device, project on the target object in measurement space equably by the grating before its infrared transmitter camera lens, target object reflects to form random speckle, again by each speckle in infrared transmitter shot record space, just obtain 3D depth image by the calculating of processor; (3) foreground extraction: the depth data obtained by Kinect device, is divided into prospect and background data; Then direct prospect of the application data, namely extract the hand of user from background; (4) profile is described: by peak-and-valley algorithm, sketch out the profile of finger from foreground data; (5) finger detects: perform Kalman filtering algorithm by circulation, identifies the concrete action that finger runs.Wherein, step (four) profile is described and step (five) finger detects use peak-and-valley algorithm, and its equation is as follows,

P,(i+j)＝min(P(i-1),P(i+k))

ifP(i+j)＜P(i-1)andP(i+j)＜P(i+k)(1)

P,(i+j)＝min(P(i-1),P(i+k))

ifP(i+j)＞P(i-1)andP(i+j)＞P(i+k)(2)

P,(i+j)＝P(i+j)else.(3)

$\∀j=\mathrm{0,1,2},...,k-1$

Wherein P (i) is the raw intensity values of pixel i, P, and (i) is the new intensity level of pixel i.Equation (1) draws the valley value of k pixel, and equation (2) draws the peak value of k pixel, and equation (3) is then draw the value of k pixel between valley and peak.

Detect finger position in step (five) and use Kalman filtering algorithm, Kalman filtering take least mean-square error as the optimum criterion estimated, seek the algorithm of a set of recurrence estimation, its basic thought is: the state-space model adopting signal and noise, utilize the observed reading of the estimated value of previous moment and now to upgrade the estimation to state variable, obtain the estimated value of now.It is suitable for process and Computing in real time.

Now set the anti-city of the discrete state of linear time varying system and observation equation as:

X(k)＝F(k,k-1)·X(k-1)+T(k,k-1)·U(k-1)(4)

Y(k)＝H(k)·X(k)+N(k)(5)

Wherein X (k) and Y (k) is state vector and the measurement vector in k moment respectively, F (k, k-1) be state-transition matrix, U (k) is k moment dynamic noise, T (k, k-1) be Systematical control matrix, H (k) is k moment observing matrix, and N (k) is k moment observation noise.The algorithm flow of Kalman filtering is:

1 pre-estimation X (k) ^=F (k, k-1) X (k-1)

2 calculate pre-estimation covariance matrix

C(k)^＝F(k,k-1)×C(k)×F(k,k-1)'+T(k,k-1)×Q(k)×T(k,k-1)'

Q(k)＝U(k)×U(k)'

3 calculate kalman gain matrix

K(k)＝C(k)^×H(k)'×[H(k)×C(k)^×H(k)'+R(k)]^(-1)

R(k)＝N(k)×N(k)'

4 more new estimation

X(k)～＝X(k)^+K(k)×[Y(k)-H(k)×X(k)^]

5 calculate the rear estimate covariance matrix of renewal

C(k)～＝[I-K(k)×H(k)]×C(k)^×[I-K(k)×H(k)]'+K(k)×R(k)×K(k)'

6X(k+1)＝X(k)～

C(k+1)＝C(k)～

Be cycled to repeat above step

Once find finger, we are just inferred by the depth value of block of pixels of equalization finger surrounding its 3D position allows 3D image change according to predefined action, thus achieves the direct interaction of user and imaging object then.

Above are only the preferred embodiment of the present invention, be to be noted that for the those of ordinary skill in the industry, can be improved by some and replace under principle of the present invention, this improvement and replacement also should be considered as protection scope of the present invention.

Claims (2)

1. one kind 360 degree visual direct interaction formula true three-dimensional virtual display systems, comprise support (1), it is characterized in that: also comprise two identical parabolic mirrors (2), optics rotates Simultaneous Monitoring device (3), DMD high speed projector (4), visual constraints emanator (5), motor transmission belt (6), turntable (7), infra-red heat catoptron (8), motor (9), 150mm convex lens (10), target object (11) and Kinect device (12); Wherein:

Described support (1) comprises a frame base and a top suspension rod;

Described Kinect device (12) be placed in described support (1) top suspension rod and aim at immediately below;

Described two identical parabolic mirrors (2) are positioned at immediately below described Kinect device (12), are placed on the upper end of described frame base; The opening upwards of one of them parabolic mirror (2), another parabolic mirror (2) end placed on it, namely in the involutory placement of clam shell, and makes the center of each parabolic mirror (2) be in the focus of another one catoptron;

Described visual constraints emanator (5), target object (11) are placed on the lower recess of the parabolic mirror (2) of below, wherein visual constraints emanator (5) is horizontal by miter angle, and target object (11) is placed on described visual constraints emanator (5) below;

The lower end of the parabolic mirror (2) of below is fixed on the turntable (7) of hollow by support, and motor (9) drives turntable (7) by motor transmission belt (6), and described turntable (7) also rotates Simultaneous Monitoring device (3) and is connected with optics;

Described infra-red heat catoptron (8) is placed on the below of turntable (7), and described 150mm convex lens (10) are placed on described infra-red heat catoptron (8) below; Described DMD high speed projector (4) be positioned at frame base bottom, and make the optical center of DMD high speed projector (4) aim at the center of parabolic mirror (2) of described two formed objects.

2., based on a gesture identification method for said apparatus, it is characterized in that comprising following steps:

(1) predefine is carried out to gesture motion;

(2) depth data conversion: launch laser light by Kinect device (12), project on target object (11) equably by the grating before its infrared transmitter camera lens, target object (11) reflects to form random speckle, again by each speckle in infrared transmitter shot record space, just obtain 3D depth image by calculating;

(3) foreground extraction: the depth data obtained by Kinect device (12), is divided into prospect and background data; Then direct prospect of the application data, namely extract the hand of user from background;

(4) profile is described: by peak-and-valley algorithm, sketch out the profile of finger from foreground data;

(5) finger detects: perform Kalman filtering algorithm by circulation, identifies the concrete action that finger runs.