Many exchange methods and device based on Kinect and Unity3D
Technical field
The present invention relates to computer augmented reality field, more particularly to it is a kind of based on many of Kinect and Unity3D
Exchange method and device.
Background technology
Augmented reality (Augmented Reality) is proposed with last century the nineties earliest, now extensive
Applied to medical treatment, education is industrial, many aspects such as business.Augmented reality one is more general to be defined in 1997 by north card
The Ronald Azuma of Luo Laina universities are proposed, including three main aspects:Virtually (Combines real are combined with reality
And virtual), immediate interactive (Interactive in real time), three-dimensional registration (Registered in 3D).Should
Virtual scene is superimposed upon on reality scene by technology on screen, and allows participant and virtual scene interactive.Increase at present
The implementation process of strong reality is generally:1) scene image is obtained by image acquiring device;2) recognize and track in scene and demarcate
Image or word, calculate its deformation calculation its displacement rotating matrix;3) according to the position of uncalibrated image and spin matrix, in three-dimensional
Correspondence dummy model positional information is registered in space;4) fusion dummy model and real scene, and it is included on screen.
But, there is following a few point defects in current common technology:1) interactive mode unification, can only by uncalibrated image or
Word triggering dummy model registration, and the operations such as translation rotation can only be carried out after registration to model, model can only follow demarcation thing
Motion, interactive mode is few and limits more;2) three-dimensional registration algorithm is cumbersome, it is necessary to determine model position according to characteristic point coordinate system
Put and posture, then convert it to camera coordinate system, finally merge dummy model and reality scene and according to indicator screen
Coordinate pair its shown.It can be seen that current art needs the calculating compared with multi-step, operation in the three-dimensional registration phase of dummy model
Not succinct enough and automation.
The content of the invention
The technical problems to be solved by the invention are in view of the shortcomings of the prior art, how voice to be known using Kinect
Other support and the triggering mode that increase dummy model three-dimensional registration is positioned to human skeleton, pass through the identification function of limb action
More interactive modes are provided the user, improve the usage experience of user, and how to utilize Unity3D d engine to mould
Type pose carries out automatic business processing, the key issue of step needed for greatly simplifying three-dimensional registration.
For this purpose, the present invention proposes a kind of many exchange methods based on Kinect and Unity3D, including:
S1:Adjust camera parameters in Unity3D consistent with Kinect valid analysing ranges;
S2:User coordinates and ground equation are determined using Kinect;
S3:Virtual analog coordinate is determined according to relative position, and registers dummy model;
S4:Design interaction posture and voice;
S5:Determine Unity3D Controlling model displacement animations and Multimedia;
S6:The image that the picture got to the video camera in Unity3D is obtained with Kinect camera is merged
And show.
Further, the step S1 further comprises:Kinect is placed to the predeterminated position of reality scene, adjustment reality
Scene is in Kinect valid analysing ranges.
Further, the step S1 further comprises:Adjust Unity3D in video camera Field of view and
Clipping Planes parameters.
Further, the step S2 further comprises:
S21:Determine to represent the plane equation on ground using SkeletonFrame.FloorClipPlane functions, wherein,
The plane equation under the Kinect coordinate systems is:Ax+By+Cz+D=0, (A, B, C) is the flat of the plane equation
Face normal vector, the plane equation under the Unity3D coordinate systems is:Y+E=0, (0,1,0) is the plane equation
Plane normal vector;
S22:(A, B, C) rotation to (0,1,0) is overlapped, Kinect coordinate systems are completed registering with Unity3D coordinate systems.
Further, the Kinect coordinate systems further comprise with the registering of Unity3D coordinate systems:Kinect coordinate systems
Lower arbitrfary point (k1, k2, k3) to Unity3D coordinate systems change when, need to around the X-axis anglec of rotation be-arctan (B/C), revolve about the z axis
Gyration is arctan (A/B), and radius of turn isRotation recoil is designated as:(k1cosα-(k2cosβ-k3sinβ)
Sin α, k1sinα+(k2cosβ-k3Sin β) cos α, k2sinβ+k3Cos β), wherein, α=arctan (A/B), β=- arctan
(B/C)。
Further, the step S6 further comprises:
S61:Two images are sampled or difference operation;
S62:Two images after operation are traveled through, compare in two images with purpose image slices vegetarian refreshments corresponding points
Depth value;
S63:Purpose image corresponding points color value is set to color value of the depth value compared with statuette vegetarian refreshments.
Further, the registration dummy model can also be moved to the side that specific position triggers default models by user
Formula.
Further, the registration dummy model can also be by way of user's speech trigger correspondence model registration.
For this purpose, the present invention proposes a kind of many interactive devices based on Kinect and Unity3D, including:
Adjusting module, it is consistent with Kinect valid analysing ranges for adjusting camera parameters in Unity3D;
Coordinate and ground equation module are determined, for determining user coordinates and ground equation using Kinect;
Dummy model Registering modules, for determining virtual analog coordinate according to relative position, and register dummy model;
Module is designed, for designing interactive posture and voice;
Effects module is determined, for determining Unity3D Controlling model displacement animations and Multimedia;
Image co-registration module, picture and Kinect camera for being got to the video camera in Unity3D are obtained
Image merged and shown.
A kind of many exchange methods based on Kinect and Unity3D disclosed in this invention, first by setting Unity3D
The position of middle video camera and attribute, simplify the conversion between real scene coordinate system and virtual scene coordinate system;Secondly by
Kinect obtains respective coordinates of the user in Unity, and represents the plane equation on ground, then can be according to void to be registered
Analog model and the relative position relation of ground and user determine three-dimensional registration coordinate, and the mechanism that it triggers registration is more flexible, can
To be triggered when user is moved to ad-hoc location, sound identification module can also be used to be triggered.Mould is enriched again
Interactive mode after type registration, can be interacted by limb action and voice operating model;Finally utilize in Unity3D
Transform components and Mecanim animation systems simplify the realization of dummy model change in displacement and animation effect.The present invention is also
Disclose a kind of many interactive devices based on Kinect and Unity3D.
Brief description of the drawings
The features and advantages of the present invention can be more clearly understood from by reference to accompanying drawing, accompanying drawing is schematical without that should manage
Solve to carry out any limitation to the present invention, in the accompanying drawings:
The step of Fig. 1 shows a kind of many exchange methods of in the embodiment of the present invention based on Kinect and Unity3D is flowed
Cheng Tu;
Fig. 2 shows a kind of structure chart of many interactive devices based on Kinect and Unity3D in the embodiment of the present invention.
Embodiment
Below in conjunction with accompanying drawing, embodiments of the present invention is described in detail.
As shown in figure 1, the invention provides a kind of many exchange methods based on Kinect and Unity3D, including specifically with
Lower step:
Step S1:Adjust camera parameters in Unity3D consistent with Kinect valid analysing ranges.Specifically, place
Kinect adjusts reality scene and is in Kinect valid analysing ranges to the predeterminated position of reality scene, wherein, effective range
It is 1.2-3.6 meters of span camera, 57 degree of level, vertical 43 degree.
Further, in the coordinate system of Kinect returned datas, origin is Kinect sensor, therefore will
Video camera is positioned over the origin of coordinates in Unity3D, and dummy model coordinate is calculated during three-dimensional register to facilitate.Adjust in Unity3D
Video camera Field of view and Clipping Planes parameters, by Field of view, Clipping Planes etc.
Parameter and Kinect effective range be arranged to numerical value.
Step S2:User coordinates and ground equation are determined using Kinect.
Specifically, determine to represent the plane equation on ground using SkeletonFrame.FloorClipPlane functions, its
In, the plane equation under Kinect coordinate systems is:Ax+By+Cz+D=0, (A, B, C) is the plane normal vector of plane equation,
Plane equation under Unity3D coordinate systems is:Y+E=0, (0,1,0) is the plane normal vector of plane equation;By (A, B, C)
Rotation to (0,1,0) is overlapped, and completes Kinect coordinate systems registering with Unity3D coordinate systems.
Further, Kinect coordinate systems further comprise with the registering of Unity3D coordinate systems:Lower of Kinect coordinate systems
Anticipate point (k1, k2, k3) to Unity3D coordinate systems change when, need to be-arctan (B/C), the about the z axis anglec of rotation around the X-axis anglec of rotation
Spend for arctan (A/B), radius of turn isRotation recoil is designated as:(k1cosα-(k2cosβ-k3sinβ)sin
α, k1sinα+(k2cosβ-k3Sin β) cos α, k2sinβ+k3Cos β), wherein, α=arctan (A/B), β=- arctan (B/
C)。
Step S3:Virtual analog coordinate is determined according to relative position, and registers dummy model.
Specifically, the conversion formula changed according to the Kinect coordinate systems of above-mentioned steps to Unity3D coordinate systems, will
The bone site point Skeleton Point that API is returned in Kinect SDK are changed to Unity3D coordinate systems.Sat in Unity3D
According to ground level under mark system, the relative position of user coordinates and dummy model and user is by model orientation to required coordinate;Or
Person's selection default models are registered, or are registered to the sound bank addition word related to model, wherein, some words
One model of correspondence, is recognized by Kinect Speech modular voices.After user says word present in sound bank,
Model threedimensional model corresponding with the word is registered in scene.
Step S4:Design interaction posture and voice.
Specifically, interaction posture is designed, the limb action set of every kind of operation is determined.For example:Hovered and represented using arm
Select object or click on button;Mobile arm represents sliding mouse or translation model;Two hands are remote, close expression zoom model;
Two hands embrace ball rotation and represent rotating model etc., and realize simple interaction by voice.For example:It has been shown that, the disappearance, multimedia of model
Broadcasting, pause etc..
Step S5:Determine Unity3D Controlling model displacement animations and Multimedia.
Specifically, corresponding operating is carried out to model according to the limb action of user.Using in Unity3D SDK
Transform components in GameObject objects are translated to model, rotation, the operation such as scaling;Use Mecanim animations
System control model is made to user and followed, and runs, the designed interactive action such as guiding;Use Audio components and Movie
Textures components control Multimedia.
Step S6:The image that the picture and Kinect camera got to the video camera in Unity3D is obtained is carried out
Merge and show.
Specifically, step S6 further comprises:
Step S61:Two images are sampled or difference operation, are allowed to zoom to purpose image size.
Step S62:Two images after operation are traveled through, compare corresponding with purpose image slices vegetarian refreshments in two images
The depth value of point;
Step S63:Purpose image corresponding points color value is set to color value of the depth value compared with statuette vegetarian refreshments.
A kind of many exchange methods based on Kinect and Unity3D disclosed by the invention, are that three-dimensional registration is simple to operate
Augmented reality.Increase dummy model three-dimensional registration is positioned to the support of speech recognition and to human skeleton using Kinect
Triggering mode, more interactive modes are provided the user by the identification function of limb action, improve the usage experience of user;
Automatic business processing is carried out to model pose using Unity3D d engine, step needed for three-dimensional registration is greatly simplify.It is i.e. comprehensive
Close and utilize body feeling interaction equipment and three-dimensional game engine, simplify three-dimensional register flow path, add three-dimensional registration triggering mode, it is rich
Rich user mutual approach, perfect user's operating experience.
As shown in Fig. 2 the invention provides a kind of many interactive devices 10 based on Kinect and Unity3D, including:Adjustment
Module 101, determine coordinate and ground equation module 102, dummy model Registering modules 103, design module 104, determine effect mould
Block 105 and image co-registration module 106.
Specifically, adjusting module 101 is used to adjust camera parameters and Kinect valid analysing ranges one in Unity3D
Cause;Determine that coordinate and ground equation module 102 are used to determine user coordinates and ground equation using Kinect;Dummy model is noted
Volume module 103 is used to determine virtual analog coordinate according to relative position, and registers dummy model;Design module 104 is used to design
Interaction posture and voice;Determine that effects module 105 is used to determine Unity3D Controlling model displacement animations and Multimedia;
Image co-registration module 106 is used for the image that the picture and Kinect got to the video camera in Unity3D camera is obtained
Merged and shown.
A kind of many exchange methods based on Kinect and Unity3D disclosed in this invention, first by setting Unity3D
The position of middle video camera and attribute, simplify the conversion between real scene coordinate system and virtual scene coordinate system;Secondly by
Kinect obtains respective coordinates of the user in Unity, and represents the plane equation on ground, then can be according to void to be registered
Analog model and the relative position relation of ground and user determine three-dimensional registration coordinate, and the mechanism that it triggers registration is more flexible, can
To be triggered when user is moved to ad-hoc location, sound identification module can also be used to be triggered.Mould is enriched again
Interactive mode after type registration, can be interacted by limb action and voice operating model;Finally utilize in Unity3D
Transform components and Mecanim animation systems simplify the realization of dummy model change in displacement and animation effect.The present invention is also
Disclose a kind of many interactive devices based on Kinect and Unity3D.
Embodiment of above is merely to illustrate the present invention, and not limitation of the present invention, about the common of technical field
Technical staff, without departing from the spirit and scope of the present invention, can also make a variety of changes and modification, therefore all
Equivalent technical scheme falls within scope of the invention, and scope of patent protection of the invention should be defined by the claims.
Although being described in conjunction with the accompanying embodiments of the present invention, those skilled in the art can not depart from this hair
Various modifications and variations are made in the case of bright spirit and scope, such modifications and variations are each fallen within by appended claims
Within limited range.