CN102142055A - True three-dimensional design method based on augmented reality interactive technology - Google Patents
True three-dimensional design method based on augmented reality interactive technology Download PDFInfo
- Publication number
- CN102142055A CN102142055A CN2011100856437A CN201110085643A CN102142055A CN 102142055 A CN102142055 A CN 102142055A CN 2011100856437 A CN2011100856437 A CN 2011100856437A CN 201110085643 A CN201110085643 A CN 201110085643A CN 102142055 A CN102142055 A CN 102142055A
- Authority
- CN
- China
- Prior art keywords
- real
- coordinate
- true
- hand
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Abstract
The invention relates to a true three-dimensional design method based on the augmented reality interaction technology. The true three-dimensional design method is the technology which organically integrates the augmented reality and the computer three-dimensional design together for the first time. The invention actually provides a new design development concept that a work area of a designer is expanded from a two-dimensional plane to a true three-dimensional space region, and the augmented reality technology is adopted to carry out man-machine interaction and feedback display to enable a user to carry out the design and development work in the shocked immersion sense. Along with the high-speed development of the multimedia technology, the true three-dimensional design method becomes an inevitable trend and inevitably causes a change of the development field of a computer aided design platform.
Description
Technical field
What the present invention relates to is a kind of very three-dimensional method for designing, a kind of specifically true three-dimensional design method based on the augmented reality interaction technique.
Background technology
(Augmented Reality is that (Virtual Reality VR) develops occur with the practical application needs a kind of with real world information and the integrated new technology of virtual world information " seamless " rapidly along with virtual reality technology AR) to the augmented reality technology.The final target that realizes of AR technology is: produce non-existent virtual objects in the actual environment by computer graphics techniques and visualization technique, interaction technique, display technique etc., and by registration technology with virtual objects exactly " placement " in true environment, allow the user be in the environment of the seamless fusion in a kind of actual situation world.Though all pursue a kind of sensation of being immersed in fully wherein to the user, the difference of AR and VR is that VR will create a complete virtual environment; AR then is the world that is devoted to create an actual situation combination, real world is compensated and strengthens with a complete virtual world." Rediscovering Our Fire " report in the SIGGRAPH meeting is pointed out: augmented reality should be subjected to increasing attention, this field has very big challenge and temptation concerning numerous scholars, and it all will keep vitality in over the next several years.
The authoritative scholar Azuma in AR field is summarized as three aspects with it: the information integration in the actual situation world; Has real-time, interactive; In the three dimension scale space, increase the location dummy object.Wherein, man-machine interaction is an important component part, has only by alternately, and the user could obtain a kind of shocking feeling of immersion.Yet, in be born so far twenties years from AR, the research interest of most researchers concentrated on three-dimensional registration and two fields of information integration, in recent years, along with technology rapid development such as tracking, sensings, the real time human-machine interaction technology among the AR becomes a research focus.
The AR technology develops early abroad, and the U.S., Europe and Japan and other countries and area are leading in AR Study on Technology and application at present, have also possessed design and manufacturing capacity on key equipment, and have had multinomial intellecture property and technical standard technically.U.S. DARPA classifies the AR technology as the important military technology, drop into a large amount of funds and carry out relevant soft, hardware system development, as in famous " Distributed Interactive Simulation " system, having used the AR technology, and with its gordian technique as U.S.'s construction " Force Twenty-One ".2003, companies such as Volkswagen, BMW, Daimler Chrysler, Airbus, Siemens and Fraunhofer Institute for Computer Graphics, University of Munich and Aachen have developed ARVIKA jointly, augmented reality have been applied to industrial.2007, German Heinrich-hertz Research on Communication Technology people such as Peter Eisert designed " virtual mirror " system, the AR technology is applied to the clothing show field.In Aichi, Janpan World Expo in 2005, enterprise of Hitachi has just used in the shop cover AR system and has demonstrated the close relationship of man and nature, people and animal.
At home, the AR technology is started late.Photoelectric project system of Beijing Institute of Technology is one of domestic unit that has early early carried out AR research, at AR system hardware (optical perspective helmet, data glove etc.) some achievements in research have been obtained with aspect such as three-dimensional registration algorithm, and the AR technology is applied to Yuanmingyuan Park field digital rebuilds, reproduced the old times elegance in China culture rarity Yuanmingyuan Park.The CAD﹠amp of Zhejiang University; CG National Key Laboratory and Northern Transportation University's information science institute are all being furtherd investigate aspect distributed virtual reality and the augmented reality gordian technique.The Central China University of Science and Technology is carrying out correlative study aspect the three-dimensional registration.The National University of Defense technology then is platform with the virtual studio, primary study the three-dimensional registration in the virtual studio, virtual camera control technology and actual situation scape superimposing technique.Shanghai University begins to start " based on the AR systematic research of PC platform " first half of the year calendar year 2001, has designed the AR system architecture based on the PC platform, has realized that suitable AR is system, easy, automatic, the camera calibrating method of high precision and robust; Began national science and technology supporting plan problem (2006BAK13B10) in 2006: " strengthening the integrated and application of virtual reality real-time interactive and large-sized special-shaped screen technology ", carrying out deep research aspect real time human-machine interaction technology and the large-sized special-shaped screen enhancing demonstration.
In fields such as product design, cartoon making, construction works, designing and mapping is the basis and the foundation of follow-up all work.A kind of good method for designing can allow the designer that design objects such as haunting product, animating image and building construction are efficiently depicted, and the designer can check the three-dimensional view of designed object intuitively from various angles, it is observed and assesses, and can be as requested in real time, naturally it is carried out various edit-modifies operations.
So-called " true three-dimensional design " is exactly that designer's drawing workspace is opened up extensively in three dimensions by two dimensional surface, and the designer can be from 360 ° of comprehensive Design views of checking oneself.But in the current method for designing that is widely used in the various main flow Three-dimensional Design Software, the designer still designs in a two-dimensional plane zone, generally only provide three-view diagram for the user, this space imagining ability and solid geometry to the designer has higher requirement, and the designer often requires a great deal of time and energy thinking and the relativeness of the designed object of computing in the space; And mainly still utilize traditional man-machine interaction mode---based on input equipments such as mouse and keyboard edit and operate, based on the visual feedback of flat-panel screens, the weak point of this traditional design method is that operating efficiency is low and directly perceived inadequately, and the user has no feeling of immersion.
The AR Study on Technology mainly concentrates on application at present, makes up a kind of new research together in conjunction with other technology and uses.According to investigating and searching data, up to the present, AR technology and Computerized three-dimensional method for designing are organically integrated, the present invention still belongs to the first.
Summary of the invention
The problem and shortage that exists of prior art in view of the above, the object of the present invention is to provide a kind of true three-dimensional design method based on the augmented reality interaction technique, allow the designer in a real three dimensions workspace, carry out design and creation, make the designer experience the creation of oneself with a kind of deep feeling of immersion.
The present invention realizes by the following technical solutions:
A kind of true three-dimensional design method based on the augmented reality interaction technique may further comprise the steps:
1): the initialization system environment is provided with systematic parameter, scanning and check system equipment;
2): algorithm for design is demarcated the binocular camera shooting head that is fixed on the Helmet Mounted Display, and gathers real scene with it, obtains raw video image;
3): algorithm for design carries out error correction to the magnetic force tracker, and utilizes corrected tracking data to finish the real-time orientation tracking of designer's head, hand and the three-dimensional registration of dummy model;
4): algorithm for design carries out normalized to the data gloves, and utilizes the data in real time after the normalization to obtain the case of bending that the designer points;
5): algorithm for design Real time identification designer's man-machine interactive operation, the orientation tracking data of hand and finger case of bending are carried out analyzing and processing, finish the Real time identification of collision detection, static hand shape and dynamic gesture;
6): the virtual scene Real-time modeling set is carried out in the input operation according to the designer, and model of place is carried out real-time conversion and play up generation feedback animation according to its interactive operation;
7): with real scene video and the seamless fusion of dummy model;
8): designing user utilizes Helmet Mounted Display to check design result.
In more detail, described step 2) the camera calibration algorithm specifically may further comprise the steps:
2)-1: take the picture 10-20 that has the gridiron pattern sign with camera earlier and open, require to comprise in the image angle point in all gridiron patterns;
2)-2: call the Corner Detection program measure planar coordinate in image of angle point among every width of cloth figure and each angle point (
U, v);
2)-3: suppose that the mark plane is positioned at the plane of the Z=0 of world coordinate system, self-defined initial point is the angle point in the upper left corner, can obtain the world coordinate system coordinate (X of each angle point in the single image
w, Y
w, 0), try to achieve homography matrix H;
2)-4:, promptly try to achieve the inner parameter of camera by the homography matrix H of multiple image;
2)-5:, utilize the camera inner parameter to try to achieve the rotation matrix and the translation vector of sign object plane in every width of cloth image, the i.e. external parameter of camera at the H of every width of cloth image.
In more detail because magnetic field intensity H is along with the increase of distance is successively decreased with geometric series, therefore near transmitter with at a distance, the magnetic field intensity difference is very big, shows H as Fig. 2 (a)
P1≠ H
P2, cause magnetic force tracking coordinate and real world coordinates no longer linear, so need proofread and correct to the magnetic force tracking data.So, abovementioned steps 3) and magnetic force tracker correcting algorithm specifically may further comprise the steps:
3)-1: the division of space lattice.Show that as Fig. 2 (a) establishing S is the used three-dimensional design work space of system, and it is divided into n small cubes C
i , i.e. S=
, wherein,
i 1≠
i 2C is arranged
i1
∩ C
i2
=Φ, Φ represents empty set.When enough hour of small cubes, then for 2 P that are arranged in same cube
2, P
3H is arranged
P2=H
P3, wherein, H
PiRepresentative point P
iThe magnetic field intensity at place;
3)-2: set up the corresponding tables that magnetic force is followed the tracks of coordinate and real world coordinates.According to the spatial division mode of step 3)-1, mainly be two corresponding relations that overlap between the coordinates of setting up eight summits of small cubes;
3)-3: the self-adaptation of space lattice is heavily divided improvement.With the X coordinate is example, shown in Fig. 2 (b), and definition
λ p =| (
x b `-x a `)/
L-(
x e `-x b `)/
l| be unitary space C
p,
q,
r And C
p+ 1,
q,
r In the linear differences between the two cover coordinates on the X coordinate direction.The corresponding tables that step 3)-2 is generated is for it specifies in linear differences threshold value on the X coordinate direction
λ x o , if
λ p ＞
λ x o , then both are merged into a new unitary space, and in corresponding tables, revise and the deletion respective record, otherwise, just do not merge.At Y, Z coordinate direction in like manner.Space lattice after the improvement is shown in Fig. 2 (c);
3)-4: locate point to be corrected.Shown in Fig. 2 (b), for measured point P, according to its magnetic force that obtains in real time follow the tracks of coordinate (
X`, y`, z`), finding step 3)-3 corresponding tables of setting up, when
x a `≤
X`＜
x b `,
y a `≤
Y`＜
y a1
`With
z a `≤
Z`＜
z d `When three conditions satisfy simultaneously, just a P is positioned unitary space C
p,
q,
r In;
3)-5: linear interpolation is asked for the real world coordinates of this point to be corrected.With the X coordinate is example, shown in Fig. 2 (b), according to the hypothesis of step 3)-1, at unitary space C
p,
q,
r In, magnetic force coordinate and true coordinate are linear, so have
| x-x a |/l=|x`-x a `|/| x b `-x a `|Set up, wherein
lLong for small cubes at the rib under the real world coordinates system, then can draw x=± (
| x`-x a `|/| x b `-x a `|)
L+x a (when X-axis is got "+" during for positive axis, when X-axis is got "-" during for negative semiaxis).In like manner can release
yWith
zThe expression formula of coordinate, sign are followed the example of the coordinate with X.
In the middle of above-mentioned five steps, step 3)-1,3)-2 and 3)-3 before system brings into operation off-line finish, online carrying out when step 3)-4 and 3)-5 moving in system finished the real-time orientation tracking of designer's head, hand and the three-dimensional registration of dummy model.
In more detail, because staff varies, finger has the long weak point that has, and each one is in the difference of dressing aspect the custom, the position with respect to finger of data glove sensor can vary with each individual, so just will inevitably cause the raw data of data glove to have very big error, so must carry out the normalized of data glove.So, abovementioned steps 4) and data glove normalized algorithm specifically may further comprise the steps:
4)-1: the designer is with data glove;
4)-2: obtain training data.Do the peaceful action of stretching of clenching fist in the most natural mode, repeat 10 times, the data glove read-record is arrived array bendMax[10] and bendMin[10];
4)-3: the normalized of real-time crooked data.Make bMax=
Max(bendMax[
i],
i∈ [0,9]), bMin=
Min(bendMin[
j],
j∈ [0,9]), bMax is the maximal value of each finger flexibility of data glove, bMin is the minimum value of each finger flexibility of data glove, bendCurrent is the real-time raw data of data glove, and then the data after the normalization are bCurrent=(bendcurrent-bMin)/(bMax-bMin).
In more detail, described step 5) real time human-machine interaction operation recognizer specifically may further comprise the steps:
5)-1: based on the static hand shape identification of rough set theory;
5)-2: based on the actual situation collision detection of Vclip and AABB bounding box;
5)-3: based on the dynamic gesture identification of geometric locus analysis and HMM model.
In more detail, the static hand shape of described step 5)-1 recognizer specifically may further comprise the steps:
5)-and 1-1: hand structure and finger motion signature analysis.Skeletal structure according to hand, static hand deformationization can be described by the motion that refers to section and joint, except that thumb, all the other four fingers are determined by finger and palm joint, interphalangeal joint, finger tip joint with respect to the motion of palm, the motion of the finger section of finger is around each articulation point motion but is not to move arbitrarily, comprise following characteristics: 1. the flexion/extension of finger-joint is limited in scope, 2. each section maniphalanx moves in same plane, and 3. the motion between interphalangeal joint and the finger tip joint is constrained each other or the like;
5)-and 1-2: the foundation of static hand shape recognition decision table.The present invention with the identification of static hand shape as a decision information system, be designated as DT=(U, C ∪ D, V,
f), wherein, U is a domain; C is the conditional attribute collection; D is a decision kind set; V is the attribute codomain; F is the mapping of U * (C ∪ D) → V .According to the principal element of step 5)-1-1 to the static hand shape of the definite influence of result of hand skeletal structure and movement characteristic analysis, and then the conditional attribute collection C of definite decision table and decision kind set D;
5)-and 1-3: Reduction of Knowledge and rule based on rough set theory are obtained.To the decision table that step 5)-1-2 generates, the conditional attribute of the repetition example in the delete list, redundancy and redundant property value are obtained its yojan successively, finally try to achieve decision rule;
5)-and 1-4: the static hand shape of utilizing the decision rule Real time identification designer that step 5)-1-3 obtains.
In more detail, described step 5)-2 actual situation collision detection algorithm specifically may further comprise the steps:
5)-and 2-1: the averaging model of setting up hand based on statistics;
5)-and 2-2: hand model and the dummy model that generates are in real time carried out protruding decomposition based on the surface.Specifically be expressed as: boundary (P)=
, wherein,
i 1≠
i 2S is arranged
i1
∩ S
i2
=Φ, wherein, P represents dummy model, S
i Be lug, the convex closure body of expression lug, the boundary surface of boundary (P) expression object P;
5)-and 2-3: based on projection set tissue construction level binary tree.Adopt top-down construction method, all projections that generate are constituted convex closure tabulation S, the hierarchical structure BVT (T) on the definition S is an one tree, and T is the root node of hierarchical tree.The geometric element that projection is tabulated in the S is divided into disjoint two groups of subclass (S according to the VRML data layout
1And S
2), ask its convex closure more respectively, create node.Convex closure S with two groups of subclass generating
1, S
2, create two child node (T respectively
1And T
2) as the child node of its root node.Step above the recurrence has only in the tabulation of intranodal projection till the projection;
5)-and 2-4: rough detection-phase.The PolyTree hierarchical structure of structure Vclip makes up an AABB bounding box to each the projection submodel in the hierarchical tree, gets rid of a large amount of obvious disjoint objects simultaneously;
5)-and 2-5: accurate detection-phase.Judge that according to each positive and negative relation of putting the determinant that constitutes of triangle the position between them concerns, thereby judge whether triangle intersects, adopt blocking of GPU to inquire about characteristic and determine to intersect leg-of-mutton quantity.
In more detail, described step 5)-3 dynamic gesture recognizer specifically may further comprise the steps:
5)-and 3-1: the dynamic gesture based on the kinematics character analysis is cut apart.After the user began, the spatial data of its hand was read in real time by the magnetic force tracker, and by system log (SYSLOG).The one section movement locus that is user's its right hand when operation shown in Figure 3, and Figure 4 shows that the speed curve diagram of the right hand corresponding to Fig. 3, based on statistics setting speed threshold value
v 0, when
v Cur＞
v 0The time, then think to have produced the dynamic interaction gesture one time;
5)-and 3-2: effective track of dynamic gesture obtains.For the dynamic interaction gesture that is partitioned among step 5)-3-1, examine its velocity variations tendency in Fig. 4, all be the process that a speed rises and afterwards descends earlier at every turn.So set two threshold speeds
v StartWith
v End, then when speed was between these two values, the track that hand channel is crossed will be as effective track of this gesture to be identified;
5)-initialization of 3-3:HMM model.The init state transition probability matrix
A, suppose state
iCan only turn back to itself or
j=
i + 1 state.For initial probability distribution, set π
1=1 and π
i =0. works as
i≠ 1 o'clock, HMM was from first state.For observing probability matrix
B, suppose
b Ik =1/M, 1≤
i≤ N, 1≤
k≤ M, like this, just finish an initialized HMM model λ=(π,
A,
B);
5)-and 3-4: the HMM model training of dynamic gesture track.Initial model to determining among step 5)-3-3 utilizes HMM parameter optimization algorithm that it is estimated again, and each parameter of HMM is reappraised in this course, obtain new λ `=(π,
A,
B);
5)-and 3-5: the Real-time and Dynamic gesture identification.After process step 5)-3-4 trains the HMM model of each dynamic gesture, model is saved in model bank.During Real time identification, with each HMM model that trains of band identification gesture feature value substitution, utilize the similarity of definite algorithm computation itself and each model of optimum condition chain, getting the maximum is recognition result.
In more detail, the real-time generation and the conversion of described step 6) virtual scene model specifically may further comprise the steps:
6)-1: design point is selected.As selecting " drafting " state, then forward step 6)-2 to, as selecting " editor " state, then forward step 6)-4 to;
6)-2: select drawing tool.Instrument specifically comprises two big classes: planar graph---straight line, rectangle, circle, ellipse, trigonometric sum solid figure---cube, ball, cylinder, circular cone, annulus, rectangular pyramid;
6)-3: draw the operation Real-time modeling set according to the designer.According to the recognition result of step 5)-1, determine rendered object starting point and terminal point with two kinds of static hand shapes;
6)-4: select edit object.Identification of hand shape and collision detection result according to step 5)-1 and step 5)-2 judge whether the designer successfully selects edit object;
6)-5: edit the current object of choosing.Dynamic gesture recognition result according to step 5)-3 triggers corresponding editing operation---translation, rotation, convergent-divergent, deletion, and generate feedback animations such as distortion, conversion in real time.
The present invention compared with prior art, have following conspicuous outstanding substantive distinguishing features and remarkable advantage: in fact the present invention has proposed a kind of brand-new theory of designing and developing---designer's workspace is extended to a very three-dimensional area of space by two dimensional surface, adopt the augmented reality technology to carry out man-machine interaction and feedback demonstration, allow the user in a kind of shocking feeling of immersion, design and develop work.Along with the high speed development of multimedia technology, this very three-dimensional method for designing will become inevitable trend, will cause a change of Computer Aided Design Platform development field.
Description of drawings
Fig. 1 is the true three-dimensional design method process flow diagram based on the augmented reality interaction technique.
Fig. 2 is magnetic force tracking correction algorithm synoptic diagram ((a) space lattice is divided among the figure, and (b) linear interpolation and unitary space merge, and (c) the grid self-adaptation is heavily divided improvement).
One of its right hand section movement locus when Fig. 3 operates for the user.
Fig. 4 is corresponding to Fig. 3 user's right hand movement velocity change curve.
Embodiment
Preferential embodiment accompanying drawings of the present invention is as follows:
Embodiment one:
This may further comprise the steps based on the true three-dimensional design method of augmented reality interaction technique:
1): the initialization system environment is provided with systematic parameter, scanning and check system equipment;
2): algorithm for design is demarcated the binocular camera shooting head that is fixed on the Helmet Mounted Display, and gathers real scene with it, obtains raw video image;
3): algorithm for design carries out error correction to the magnetic force tracker, and utilizes corrected tracking data to finish the real-time orientation tracking of designer's head, hand and the three-dimensional registration of dummy model;
4): algorithm for design carries out normalized to the data gloves, and utilizes the data in real time after the normalization to obtain the case of bending that the designer points;
5): algorithm for design Real time identification designer's man-machine interactive operation, the orientation tracking data of hand and finger case of bending are carried out analyzing and processing, finish the Real time identification of collision detection, static hand shape and dynamic gesture;
6): the virtual scene Real-time modeling set is carried out in the input operation according to the designer, and model of place is carried out real-time conversion and play up generation feedback animation according to its interactive operation;
7): with real scene video and the seamless fusion of dummy model;
8): designing user utilizes Helmet Mounted Display to check design result.
In more detail, described step 2) the camera calibration algorithm specifically may further comprise the steps:
2)-1: take the picture 10-20 that has the gridiron pattern sign with camera earlier and open, require to comprise in the image angle point in all gridiron patterns;
2)-2: call the Corner Detection program measure planar coordinate in image of angle point among every width of cloth figure and each angle point (
U, v);
2)-3: suppose that the mark plane is positioned at the plane of the Z=0 of world coordinate system, self-defined initial point is the angle point in the upper left corner, can obtain the world coordinate system coordinate (X of each angle point in the single image
w, Y
w, 0), try to achieve homography matrix H;
2)-4:, promptly try to achieve the inner parameter of camera by the homography matrix H of multiple image;
2)-5:, utilize the camera inner parameter to try to achieve the rotation matrix and the translation vector of sign object plane in every width of cloth image, the i.e. external parameter of camera at the H of every width of cloth image.
In more detail because magnetic field intensity H is along with the increase of distance is successively decreased with geometric series, therefore near transmitter with at a distance, the magnetic field intensity difference is very big, shows H as Fig. 2 (a)
P1≠ H
P2, cause magnetic force tracking coordinate and real world coordinates no longer linear, so need proofread and correct to the magnetic force tracking data.So, abovementioned steps 3) and magnetic force tracker correcting algorithm specifically may further comprise the steps:
3)-1: the division of space lattice.Show that as Fig. 2 (a) establishing S is the used three-dimensional design work space of system, and it is divided into n small cubes C
i , i.e. S=
, wherein,
i 1≠
i 2C is arranged
i1
∩ C
i2
=Φ, Φ represents empty set.Enough little then for 2 P that are arranged in same cube
2, P
3H is arranged
P2=H
P3, wherein, H
PiRepresentative point P
iThe magnetic field intensity at place;
3)-2: set up the corresponding tables that magnetic force is followed the tracks of coordinate and real world coordinates.According to the spatial division mode of step 3)-1, mainly be two corresponding relations that overlap between the coordinates of setting up eight summits of small cubes;
3)-3: the self-adaptation of space lattice is heavily divided improvement.With the X coordinate is example, shown in Fig. 2 (b), and definition
λ p =| (
x b `-x a `)/
L-(
x e `-x b `)/
l| be unitary space C
p,
q,
r And C
p+ 1,
q,
r In the linear differences between the two cover coordinates on the X coordinate direction.The corresponding tables that step 3)-2 is generated is for it specifies in linear differences threshold value on the X coordinate direction
λ x o , if
λ p ＞
λ x o , then both are merged into a new unitary space, and in corresponding tables, revise and the deletion respective record, otherwise, just do not merge.At Y, Z coordinate direction in like manner.Space lattice after the improvement is shown in Fig. 2 (c);
3)-4: locate point to be corrected.Shown in Fig. 2 (b), for measured point P, according to its magnetic force that obtains in real time follow the tracks of coordinate (
X`, y`, z`), finding step 3)-3 corresponding tables of setting up, when
x a `≤
X`＜
x b `,
y a `≤
Y`＜
y a1
`With
z a `≤
Z`＜
z d `When three conditions satisfy simultaneously, just a P is positioned unitary space C
p,
q,
r In;
3)-5: linear interpolation is asked for the real world coordinates of this point to be corrected.With the X coordinate is example, shown in Fig. 2 (b), according to the hypothesis of step 3)-1, at unitary space C
p,
q,
r In, magnetic force coordinate and true coordinate are linear, so have
| x-x a |/l=|x`-x a `|/| x b `-x a `|Set up, wherein
lLong for small cubes at the rib under the real world coordinates system, then can draw x=± (
| x`-x a `|/| x b `-x a `|)
L+x a (when X-axis is got "+" during for positive axis, when X-axis is got "-" during for negative semiaxis).In like manner can release
yWith
zThe expression formula of coordinate, sign are followed the example of the coordinate with X.
In the middle of above-mentioned five steps, step 3)-1,3)-2 and 3)-3 before system brings into operation off-line finish, online carrying out when step 3)-4 and 3)-5 moving in system finished the real-time orientation tracking of designer's head, hand and the three-dimensional registration of dummy model.
In more detail, because staff varies, finger has the long weak point that has, and each one is in the difference of dressing aspect the custom, the position with respect to finger of data glove sensor can vary with each individual, so just will inevitably cause the raw data of data glove to have very big error, so must carry out the normalized of data glove.So, abovementioned steps 4) and data glove normalized algorithm specifically may further comprise the steps:
4)-1: the designer is with data glove;
4)-2: obtain training data.Do the peaceful action of stretching of clenching fist in the most natural mode, repeat 10 times, the data glove read-record is arrived array bendMax[10] and bendMin[10];
4)-3: the normalized of real-time crooked data.Make bMax=
Max(bendMax[
i],
i∈ [0,9]), bMin=
Min(bendMin[
j],
j∈ [0,9]), bendCurrent is the real-time raw data of data glove, and then the data after the normalization are bCurrent=(bendcurrent-bMin)/(bMax-bMin).
In more detail, described step 5) real time human-machine interaction operation recognizer specifically may further comprise the steps:
5)-1: based on the static hand shape identification of rough set theory;
5)-2: based on the actual situation collision detection of Vclip and AABB bounding box;
5)-3: based on the dynamic gesture identification of geometric locus analysis and HMM model.
In more detail, the static hand shape of described step 5)-1 recognizer specifically may further comprise the steps:
5)-and 1-1: hand structure and finger motion signature analysis.Skeletal structure according to hand, static hand deformationization can be described by the motion that refers to section and joint, except that thumb, all the other four fingers are determined by finger and palm joint, interphalangeal joint, finger tip joint with respect to the motion of palm, the motion of the finger section of finger is around each articulation point motion but is not to move arbitrarily, comprise following characteristics: 1. the flexion/extension of finger-joint is limited in scope, 2. each section maniphalanx moves in same plane, and 3. the motion between interphalangeal joint and the finger tip joint is constrained each other or the like;
5)-and 1-2: the foundation of static hand shape recognition decision table.The present invention with the identification of static hand shape as a decision information system, be designated as DT=(U, C ∪ D, V,
f), wherein, U is a domain; C is the conditional attribute collection; D is a decision kind set; V is the attribute codomain; F is the mapping of U * (C ∪ D) → V .According to the principal element of step 5)-1-1 to the static hand shape of the definite influence of result of hand skeletal structure and movement characteristic analysis, and then the conditional attribute collection C of definite decision table and decision kind set D;
5)-and 1-3: Reduction of Knowledge and rule based on rough set theory are obtained.To the decision table that step 5)-1-2 generates, the conditional attribute of the repetition example in the delete list, redundancy and redundant property value are obtained its yojan successively, finally try to achieve decision rule;
5)-and 1-4: the static hand shape of utilizing the decision rule Real time identification designer that step 5)-1-3 obtains.
In more detail, described step 5)-2 actual situation collision detection algorithm specifically may further comprise the steps:
5)-and 2-1: the averaging model of setting up hand based on statistics;
5)-and 2-2: hand model and the dummy model that generates are in real time carried out protruding decomposition based on the surface.Specifically be expressed as: boundary (P)=
, wherein,
i 1≠
i 2S is arranged
i1
∩ S
i2
=Φ, wherein, P represents dummy model, S
i Be lug, the convex closure body of expression lug, the boundary surface of boundary (P) expression object P;
5)-and 2-3: based on projection set tissue construction level binary tree.Adopt top-down construction method, all projections that generate are constituted convex closure tabulation S, the hierarchical structure BVT (T) on the definition S is an one tree, and T is the root node of hierarchical tree.The geometric element that projection is tabulated in the S is divided into disjoint two groups of subclass (S according to the VRML data layout
1And S
2), ask its convex closure more respectively, create node.Convex closure S with two groups of subclass generating
1, S
2, create two child node (T respectively
1And T
2) as the child node of its root node.Step above the recurrence has only in the tabulation of intranodal projection till the projection;
5)-and 2-4: rough detection-phase.The PolyTree hierarchical structure of structure Vclip makes up an AABB bounding box to each the projection submodel in the hierarchical tree, gets rid of a large amount of obvious disjoint objects simultaneously;
5)-and 2-5: accurate detection-phase.Judge that according to each positive and negative relation of putting the determinant that constitutes of triangle the position between them concerns, thereby judge whether triangle intersects, adopt blocking of GPU to inquire about characteristic and determine to intersect leg-of-mutton quantity.
In more detail, described step 5)-3 dynamic gesture recognizer specifically may further comprise the steps:
5)-and 3-1: the dynamic gesture based on the kinematics character analysis is cut apart.After the user began, the spatial data of its hand was read in real time by the magnetic force tracker, and by system log (SYSLOG).The one section movement locus that is user's its right hand when operation shown in Figure 3, and Figure 4 shows that the speed curve diagram of the right hand corresponding to Fig. 3, based on statistics setting speed threshold value
v 0, when
v Cur＞
v 0The time, then think to have produced the dynamic interaction gesture one time;
5)-and 3-2: effective track of dynamic gesture obtains.For the dynamic interaction gesture that is partitioned among step 5)-3-1, examine its velocity variations tendency in Fig. 4, all be the process that a speed rises and afterwards descends earlier at every turn.So set two threshold speeds
v StartWith
v End, then when speed was between these two values, the track that hand channel is crossed will be as effective track of this gesture to be identified;
5)-initialization of 3-3:HMM model.The init state transition probability matrix
A, suppose state
iCan only turn back to itself or
j=
i + 1 state.For initial probability distribution, set π
1=1 and π
i =0. works as
i≠ 1 o'clock, HMM was from first state.For observing probability matrix
B, suppose
b Ik =1/M, 1≤
i≤ N, 1≤
k≤ M, like this, just finish an initialized HMM model λ=(π,
A,
B);
5)-and 3-4: the HMM model training of dynamic gesture track.Initial model to determining among step 5)-3-3 utilizes HMM parameter optimization algorithm that it is estimated again, and each parameter of HMM is reappraised in this course, obtain new λ `=(π,
A,
B);
5)-and 3-5: the Real-time and Dynamic gesture identification.After process step 5)-3-4 trains the HMM model of each dynamic gesture, model is saved in model bank.During Real time identification, with each HMM model that trains of band identification gesture feature value substitution, utilize the similarity of definite algorithm computation itself and each model of optimum condition chain, getting the maximum is recognition result.
In more detail, the real-time generation and the conversion of described step 6) virtual scene model specifically may further comprise the steps:
6)-1: design point is selected.As selecting " drafting " state, then forward step 6)-2 to, as selecting " editor " state, then forward step 6)-4 to;
6)-2: select drawing tool.Instrument specifically comprises two big classes: planar graph---straight line, rectangle, circle, ellipse, trigonometric sum solid figure---cube, ball, cylinder, circular cone, annulus, rectangular pyramid;
6)-3: draw the operation Real-time modeling set according to the designer.According to the recognition result of step 5)-1, determine rendered object starting point and terminal point with two kinds of static hand shapes;
6)-4: select edit object.Identification of hand shape and collision detection result according to step 5)-1 and step 5)-2 judge whether the designer successfully selects edit object;
6)-5: edit the current object of choosing.Dynamic gesture recognition result according to step 5)-3 triggers corresponding editing operation---translation, rotation, convergent-divergent, deletion, and generate feedback animations such as distortion, conversion in real time.
Embodiment two:
As shown in Figure 1, this specifically comprises following in the machine operation step based on the true three-dimensional design method of augmented reality interaction technique:
Step 1): system brings into operation;
Step 2): device initialize;
Step 3): the user begins;
Step 4): the magnetic force tracker is proofreaied and correct;
Step 5): data glove is proofreaied and correct;
Step 6): real scene collection and processing;
Step 7): open/newly-built design documentation;
Step 8): state is selected, and as selecting " drafting ", then continues, and as selecting " editor ", then forwards step 12) to;
Step 9): select drawing tool;
Step 10): real-time rendering;
Step 11): select whether to want edit object,, then continue,, then forward step 21 to) as selecting "No" as selecting "Yes";
Step 12): collision detection;
Step 13): whether bump, as bump, then continue,, then forward step 12) to as not bumping;
Step 14): operand has been chosen in prompting;
Step 15): select edit mode,, then continue,, then forward step 18 to) as selecting " synchronously complete " mode as selecting " fine setting " mode;
Step 16): dynamic gesture identification;
Step 17): trigger corresponding fine setting editing operation (translation, group commentaries on classics, upset, deletion or the like), change step 21);
Step 18): the identification of static extracting hand shape;
Step 19): judge whether to grasp successfully;
Step 20): full synchro edit operation (move, rotation, convergent-divergent, deletion or the like);
Step 21): select whether to continue design; As select "No", and then continue, as selecting "Yes", then forward step 8) to;
Step 22): preserve design documentation;
Step 23): the user finishes;
Step 24): closing device;
Step 25): log off.
Claims (9)
1. true three-dimensional design method based on the augmented reality interaction technique is characterized in that comprising following operation steps:
1): initialization system environment, configuration-system parameter, scanning and check system equipment;
2): the binocular camera shooting head that is fixed on the Helmet Mounted Display is demarcated, and, obtained raw video image with gathering real scene;
3): algorithm for design carries out error correction to the magnetic force tracker, and utilizes corrected tracking data to finish the real-time orientation tracking of designer's head, hand and the three-dimensional registration of dummy model;
4): algorithm for design carries out normalized to the data gloves, and utilizes the data in real time of normalization to obtain the case of bending that the designer points;
5): algorithm for design carries out analyzing and processing to the orientation tracking data and the finger case of bending of designer's hand, finishes the Real time identification of collision detection, static hand shape and dynamic gesture;
6): the virtual scene Real-time modeling set is carried out in the input operation according to the designer, and model of place is carried out real-time conversion and play up generation feedback animation according to its viewpoint variation and interactive operation;
7): with real scene video and the seamless fusion of dummy model;
8): based on stereovision technique, all types of user utilizes Helmet Mounted Display, Projection Display or tabletop display isotype to check design result respectively.
2. the true three-dimensional design method based on the augmented reality interaction technique according to claim 1 is characterized in that described step 2) specifically may further comprise the steps:
2)-1: take the picture 10-20 that has the gridiron pattern sign with camera earlier and open, require to comprise in the image angle point in all gridiron patterns;
2)-2: call the Corner Detection program measure planar coordinate in image of angle point among every width of cloth figure and each angle point (
U, v);
2)-3: suppose that the mark plane is positioned at the plane of the Z=0 of world coordinate system, self-defined initial point is the angle point in the upper left corner, can obtain the world coordinate system coordinate (X of each angle point in the single image
w, Y
w, 0), try to achieve homography matrix H;
2)-4:, promptly try to achieve the inner parameter of camera by the homography matrix H of multiple image;
2)-5:, utilize the camera inner parameter to try to achieve the rotation matrix and the translation vector of sign object plane in every width of cloth image, the i.e. external parameter of camera at the H of every width of cloth image.
3. the true three-dimensional design method based on the augmented reality interaction technique according to claim 1 is characterized in that described step 3) specifically may further comprise the steps:
3)-1: the division of space lattice; Show that as Fig. 2 (a) establishing S is the used three-dimensional design work space of system, and it is divided into n small cubes C
i , i.e. S=
, wherein,
i 1≠
i 2C is arranged
i1
∩ C
i2
=Φ, Φ represents empty set;
When enough hour of small cubes, then for 2 P that are arranged in same cube
2, P
3H is arranged
P2=H
P3, wherein, H
PiRepresentative point P
iThe magnetic field intensity at place;
3)-2: set up the corresponding tables that magnetic force is followed the tracks of coordinate and real world coordinates; According to the spatial division mode of step 3)-1, mainly be two corresponding relations that overlap between the coordinates of setting up eight summits of small cubes;
3)-3: the self-adaptation of space lattice is heavily divided improvement; With the X coordinate is example, shown in Fig. 2 (b), and definition
λ p =| (
x b `-x a `)/
L-(
x e `-x b `)/
l| be unitary space C
p,
q,
r And C
p+ 1,
q,
r In the linear differences between the two cover coordinates on the X coordinate direction; The corresponding tables that step 3)-2 is generated is for it specifies in linear differences threshold value on the X coordinate direction
λ x o , if
λ p ＞
λ x o , then both are merged into a new unitary space, and in corresponding tables, revise and the deletion respective record, otherwise, just do not merge; At Y, Z coordinate direction in like manner; Space lattice after the improvement is shown in Fig. 2 (c);
3)-4: locate point to be corrected; Shown in Fig. 2 (b), for measured point P, according to its magnetic force that obtains in real time follow the tracks of coordinate (
X`, y`, z`), finding step 3)-3 corresponding tables of setting up, when
x a `≤
X`＜
x b `,
y a `≤
Y`＜
y a1
`With
z a `≤
Z`＜
z d `When three conditions satisfy simultaneously, just a P is positioned unitary space C
p,
q,
r In;
3)-5: linear interpolation is asked for the real world coordinates of this point to be corrected; With the X coordinate is example, shown in Fig. 2 (b), according to the hypothesis of step 3)-1, at unitary space C
p,
q,
r In, magnetic force coordinate and true coordinate are linear, so have
| x-x a |/l=|x`-x a `|/| x b `-x a `|Set up, wherein
lLong for small cubes at the rib under the real world coordinates system, then can draw x=± (
| x`-x a `|/| x b `-x a `|)
L+x a (when X-axis is got "+" during for positive axis, when X-axis is got "-" during for negative semiaxis); In like manner can release
yWith
zThe expression formula of coordinate, sign are followed the example of the coordinate with X;
In the middle of above-mentioned five steps, step 3)-1,3)-2 and 3)-3 before system brings into operation off-line finish, online carrying out when step 3)-4 and 3)-5 moving in system finished the real-time orientation tracking of designer's head, hand and the three-dimensional registration of dummy model.
4. the true three-dimensional design method based on the augmented reality interaction technique according to claim 1 is characterized in that described step 4) specifically may further comprise the steps:
4)-1: the designer is with data glove;
4)-2: obtain training data; Do the peaceful action of stretching of clenching fist in the most natural mode, repeat 10 times, the data glove read-record is arrived array bendMax[10] and bendMin[10];
4)-3: the normalized of real-time crooked data; Make bMax=
Max(bendMax[
i],
i∈ [0,9]), bMin=
Min(bendMin[
j],
j∈ [0,9]), bMax is the maximal value of each finger flexibility of data glove, bMin is the minimum value of each finger flexibility of data glove, bendCurrent is the real-time raw data of data glove, and then the data after the normalization are bCurrent=(bendcurrent-bMin)/(bMax-bMin).
5. the true three-dimensional design method based on the augmented reality interaction technique according to claim 1 is characterized in that described step 5) specifically may further comprise the steps:
5)-1: based on the static hand shape identification of rough set theory;
5)-2: based on the actual situation collision detection of Vclip and AABB bounding box;
5)-3: based on the dynamic gesture identification of geometric locus analysis and HMM model.
6. the true three-dimensional design method based on the augmented reality interaction technique according to claim 5 is characterized in that described step 5)-1 specifically may further comprise the steps:
5)-and 1-1: hand structure and finger motion signature analysis; Skeletal structure according to hand, static hand deformationization can be described by the motion that refers to section and joint, except that thumb, all the other four fingers are determined by finger and palm joint, interphalangeal joint, finger tip joint with respect to the motion of palm, the motion of the finger section of finger is around each articulation point motion but is not to move arbitrarily, comprise following characteristics: 1. the flexion/extension of finger-joint is limited in scope, 2. each section maniphalanx moves in same plane, and 3. the motion between interphalangeal joint and the finger tip joint is constrained each other or the like;
5)-and 1-2: the foundation of static hand shape recognition decision table; The present invention with the identification of static hand shape as a decision information system, be designated as DT=(U, C ∪ D, V, f), wherein, U is a domain; C is the conditional attribute collection; D is a decision kind set; V is the attribute codomain; F is the mapping of U * (C ∪ D) → V ; According to the principal element of step 5)-1-1 to the static hand shape of the definite influence of result of hand skeletal structure and movement characteristic analysis, and then the conditional attribute collection C of definite decision table and decision kind set D;
5)-and 1-3: Reduction of Knowledge and rule based on rough set theory are obtained; To the decision table that step 5)-1-2 generates, the conditional attribute of the repetition example in the delete list, redundancy and redundant property value are obtained its yojan successively, finally try to achieve decision rule;
5)-and 1-4: the static hand shape of utilizing the decision rule Real time identification designer that step 5)-1-3 obtains.
7. the true three-dimensional design method based on the augmented reality interaction technique according to claim 5 is characterized in that described step 5)-2 specifically may further comprise the steps:
5)-and 2-1: the averaging model of setting up hand based on statistics;
5)-and 2-2: hand model and the dummy model that generates are in real time carried out protruding decomposition based on the surface; Specifically be expressed as: boundary (P)=
, wherein,
I1 ≠ i2 has S i1 ∩ Si2=Φ, and wherein, P represents dummy model, and Si is a lug, the convex closure body of expression lug, the boundary surface of boundary (P) expression object P;
5)-2-3: based on projection set tissue construction level binary tree; Adopt top-down construction method, all projections that generate are constituted convex closure tabulation S, the hierarchical structure BVT (T) on the definition S is an one tree, and T is the root node of hierarchical tree; The geometric element that projection is tabulated in the S is divided into disjoint two groups of subclass (S1 and S2) according to the VRML data layout, asks its convex closure more respectively, creates node; With the convex closure S1 of two groups of subclass generating, S2 creates the child node of two child nodes (T1 and T2) as its root node respectively; Step above the recurrence has only in the tabulation of intranodal projection till the projection;
5)-and 2-4: rough detection-phase; The PolyTree hierarchical structure of structure Vclip makes up an AABB bounding box to each the projection submodel in the hierarchical tree, gets rid of a large amount of obvious disjoint objects simultaneously;
5)-and 2-5: accurate detection-phase; Judge that according to each positive and negative relation of putting the determinant that constitutes of triangle the position between them concerns, thereby judge whether triangle intersects, adopt blocking of GPU to inquire about characteristic and determine to intersect leg-of-mutton quantity.
8. the true three-dimensional design method based on the augmented reality interaction technique according to claim 5 is characterized in that described step 5)-3 specifically may further comprise the steps:
5)-and 3-1: the dynamic gesture based on the kinematics character analysis is cut apart; After the user began, the spatial data of its hand was read in real time by the magnetic force tracker, and by system log (SYSLOG); The one section movement locus that is the user at its right hand of when operation shown in Figure 3, and Figure 4 shows that the speed curve diagram of the right hand corresponding to Fig. 3 based on statistics setting speed threshold value v0, when vcur＞v0, is then thought to have produced the dynamic interaction gesture one time;
5)-and 3-2: effective track of dynamic gesture obtains; For the dynamic interaction gesture that is partitioned among step 5)-3-1, examine its velocity variations tendency in Fig. 4, all be the process that a speed rises and afterwards descends earlier at every turn; So set two threshold speed vstart and vend, then when speed was between these two values, the track that hand channel is crossed will be as effective track of this gesture to be identified;
5)-initialization of 3-3:HMM model; Init state transition probability matrix A supposes that state i can only turn back to own or j=i+1 state; For initial probability distribution, set π 1=1 and π i=0. when i ≠ 1, HMM is from first state; For observing probability matrix B, suppose bik=1/M, 1≤i≤N, 1≤k≤M, like this, just finish an initialized HMM model λ=(π, A, B);
5)-and 3-4: the HMM model training of dynamic gesture track; Initial model to determining among step 5)-3-3 utilizes HMM parameter optimization algorithm that it is estimated again, and each parameter of HMM is reappraised in this course, obtain new λ `=(π, A, B);
5)-and 3-5: the Real-time and Dynamic gesture identification; After process step 5)-3-4 trains the HMM model of each dynamic gesture, model is saved in model bank; During Real time identification, with each HMM model that trains of band identification gesture feature value substitution, utilize the similarity of definite algorithm computation itself and each model of optimum condition chain, getting the maximum is recognition result.
9. the true three-dimensional design method based on the augmented reality interaction technique according to claim 1 is characterized in that described step 6) specifically may further comprise the steps:
6)-1: design point is selected; As selecting " drafting " state, then forward step 6)-2 to, as selecting " editor " state, then forward step 6)-4 to;
6)-2: select drawing tool; Instrument specifically comprises two big classes: planar graph---straight line, rectangle, circle, ellipse, trigonometric sum solid figure---cube, ball, cylinder, circular cone, annulus, rectangular pyramid;
6)-3: draw the operation Real-time modeling set according to the designer; According to the recognition result of step 5)-1, determine rendered object starting point and terminal point with two kinds of static hand shapes;
6)-4: select edit object; Identification of hand shape and collision detection result according to step 5)-1 and step 5)-2 judge whether the designer successfully selects edit object;
6)-5: edit the current object of choosing; Dynamic gesture recognition result according to step 5)-3 triggers corresponding editing operation---translation, rotation, convergent-divergent, deletion, and generation in real time is out of shape, the feedback animation of conversion.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2011100856437A CN102142055A (en) | 2011-04-07 | 2011-04-07 | True three-dimensional design method based on augmented reality interactive technology |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2011100856437A CN102142055A (en) | 2011-04-07 | 2011-04-07 | True three-dimensional design method based on augmented reality interactive technology |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN102142055A true CN102142055A (en) | 2011-08-03 |
Family
ID=44409560
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2011100856437A Pending CN102142055A (en) | 2011-04-07 | 2011-04-07 | True three-dimensional design method based on augmented reality interactive technology |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102142055A (en) |
Cited By (43)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102693413A (en) * | 2011-02-18 | 2012-09-26 | 微软公司 | Motion recognition |
| CN102945564A (en) * | 2012-10-16 | 2013-02-27 | 上海大学 | True 3D modeling system and method based on video perspective type augmented reality |
| CN103257709A (en) * | 2013-05-06 | 2013-08-21 | 上海大学 | Three-dimensional space two-point touch and control system based on optical reflection and three-dimensional space two-point touch and control method based on optical reflection |
| CN103353941A (en) * | 2013-06-13 | 2013-10-16 | 西安电子科技大学 | Natural marker registration method based on viewpoint classification |
| CN103761734A (en) * | 2014-01-08 | 2014-04-30 | 北京航空航天大学 | Binocular stereoscopic video scene fusion method for keeping time domain consistency |
| CN104102678A (en) * | 2013-04-15 | 2014-10-15 | 腾讯科技（深圳）有限公司 | Method and device for realizing augmented reality |
| CN104460991A (en) * | 2014-11-19 | 2015-03-25 | 中山大学 | Gesture interaction control system based on digital household equipment |
| CN104699887A (en) * | 2013-12-10 | 2015-06-10 | 达索系统公司 | Augmented reality updating of 3d cad models |
| CN104914993A (en) * | 2015-05-15 | 2015-09-16 | 北京航空航天大学 | Experience type design method for controlling civil aircraft passenger cabin seat adjustment by gestures |
| CN105023215A (en) * | 2015-07-21 | 2015-11-04 | 中国矿业大学(北京) | Dangerous trade safety training system based on head-mounted mixed reality equipment |
| CN105164731A (en) * | 2013-03-14 | 2015-12-16 | 微软技术许可有限责任公司 | Mapping augmented reality experience to various environments |
| CN105389005A (en) * | 2015-10-27 | 2016-03-09 | 武汉体育学院 | Three-dimensional interactive display method for twenty-four-form Tai Chi Chuan |
| CN105488825A (en) * | 2015-12-11 | 2016-04-13 | 重庆环漫科技有限公司 | AR system configuration method |
| CN105555373A (en) * | 2013-09-30 | 2016-05-04 | 高通股份有限公司 | Augmented reality apparatus, method and program |
| CN105807936A (en) * | 2016-03-31 | 2016-07-27 | 联想(北京)有限公司 | Information processing method and electronic equipment |
| CN105844705A (en) * | 2016-03-29 | 2016-08-10 | 联想(北京)有限公司 | Three-dimensional virtual object model generation method and electronic device |
| CN106062862A (en) * | 2014-10-24 | 2016-10-26 | 何安莉 | System and method for immersive and interactive multimedia generation |
| CN106251404A (en) * | 2016-07-19 | 2016-12-21 | 央数文化(上海)股份有限公司 | Orientation tracking, the method realizing augmented reality and relevant apparatus, equipment |
| CN106408515A (en) * | 2016-08-31 | 2017-02-15 | 郑州捷安高科股份有限公司 | Augmented reality-based vision synthesis system |
| CN103955267B (en) * | 2013-11-13 | 2017-03-15 | 上海大学 | Both hands man-machine interaction method in x ray fluoroscopy x augmented reality system |
| CN106796446A (en) * | 2014-08-04 | 2017-05-31 | 惠普发展公司，有限责任合伙企业 | Workspace-metadata is managed |
| CN106887183A (en) * | 2017-03-24 | 2017-06-23 | 广西七三科技有限公司 | A kind of interactive demonstration method and system of BIM augmented realities in sand table is built |
| CN106951631A (en) * | 2017-03-18 | 2017-07-14 | 李远阳 | A kind of circular and Rectangular Bounding Volume collision checking method with the anglec of rotation |
| CN107219918A (en) * | 2017-04-28 | 2017-09-29 | 西北工业大学 | A kind of design method of data glove interface module |
| CN107331220A (en) * | 2017-09-01 | 2017-11-07 | 国网辽宁省电力有限公司锦州供电公司 | Transformer O＆M simulation training system and method based on augmented reality |
| CN107392853A (en) * | 2017-07-13 | 2017-11-24 | 河北中科恒运软件科技股份有限公司 | Double-camera video frequency merges distortion correction and viewpoint readjustment method and system |
| CN107548470A (en) * | 2015-04-15 | 2018-01-05 | 索尼互动娱乐股份有限公司 | Nip and holding gesture navigation on head mounted display |
| CN107608507A (en) * | 2017-09-05 | 2018-01-19 | 清华大学 | The method for selecting of locomotive component and locomotive auxiliary maintaining system under low light condition |
| CN107728792A (en) * | 2017-11-17 | 2018-02-23 | 浙江大学 | A kind of augmented reality three-dimensional drawing system and drawing practice based on gesture identification |
| CN107919033A (en) * | 2016-10-10 | 2018-04-17 | 北京七展国际数字科技有限公司 | A kind of telegraphy official's Experience of Ecological system by merging multimedia, micro climate and bio-landscape |
| CN108182728A (en) * | 2018-01-19 | 2018-06-19 | 武汉理工大学 | A kind of online body-sensing three-dimensional modeling method and system based on Leap Motion |
| CN108325208A (en) * | 2018-03-20 | 2018-07-27 | 昆山时记信息科技有限公司 | Augmented reality implementation method applied to field of play |
| CN108399638A (en) * | 2018-02-08 | 2018-08-14 | 重庆爱奇艺智能科技有限公司 | A kind of augmented reality exchange method, device and electronic equipment based on label |
| CN108460399A (en) * | 2017-12-29 | 2018-08-28 | 华南师范大学 | A kind of child building block builds householder method and system |
| CN108693954A (en) * | 2017-04-05 | 2018-10-23 | 北京暴风魔镜科技有限公司 | Scene edit methods and device, gesture information generating means and scene editing system |
| CN109948592A (en) * | 2019-04-04 | 2019-06-28 | 北京理工大学 | A kind of design idea method of discrimination and system based on hand signal identification |
| CN110070777A (en) * | 2019-06-13 | 2019-07-30 | 大连民族大学 | A kind of Hezhe's fish-skin draws simulation training system and implementation method |
| CN110389664A (en) * | 2019-06-25 | 2019-10-29 | 浙江大学 | A kind of fire scenario sunykatuib analysis device and method based on augmented reality |
| CN112164258A (en) * | 2019-12-26 | 2021-01-01 | 昆山时记信息科技有限公司 | AR intelligent teaching method, device, teaching aid system and computer equipment |
| CN113012298A (en) * | 2020-12-25 | 2021-06-22 | 浙江大学 | Curved MARK three-dimensional registration augmented reality method based on region detection |
| CN113066189A (en) * | 2021-04-06 | 2021-07-02 | 海信视像科技股份有限公司 | Augmented reality equipment and virtual and real object shielding display method |
| CN113553884A (en) * | 2020-04-26 | 2021-10-26 | 武汉Tcl集团工业研究院有限公司 | Gesture recognition method, terminal device and computer-readable storage medium |
| CN113961080A (en) * | 2021-11-09 | 2022-01-21 | 南京邮电大学 | Three-dimensional modeling software framework based on gesture interaction and design method |
-
2011
- 2011-04-07 CN CN2011100856437A patent/CN102142055A/en active Pending
Non-Patent Citations (7)
| Title |
|---|
| 《2008 IEEE Pacific-Asia Workshop on Computational Intelligence and Industrial Application》 20081231 Zhengwei Yao, et al. Throwing Recognition Based on Magnetic Tracking and Trajectory Computation in an Augmented Reality System 第544－545页、图2,3 8 , * |
| 《上海交通大学学报》 19981031 焦圣品，等 虚拟现实中手的建模 第134页倒数第1段－第135页第1段、图1 6 第32卷, 第10期 * |
| 《中国优秀硕士学位论文全文数据库 信息科技辑》 20080315 庄化 基于图像空间的碰撞检测 正文第45页－第68页 7 , * |
| 《电脑与信息技术》 20110228 潘异峰，等 基于粗糙集和证据理论的目标识别方法研究 第21页右栏第2段,第23页右栏第2-13行 6 第19卷, 第1期 * |
| 《计算机工程与设计》 20051130 孔晓明，等 基于视觉的动态手势识别 第2935-2936页 8 第26卷, 第11期 * |
| 《计算机工程与设计》 20091231 李启明，等 增强现实系统中磁力跟踪注册的精确性研究 第5786－5788页、图1,3 1-5 第30卷, 第24期 * |
| 《计算机辅助设计与图形学学报》 20091130 姚争为，等 增强现实系统中基于视觉感知的抓取识别 第1583页右栏－第1585页左栏、图2 1-5 第21卷, 第11期 * |
Cited By (62)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102693413A (en) * | 2011-02-18 | 2012-09-26 | 微软公司 | Motion recognition |
| CN102945564A (en) * | 2012-10-16 | 2013-02-27 | 上海大学 | True 3D modeling system and method based on video perspective type augmented reality |
| CN105164731A (en) * | 2013-03-14 | 2015-12-16 | 微软技术许可有限责任公司 | Mapping augmented reality experience to various environments |
| CN104102678A (en) * | 2013-04-15 | 2014-10-15 | 腾讯科技（深圳）有限公司 | Method and device for realizing augmented reality |
| WO2014169692A1 (en) * | 2013-04-15 | 2014-10-23 | Tencent Technology (Shenzhen) Company Limited | Method,device and storage medium for implementing augmented reality |
| CN104102678B (en) * | 2013-04-15 | 2018-06-05 | 腾讯科技（深圳）有限公司 | The implementation method and realization device of augmented reality |
| CN103257709A (en) * | 2013-05-06 | 2013-08-21 | 上海大学 | Three-dimensional space two-point touch and control system based on optical reflection and three-dimensional space two-point touch and control method based on optical reflection |
| CN103353941A (en) * | 2013-06-13 | 2013-10-16 | 西安电子科技大学 | Natural marker registration method based on viewpoint classification |
| CN103353941B (en) * | 2013-06-13 | 2016-08-10 | 西安电子科技大学 | Natural marker registration method based on viewpoint classification |
| US10217284B2 (en) | 2013-09-30 | 2019-02-26 | Qualcomm Incorporated | Augmented virtuality |
| CN105555373A (en) * | 2013-09-30 | 2016-05-04 | 高通股份有限公司 | Augmented reality apparatus, method and program |
| CN103955267B (en) * | 2013-11-13 | 2017-03-15 | 上海大学 | Both hands man-machine interaction method in x ray fluoroscopy x augmented reality system |
| CN104699887A (en) * | 2013-12-10 | 2015-06-10 | 达索系统公司 | Augmented reality updating of 3d cad models |
| CN104699887B (en) * | 2013-12-10 | 2020-08-14 | 达索系统公司 | Augmented reality update of 3D CAD models |
| CN103761734A (en) * | 2014-01-08 | 2014-04-30 | 北京航空航天大学 | Binocular stereoscopic video scene fusion method for keeping time domain consistency |
| CN103761734B (en) * | 2014-01-08 | 2016-09-28 | 北京航空航天大学 | A kind of binocular stereoscopic video scene fusion method of time domain holding consistency |
| CN106796446B (en) * | 2014-08-04 | 2020-05-12 | 惠普发展公司，有限责任合伙企业 | Workspace metadata management |
| CN106796446A (en) * | 2014-08-04 | 2017-05-31 | 惠普发展公司，有限责任合伙企业 | Workspace-metadata is managed |
| CN106062862B (en) * | 2014-10-24 | 2020-04-21 | 杭州凌感科技有限公司 | System and method for immersive and interactive multimedia generation |
| CN106062862A (en) * | 2014-10-24 | 2016-10-26 | 何安莉 | System and method for immersive and interactive multimedia generation |
| CN104460991A (en) * | 2014-11-19 | 2015-03-25 | 中山大学 | Gesture interaction control system based on digital household equipment |
| CN107548470B (en) * | 2015-04-15 | 2020-06-26 | 索尼互动娱乐股份有限公司 | Pinch and hold gesture navigation on head mounted display |
| CN107548470A (en) * | 2015-04-15 | 2018-01-05 | 索尼互动娱乐股份有限公司 | Nip and holding gesture navigation on head mounted display |
| CN104914993A (en) * | 2015-05-15 | 2015-09-16 | 北京航空航天大学 | Experience type design method for controlling civil aircraft passenger cabin seat adjustment by gestures |
| CN105023215A (en) * | 2015-07-21 | 2015-11-04 | 中国矿业大学(北京) | Dangerous trade safety training system based on head-mounted mixed reality equipment |
| CN105389005A (en) * | 2015-10-27 | 2016-03-09 | 武汉体育学院 | Three-dimensional interactive display method for twenty-four-form Tai Chi Chuan |
| CN105488825A (en) * | 2015-12-11 | 2016-04-13 | 重庆环漫科技有限公司 | AR system configuration method |
| CN105488825B (en) * | 2015-12-11 | 2018-06-26 | 重庆环漫科技有限公司 | A kind of AR ' In System Reconfiguration Methods |
| CN105844705B (en) * | 2016-03-29 | 2018-11-09 | 联想(北京)有限公司 | A kind of three-dimensional object model generation method and electronic equipment |
| CN105844705A (en) * | 2016-03-29 | 2016-08-10 | 联想(北京)有限公司 | Three-dimensional virtual object model generation method and electronic device |
| CN105807936B (en) * | 2016-03-31 | 2021-02-19 | 联想(北京)有限公司 | Information processing method and electronic equipment |
| CN105807936A (en) * | 2016-03-31 | 2016-07-27 | 联想(北京)有限公司 | Information processing method and electronic equipment |
| CN106251404B (en) * | 2016-07-19 | 2019-02-01 | 央数文化(上海)股份有限公司 | Orientation tracking, the method and relevant apparatus, equipment for realizing augmented reality |
| CN106251404A (en) * | 2016-07-19 | 2016-12-21 | 央数文化(上海)股份有限公司 | Orientation tracking, the method realizing augmented reality and relevant apparatus, equipment |
| CN106408515A (en) * | 2016-08-31 | 2017-02-15 | 郑州捷安高科股份有限公司 | Augmented reality-based vision synthesis system |
| CN107919033A (en) * | 2016-10-10 | 2018-04-17 | 北京七展国际数字科技有限公司 | A kind of telegraphy official's Experience of Ecological system by merging multimedia, micro climate and bio-landscape |
| CN106951631A (en) * | 2017-03-18 | 2017-07-14 | 李远阳 | A kind of circular and Rectangular Bounding Volume collision checking method with the anglec of rotation |
| CN106887183A (en) * | 2017-03-24 | 2017-06-23 | 广西七三科技有限公司 | A kind of interactive demonstration method and system of BIM augmented realities in sand table is built |
| CN108693954A (en) * | 2017-04-05 | 2018-10-23 | 北京暴风魔镜科技有限公司 | Scene edit methods and device, gesture information generating means and scene editing system |
| CN107219918A (en) * | 2017-04-28 | 2017-09-29 | 西北工业大学 | A kind of design method of data glove interface module |
| CN107392853B (en) * | 2017-07-13 | 2020-05-26 | 河北中科恒运软件科技股份有限公司 | Method and system for video fusion distortion correction and viewpoint fine adjustment of double cameras |
| CN107392853A (en) * | 2017-07-13 | 2017-11-24 | 河北中科恒运软件科技股份有限公司 | Double-camera video frequency merges distortion correction and viewpoint readjustment method and system |
| CN107331220A (en) * | 2017-09-01 | 2017-11-07 | 国网辽宁省电力有限公司锦州供电公司 | Transformer O＆M simulation training system and method based on augmented reality |
| CN107608507A (en) * | 2017-09-05 | 2018-01-19 | 清华大学 | The method for selecting of locomotive component and locomotive auxiliary maintaining system under low light condition |
| CN107608507B (en) * | 2017-09-05 | 2021-04-13 | 清华大学 | Method for selecting locomotive components under weak light condition and locomotive auxiliary maintenance system |
| CN107728792A (en) * | 2017-11-17 | 2018-02-23 | 浙江大学 | A kind of augmented reality three-dimensional drawing system and drawing practice based on gesture identification |
| CN108460399B (en) * | 2017-12-29 | 2021-10-15 | 华南师范大学 | Auxiliary method and system for building block for children |
| CN108460399A (en) * | 2017-12-29 | 2018-08-28 | 华南师范大学 | A kind of child building block builds householder method and system |
| CN108182728A (en) * | 2018-01-19 | 2018-06-19 | 武汉理工大学 | A kind of online body-sensing three-dimensional modeling method and system based on Leap Motion |
| CN108399638A (en) * | 2018-02-08 | 2018-08-14 | 重庆爱奇艺智能科技有限公司 | A kind of augmented reality exchange method, device and electronic equipment based on label |
| CN108399638B (en) * | 2018-02-08 | 2021-07-20 | 重庆爱奇艺智能科技有限公司 | Augmented reality interaction method and device based on mark and electronic equipment |
| CN108325208A (en) * | 2018-03-20 | 2018-07-27 | 昆山时记信息科技有限公司 | Augmented reality implementation method applied to field of play |
| CN109948592A (en) * | 2019-04-04 | 2019-06-28 | 北京理工大学 | A kind of design idea method of discrimination and system based on hand signal identification |
| CN110070777A (en) * | 2019-06-13 | 2019-07-30 | 大连民族大学 | A kind of Hezhe's fish-skin draws simulation training system and implementation method |
| CN110389664A (en) * | 2019-06-25 | 2019-10-29 | 浙江大学 | A kind of fire scenario sunykatuib analysis device and method based on augmented reality |
| CN112164258A (en) * | 2019-12-26 | 2021-01-01 | 昆山时记信息科技有限公司 | AR intelligent teaching method, device, teaching aid system and computer equipment |
| CN113553884A (en) * | 2020-04-26 | 2021-10-26 | 武汉Tcl集团工业研究院有限公司 | Gesture recognition method, terminal device and computer-readable storage medium |
| CN113012298A (en) * | 2020-12-25 | 2021-06-22 | 浙江大学 | Curved MARK three-dimensional registration augmented reality method based on region detection |
| CN113066189A (en) * | 2021-04-06 | 2021-07-02 | 海信视像科技股份有限公司 | Augmented reality equipment and virtual and real object shielding display method |
| CN113066189B (en) * | 2021-04-06 | 2022-06-14 | 海信视像科技股份有限公司 | Augmented reality equipment and virtual and real object shielding display method |
| CN113961080A (en) * | 2021-11-09 | 2022-01-21 | 南京邮电大学 | Three-dimensional modeling software framework based on gesture interaction and design method |
| CN113961080B (en) * | 2021-11-09 | 2023-08-18 | 南京邮电大学 | Three-dimensional modeling software framework based on gesture interaction and design method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102142055A (en) | True three-dimensional design method based on augmented reality interactive technology | |
| Haban et al. | Global events and global breakpoints in distributed systems | |
| CN103955267B (en) | Both hands man-machine interaction method in x ray fluoroscopy x augmented reality system | |
| Tang et al. | Latent regression forest: structured estimation of 3d hand poses | |
| CN102508867B (en) | Human-motion diagram searching method | |
| CN105528082A (en) | Three-dimensional space and hand gesture recognition tracing interactive method, device and system | |
| Jordao et al. | Crowd sculpting: A space‐time sculpting method for populating virtual environments | |
| Liu et al. | Toward real-world category-level articulation pose estimation | |
| Taylor et al. | Visual perception and robotic manipulation: 3D object recognition, tracking and hand-eye coordination | |
| Malik et al. | Structure-aware 3d hand pose regression from a single depth image | |
| Ho et al. | Topology aware data‐driven inverse kinematics | |
| Le et al. | A survey on 3D hand skeleton and pose estimation by convolutional neural network | |
| Jian et al. | Affordpose: A large-scale dataset of hand-object interactions with affordance-driven hand pose | |
| Akman et al. | Generation of 3D human models and animations using simple sketches | |
| CN112837339B (en) | Track drawing method and device based on motion capture technology | |
| Chen et al. | A Motion‐guided Interface for Modeling 3D Multi‐functional Furniture | |
| Yin | A hierarchical approach to continuous gesture analysis for natural multi-modal interaction | |
| Chen et al. | Dual quaternion based virtual hand interaction modeling | |
| CN110263702A (en) | A kind of real-time three-dimensional gesture method for tracing based on method of geometry | |
| Liu et al. | Massive-scale complicated human action recognition: Theory and applications | |
| Chen et al. | Accurate fingertip detection from binocular mask images | |
| Chen et al. | Optimization of the collision detection technology in 3D skeleton animation | |
| Kavakli et al. | Virtual hand: an interface for interactive sketching in virtual reality | |
| Yang et al. | Hand Tracking Based on the Kinect and Kalman Filter | |
| Lu et al. | A combined strategy of hand tracking for desktop VR |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C02 | Deemed withdrawal of patent application after publication (patent law 2001) | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20110803 |