Summary of the invention
In view of the above problems, it is proposed that the present invention, in order to provide one to overcome the problems referred to above or at least portion
Ground is divided to solve a kind of method and apparatus presenting multimedia messages of the problems referred to above.
According to a first aspect of the present invention, it is provided that a kind of method presenting multimedia messages, including:
Obtain presentation information, that described presentation information includes observing with the naked eye and/or can be with setting
The standby electromagnetic field spectral information for characterizing object collected;
The four-dimensional spacetime model for characterizing described presentation information is set up according to the presentation information got,
Described four-dimensional spacetime model has can characterize time dependent described presentation letter in digitized form
The attribute of breath;
The presentation information that described four-dimensional spacetime model characterizes is presented.
Alternatively, in the method presenting multimedia messages according to the abovementioned embodiments of the present invention, described
Presentation information also includes sound field information that is that can feel and/or that can collect with equipment with ear;
Before setting up described four-dimensional spacetime model according to the presentation information got, described method is also wrapped
Include:
The sound field information of the object corresponding with described presentation information is calculated according to described presentation information;
The four-dimensional spacetime model for characterizing described presentation information is set up, specifically according to described presentation information
Including:
According to described presentation information and described sound field information, set up and be used for characterizing described presentation information and institute
State the four-dimensional spacetime model of sound field information.
Alternatively, in the method presenting multimedia messages according to any of the above-described embodiment of the present invention,
Set up the four-dimensional spacetime model for characterizing described presentation information according to the presentation information got, specifically wrap
Include:
Described presentation information is processed, obtains the first markup information;
According to described first markup information and described presentation information, obtain including first cloud of geological information
Information, include the second point cloud information of texture information;
Described first cloud information and described second point cloud information are merged, obtains impact point cloud letter
Breath;
Visual information is obtained according to described impact point cloud information;
Obtain spatial model according to described visual information, will melt for spatial model the most in the same time
Close;
Space module, described first markup information and described second markup information obtained according to fusion,
To described four-dimensional spacetime model.
Alternatively, in the method presenting multimedia messages according to any of the above-described embodiment of the present invention,
Described presentation information also includes sound field that is that can feel and/or that can collect with equipment with ear
Information;
Before obtaining spatial model according to described visual information, described method also includes:
The sound field information of the object corresponding with described presentation information is calculated according to described presentation information;
Obtain spatial model according to described visual information, specifically include:
Described visual information and described sound field information are merged, obtains described spatial model.
Alternatively, in the method presenting multimedia messages according to any of the above-described embodiment of the present invention,
Described first cloud information and described second point cloud information are merged, after obtaining impact point cloud information,
Before obtaining described visual information, described method also includes:
Described impact point cloud information is processed, obtains the second markup information;
According to described impact point cloud information, obtain visual information, specifically include:
According to described second markup information and described impact point cloud information, obtain described visual information.
Alternatively, in the method presenting multimedia messages according to any of the above-described embodiment of the present invention,
According to described second markup information and described impact point cloud information, obtain described visual information, specifically include:
Described impact point cloud information is carried out geometry vertex position optimization and normal direction calculates, obtain the first knot
Really;
Described first result is carried out surface fitting and triangle gridding processes, obtain the second result;
According to described second result, obtain described visual information.
Alternatively, in the method presenting multimedia messages according to any of the above-described embodiment of the present invention,
Described presentation information is processed, obtains the first markup information, specifically include:
Described presentation information is carried out Digital Image Processing analysis, obtains described first markup information.
Alternatively, in the method presenting multimedia messages according to any of the above-described embodiment of the present invention,
According to described first markup information and described presentation information, obtain including first cloud letter of geological information
Breath, specifically includes:
According to described first markup information, described presentation information is processed, obtain and described presentation information
The coordinate information of corresponding object;
According to described coordinate information, generate first the cloud information including described geological information.
Alternatively, in the method presenting multimedia messages according to any of the above-described embodiment of the present invention,
According to described first markup information and described presentation information, obtain including the second point cloud letter of texture information
Breath, specifically includes:
Use pointwise and/or the mode of image synthesis, according to described first markup information, to described presentation
Information carries out extracting information processing, obtains including the second point cloud information of texture information.
Alternatively, in the method presenting multimedia messages according to any of the above-described embodiment of the present invention,
According to described second markup information and described impact point cloud information, obtain visual information, including:
According to described second markup information and described impact point cloud information, calculate the surface normal letter of object
Breath;
Visual information is obtained according to described surface normal information.
Alternatively, in the method presenting multimedia messages according to any of the above-described embodiment of the present invention,
Before the presentation information that described four-dimensional spacetime model characterizes being presented, described method also includes:
Described four-dimensional spacetime model and the first space-time model are merged, obtains target four-dimensional spacetime mould
Type, described first space-time model is for characterizing the presentation letter of the object presenting place residing for multimedia messages
Breath;
The presentation information that described four-dimensional spacetime model characterizes is presented, specifically includes:
According to described target four-dimensional spacetime model, the presentation information that described four-dimensional spacetime model is characterized
The presentation information characterized with described first space-time model presents.
Alternatively, in the method presenting multimedia messages according to any of the above-described embodiment of the present invention,
Before the presentation information that described four-dimensional spacetime model characterizes being presented, described method also includes:
First space-time model, second space-time model of described four-dimensional spacetime model with local terminal are melted
Closing, obtain target four-dimensional spacetime model, described first space-time model presents multimedia messages for sign
The presentation information of the object in residing place, described second space-time model is for characterizing the presentation of dummy object
Information;
The presentation information that described four-dimensional spacetime model characterizes is presented, specifically includes:
According to described target four-dimensional spacetime model, the presentation letter that described four-dimensional spacetime model is characterized
The table that breath, the presentation information that characterized of described first space-time model and described second space-time model are characterized
Image information presents.
Alternatively, in the method presenting multimedia messages according to any of the above-described embodiment of the present invention,
Described presentation information also includes sound field that is that can feel and/or that can collect with equipment with ear
Information;
Described four-dimensional spacetime model is additionally operable to characterize the sound field letter of the object corresponding with described presentation information
Breath;
Described method also includes:
The sound field information that described four-dimensional spacetime model characterizes is played out.
Alternatively, in the method presenting multimedia messages according to any of the above-described embodiment of the present invention,
Before the presentation information that described four-dimensional spacetime model characterizes is presented, also include:
Determine present multimedia messages device just facing to;
The presentation information that described four-dimensional spacetime model characterizes is presented, including:
According to described just facing to, by described four-dimensional spacetime model characterize presentation information present.
Alternatively, in the method presenting multimedia messages according to any of the above-described embodiment of the present invention,
Described method also includes:
Determine present multimedia messages device just facing to destination multimedia information;
Just feed back to send setting of described presentation information facing to described destination multimedia information by described
Standby.
According to a second aspect of the present invention, it is provided that a kind of device presenting multimedia messages, including:
Acquiring unit, is used for obtaining presentation information, and described presentation information includes observing with the naked eye
And/or the electromagnetic field spectral information for characterizing object that can collect with equipment;
Unit set up by model, for setting up for characterizing described presentation letter according to the presentation information got
The four-dimensional spacetime model of breath, described four-dimensional spacetime model has and can characterize in time in digitized form
The attribute of the described presentation information of change;
Display unit, for presenting the presentation information that described four-dimensional spacetime model characterizes.
Alternatively, in the device presenting multimedia messages according to the abovementioned embodiments of the present invention, described
Presentation information also includes sound field information that is that can feel and/or that can collect with equipment with ear;
Described device also includes sound field information computing unit, for calculating and institute according to described presentation information
State the sound field information of object corresponding to presentation information;
Described model is set up unit and is set up for characterizing described presentation information according to described presentation information
During four-dimensional spacetime model, particularly as follows:
According to described presentation information and described sound field information, set up and be used for characterizing described presentation information and institute
State the four-dimensional spacetime model of sound field information.
Alternatively, in the device presenting multimedia messages according to any of the above-described embodiment of the present invention,
Described model is set up unit and is included the first markup information signal generating unit, some cloud information generating unit, some cloud letter
Breath integrated unit, visual information signal generating unit, four-dimensional spacetime model generation unit, wherein:
Described first markup information signal generating unit, for processing described presentation information, obtains first
Markup information;
Described some cloud information generating unit, is used for according to described first markup information and described presentation information,
Obtain including first cloud information of geological information, including the second point cloud information of texture information;
Described some cloud information fusion unit, for by described first cloud information and described second point cloud information
Merge, obtain impact point cloud information;
Described visual information signal generating unit, for obtaining visual information according to described impact point cloud information;
Described four-dimensional spacetime model generation unit, for obtaining spatial model according to described visual information, will
Merge for spatial model the most in the same time, according to merging the space module obtained, described first mark
Note information and described second markup information, obtain described four-dimensional spacetime model.
Alternatively, in the device presenting multimedia messages according to any of the above-described embodiment of the present invention,
Described presentation information also includes sound field that is that can feel and/or that can collect with equipment with ear
Information;
Described device also includes sound field information computing unit, for calculating described table according to described presentation information
The sound field information of the object that image information is corresponding;
When described four-dimensional spacetime model generation unit obtains spatial model according to described visual information, specifically
For:
Described visual information and described sound field information are merged, obtains described spatial model.
Alternatively, in the device presenting multimedia messages according to any of the above-described embodiment of the present invention,
Described some cloud information generating unit includes the second markup information signal generating unit, for believing described impact point cloud
Breath processes, and obtains the second markup information;
Described visual information signal generating unit is according to described impact point cloud information, when obtaining visual information, specifically
For:
According to described second markup information and described impact point cloud information, obtain described visual information
Alternatively, in the device presenting multimedia messages according to any of the above-described embodiment of the present invention,
Described visual information signal generating unit is additionally operable to:
Described impact point cloud information is carried out geometry vertex position optimization and normal direction calculates, obtain the first knot
Really;
Described first result is carried out surface fitting and triangle gridding processes, obtain the second result;
According to described second result, obtain described visual information.
Alternatively, in the device presenting multimedia messages according to any of the above-described embodiment of the present invention,
Described presentation information is processed by described first markup information signal generating unit, obtains the first markup information
Time, particularly as follows:
Described presentation information is carried out Digital Image Processing analysis, obtains described first markup information.
Alternatively, in the device presenting multimedia messages according to any of the above-described embodiment of the present invention,
Described some cloud information generating unit according to described first markup information and described presentation information, including
During first cloud information of geological information, particularly as follows:
According to described first markup information, described presentation information is processed, obtain and described presentation information
The coordinate information of corresponding object;
According to described coordinate information, generate first the cloud information including described geological information.
Alternatively, in the device presenting multimedia messages according to any of the above-described embodiment of the present invention,
Described some cloud information generating unit according to described first markup information and described presentation information, including
During the second point cloud information of texture information, particularly as follows:
Use pointwise and/or the mode of image synthesis, according to described first markup information, to described presentation
Information carries out extracting information processing, obtains including the second point cloud information of texture information.
Alternatively, in the device presenting multimedia messages according to any of the above-described embodiment of the present invention,
Described visual information signal generating unit according to described second markup information and described impact point cloud information, depending on
During visual information, particularly as follows:
According to described second markup information and described impact point cloud information, calculate body surface normal direction information;
Visual information is obtained according to described surface normal information.
Alternatively, in the device presenting multimedia messages according to any of the above-described embodiment of the present invention,
Described device also includes Model Fusion unit, for by described four-dimensional spacetime model and the first space-time model
Merging, obtain target four-dimensional spacetime model, described first space-time model presents many matchmakers for sign
The presentation information of the object in place residing for body information;
The presentation information that described four-dimensional spacetime model characterizes is being carried out in now by described display unit,
Particularly as follows:
According to described target four-dimensional spacetime model, the presentation information that described four-dimensional spacetime model is characterized
The presentation information characterized with described first space-time model presents.
Alternatively, in the device presenting multimedia messages according to any of the above-described embodiment of the present invention,
Described device also includes Model Fusion unit, for during by the first of described four-dimensional spacetime model and local terminal
Empty model, the second space-time model merge, and obtain target four-dimensional spacetime model, described first space-time
Model is for characterizing the presentation information of the object presenting place residing for multimedia messages, described second space-time
Model is for characterizing the presentation information of dummy object;
The presentation information that described four-dimensional spacetime model characterizes is being carried out in now by described display unit,
Particularly as follows:
According to described target four-dimensional spacetime model, the presentation letter that described four-dimensional spacetime model is characterized
The table that breath, the presentation information that characterized of described first space-time model and described second space-time model are characterized
Image information presents.
Alternatively, in the device presenting multimedia messages according to any of the above-described embodiment of the present invention,
Described presentation information also includes sound field that is that can feel and/or that can collect with equipment with ear
Information;
Described four-dimensional spacetime model is additionally operable to characterize the sound field letter of the object corresponding with described presentation information
Breath;
Described device also includes broadcast unit, for the sound field letter characterized by described four-dimensional spacetime model
Breath plays out.
Alternatively, in the device presenting multimedia messages according to any of the above-described embodiment of the present invention,
Described device also includes processing unit, for determine present multimedia messages device just facing to;
The presentation information that described four-dimensional spacetime model characterizes is being carried out in now by described display unit,
Particularly as follows:
According to described forward towards, by described four-dimensional spacetime model characterize presentation information present.
Alternatively, in the device presenting multimedia messages according to any of the above-described embodiment of the present invention,
Described device also includes processing unit, determine present multimedia messages device just facing to many with target
Media information;
Described device also includes feedback unit, for by described just facing to described destination multimedia information
Feed back to send the equipment of described presentation information.
In the embodiment of the present invention, disclose a kind of method and apparatus presenting multimedia messages: obtain table
Image information is that described presentation information includes observing with the naked eye and/or can collect with equipment
For characterizing the electromagnetic field spectral information of object;Set up for characterizing according to the presentation information got
Stating the four-dimensional spacetime model of presentation information, described four-dimensional spacetime model has can table in digitized form
Levy the attribute of time dependent described presentation information;The presentation that described four-dimensional spacetime model is characterized
Information presents;In this scenario, four-dimensional spacetime model have can characterize in digitized form with
The attribute of the presentation information of time change, so, to some extent solves the presentation information presented
The problem that there is time delay, therefore, this programme to some extent solves tool present in prior art
There is the defect of time delay.
Described above is only the general introduction of technical solution of the present invention, in order to better understand the skill of the present invention
Art means, and can being practiced according to the content of description, and in order to allow, the present invention's is above and other
Objects, features and advantages can become apparent, below especially exemplified by the detailed description of the invention of the present invention.
Detailed description of the invention
In order to make the object, technical solutions and advantages of the present invention clearer, below in conjunction with accompanying drawing to this
Invention is described in further detail, it is clear that described embodiment is only that a part of the present invention is implemented
Example rather than whole embodiments.Based on the embodiment in the present invention, those of ordinary skill in the art exist
Do not make all other embodiments obtained under creative work premise, broadly fall into present invention protection
Scope.
The proposed by the invention method and apparatus processing multimedia messages can be applied in following scene,
But it is not limited to following scene:
Real-time communication scene: such as, first collects real-time Transmission to second self and surrounding,
Second roams and and first interaction wherein;The most such as, first and second are all self and surrounding collection
Real-time Transmission of getting off is to the other side, at the environment or any 3rd including that two people are physically residing alternatively
Roaming and interaction in side's environment;
Remote observation and monitoring scene;
Operative scenario: such as, individual or many people telecommuting, the most immersively meeting, remote
Journey is immersively cooperated or helps client to solve problem, or, immersively train;
Education scene: for example, it is possible to immersively virtual classroom and and teacher in virtual environment
Middle interaction;
Medical treatment scene: such as, tele-medicine and and doctor's interaction in virtual environment;
Business scenario: such as, teleshopping and and businessman interaction, comprehensive examination in virtual environment
Clothing mirror;
Sports Scene: such as, individual or many people and dash champion compete dash in virtual environment;
Entertainment Scene: such as, individual or game that many people is in Virtual Space, can be immersively
Participating television is live or interactive with film appearances;
Personal lifestyle scene: such as, the record of four-dimensional diary and projection, remote visit museum, far
Journey accompanies household or house pet, application of being remotely grown up;
Can also be used for being applied to following scene:
The scene that virtual reality or augmented reality content generate: include film, TV, play, regard
Frequently content production;
Place, special time space is done the scene of four-dimensional historical record.
Figure 1A shows the schematic flow sheet of a kind of method presenting multimedia messages according to the present invention,
Detailed process is as follows:
Step 100: obtain presentation information, that described presentation information includes observing with the naked eye and/
The electromagnetic field spectral information for characterizing object that maybe can collect with equipment.
In the embodiment of the present invention, when step 100 obtains presentation information, can be self to gather to obtain
, it is also possible to it is to send over after other equipment collections, is not specifically limited at this.
In the embodiment of the present invention, the presentation information got can be different spaces point and/or different time
The presentation information that point gets.
Step 110: set up for the four-dimension characterizing described presentation information according to the presentation information got
Space-time model, described four-dimensional spacetime model has can characterize time dependent institute in digitized form
State the attribute of presentation information.
In the embodiment of the present invention, four-dimensional spacetime model the most at least includes as properties:
Locus attribute: can refer to that the every bit on any time object does not changes over time at one
Coordinate system under coordinate;
Appearance attribute: texture and spectral signature (such as, the face of any time body surface can be referred to
Color), the geometrical property (such as, normal direction, curvature, smoothness etc.) of body surface;
Voice attribute;
Movement properties: the movement velocity vector of every bit on any time object, acceleration can be referred to
Vector;Or, it is also possible to refer to regard the angular velocity vector of every part of rigid body on object as, angle adds
Velocity;
Other attribute: can refer to that the classification of object, identity, material, mutual relation etc. are all can be by table
At least one in the information that image information and presentation information are inferred over time.
In form, four-dimensional spacetime model is to be present in storage medium with digitized data mode, this
The form of kind can be stored, and presents, retrieval, editor, transmission, encryption, and is used for higher level
Intelligent use.
In the embodiment of the present invention, after setting up four-dimensional spacetime model, further, it is also possible to amendment,
Strengthen, optimize four-dimensional spacetime model.
In actual applications, presentation information is except that can include observing with the naked eye and/or can be with setting
Standby collect outside the electromagnetic field spectral information characterizing object, it is also possible to include sound field information,
The most described presentation information also includes sound that is that can feel and/or that can collect with equipment with ear
Field information, now, before setting up described four-dimensional spacetime model according to the presentation information got, described
Method also includes operating as follows:
The sound field information of the object corresponding with described presentation information is calculated according to described presentation information;
The four-dimensional spacetime model for characterizing described presentation information is set up, specifically according to described presentation information
Including:
According to described presentation information and described sound field information, set up and be used for characterizing described presentation information and institute
State the four-dimensional spacetime model of sound field information.
Sound field information described in the embodiment of the present invention refers not only to audio-frequency information itself, the most implicit
Sound source spatial positional information inside, can include collectable information of acoustic wave and/or ultrasound information.
In the embodiment of the present invention, set up for characterizing described presentation information according to the presentation information got
During four-dimensional spacetime model, it is alternatively possible in the following way:
Described presentation information is processed, obtains the first markup information;
According to described first markup information and described presentation information, obtain including first cloud of geological information
Information, include the second point cloud information of texture information;
Described first cloud information and described second point cloud information are merged, obtains impact point cloud letter
Breath;
Visual information is obtained according to described impact point cloud information;
Obtain spatial model according to described visual information, will melt for spatial model the most in the same time
Close;
Space module, described first markup information and described second markup information obtained according to fusion,
To described four-dimensional spacetime model.
In the embodiment of the present invention, that described presentation information also includes can feeling with ear and/or can
The sound field information collected with equipment;
Before obtaining spatial model according to described visual information, described method also includes operating as follows:
The sound field information of the object corresponding with described presentation information is calculated according to described presentation information;
In the embodiment of the present invention, when obtaining spatial model according to described visual information, it is alternatively possible to adopt
By following manner:
Described visual information and described sound field information are merged, obtains described spatial model.
In the embodiment of the present invention, described first cloud information and described second point cloud information are merged,
After obtaining impact point cloud information, before obtaining described visual information, described method also includes operating as follows:
Described impact point cloud information is processed, obtains the second markup information;
According to described impact point cloud information, obtain visual information, specifically include:
According to described second markup information and described impact point cloud information, obtain described visual information.
In the embodiment of the present invention, according to described second markup information and described impact point cloud information, obtain institute
When stating visual information, it is alternatively possible in the following way:
Described impact point cloud information is carried out geometry vertex position optimization and normal direction calculates, obtain the first knot
Really;
Described first result is carried out surface fitting and triangle gridding processes, obtain the second result;
According to described second result, obtain described visual information.
In the embodiment of the present invention, described presentation information is processed, when obtaining the first markup information,
It is alternatively possible in the following way:
Described presentation information is carried out Digital Image Processing analysis, obtains described first markup information.
In the embodiment of the present invention, when presentation information is carried out Digital Image Processing analysis, it is alternatively possible to
In the following way:
Split presentation information, detect, follow the tracks of, identification etc. processes.
In the embodiment of the present invention, split, detect, follow the tracks of, identify and there is no clearly between these steps
Ordering relation, for example, it is possible to presentation information is first split, then detect;Can also first detect, then
Segmentation.In order to improve the accuracy of the first markup information obtained, can circulate execution several times segmentation,
Detect, follow the tracks of and identify.Such as, after having performed once to split, detect, follow the tracks of and identify, root
According to current results, then perform at least one to take turns segmentation, detect, follow the tracks of and identify, so can improve essence
Degree.
In the embodiment of the present invention, segmentation can refer to divide the image into as prospect and background, such as, segmentation
Become sky, ground or other, detection can refer to detect pedestrian, detection car plate, tracking can refer to
The arm motion of track people, identification can refer to identify vehicle.
In the embodiment of the present invention, according to described first markup information and described presentation information, including
During first cloud information of geological information, can be in the following way:
According to described first markup information, described presentation information is processed, obtain and described presentation information
The coordinate information of corresponding object;
According to described coordinate information, generate first the cloud information including described geological information.
In the embodiment of the present invention, the most in the same time may with the coordinate information of the object corresponding to presentation information
Corresponding to different coordinate systems, now, in order to improve the accuracy of first the cloud information obtained, obtain
After the coordinate information of the object corresponding to presentation information, it is also possible to different local the most in the same time are sat
The coordinate information of the object under mark system is fused under same coordinate system, then according to being fused to same seat
Coordinate information under mark system, generates first the cloud information including geological information.
In the embodiment of the present invention, according to described first markup information and described presentation information, including
During the second point cloud information of texture information, it is alternatively possible in the following way:
Use pointwise and/or the mode of image synthesis, according to described first markup information, to described presentation
Information carries out extracting information processing, obtains including the second point cloud information of texture information.
In the embodiment of the present invention, according to described second markup information and described impact point cloud information, depending on
During visual information, it is alternatively possible in the following way:
According to described second markup information and described impact point cloud information, calculate the surface normal letter of object
Breath;
Visual information is obtained according to described surface normal information.
The present invention gives the detailed process setting up four-dimensional spacetime model, refering to shown in Figure 1B, according to table
Image information obtains the first markup information and sound field information, and obtains according to presentation information and the first markup information
Some cloud information and second point cloud information, obtain target by first cloud information and second point cloud information fusion
Point cloud information, obtains the second markup information according to impact point cloud information, and carries out several to impact point cloud information
What vertex position optimization and normal direction are calculated the first result, and the first result is carried out surface fitting and triangle
Gridding processes, and obtains the second result, obtains visual information according to the second result and the second markup information,
By visual information and sound field information fusion, obtain spatial model, after being merged by spatial model
Spatial model, spatial model, the first markup information and the second markup information after merging process and obtain four
Dimension space-time model.
Step 120: the presentation information that described four-dimensional spacetime model characterizes is presented.
In the embodiment of the present invention, further, it is also possible to this one end of device of multimedia messages will be presented
Scene present, therefore, further, by described four-dimensional spacetime model characterize presentation letter
Before breath presents, described method also includes operating as follows:
Described four-dimensional spacetime model and the first space-time model are merged, obtains target four-dimensional spacetime mould
Type, described first space-time model is for characterizing the presentation letter of the object presenting place residing for multimedia messages
Breath;
Now, the presentation information that described four-dimensional spacetime model characterizes is carried out in now, alternatively,
Can be in the following way:
According to described target four-dimensional spacetime model, the presentation information that described four-dimensional spacetime model is characterized
The presentation information characterized with described first space-time model presents.
Such as, the scene that the presentation information of four-dimensional spacetime model sign is corresponding is scene by the sea, the
The scene corresponding to presentation information that one space-time model characterizes is the scene in desk office, at this
In the case of Zhong, the scene presented can be the scene of this fusion being seashore before desk.
Further, it is also possible to human body and object detection, follow the tracks of and identify: can be by actual physical district
Territory is superimposed to virtual region, and such as, observer wears the VR helmet in the face of a grassland, his place in reality
There is wall in room, and this time " based on object detection " just can be the information of the wall of actual physical
It is superimposed to the grassland of the VR helmet, presents the translucent walls in a grassland;The most such as, staff detects,
Can the most then virtual hands superposition be entered in four dimension module by the gestures detection of real hands, say, that
Some virtual scenes, therefore, the presentation letter characterized by described four-dimensional spacetime model can also be merged
Before breath presents, described method also includes operating as follows:
First space-time model, second space-time model of described four-dimensional spacetime model with local terminal are melted
Closing, obtain target four-dimensional spacetime model, described first space-time model presents multimedia messages for sign
The presentation information of the object in residing place, described second space-time model is for characterizing the presentation of dummy object
Information;
Now, the presentation information that described four-dimensional spacetime model characterizes is carried out in now, alternatively,
Can be in the following way:
According to described target four-dimensional spacetime model, the presentation letter that described four-dimensional spacetime model is characterized
The table that breath, the presentation information that characterized of described first space-time model and described second space-time model are characterized
Image information presents.
Such as, the scene that the presentation information of four-dimensional spacetime model sign is corresponding is scene by the sea, the
The scene corresponding to presentation information that one space-time model characterizes is the scene in desk office, at this
In the case of Zhong, the scene presented can be the scene of this fusion being seashore before desk, enters one
Step, want on the desk presented, put a basin flower, but reality is the freshest on desk
Colored, at this point it is possible to characterize flower with the second space-time model, and by described four-dimensional spacetime model and basis
First space-time model of end, the second space-time model merge, and obtain target four-dimensional spacetime model, this
The scene presented in the case of Zhong can be to be this that put flower on seashore, and desk before desk
Plant the scene merged.
In the embodiment of the present invention, the scene presented is possible not only to picture, it is also possible to have sound, therefore,
Described presentation information also includes sound field that is that can feel and/or that can collect with equipment with ear
Information, described four-dimensional spacetime model is additionally operable to characterize the sound field letter of the object corresponding with described presentation information
Breath;Now, described method also includes operating as follows:
The sound field information that described four-dimensional spacetime model characterizes is played out.
In the embodiment of the present invention, in order to improve the scene corresponding to the presentation information presented and real scene
Similarity, when presenting the presentation information that four-dimensional spacetime model characterizes, be with reference to presenting multimedia messages
The front orientation information of device, therefore, the presentation information that described four-dimensional spacetime model characterizes is entered
Before row presents, also include operating as follows:
Determine present multimedia messages device just facing to;
Now, the presentation information that described four-dimensional spacetime model characterizes is carried out in now, alternatively,
Can be in the following way:
According to described just facing to, by described four-dimensional spacetime model characterize presentation information present.
In the embodiment of the present invention, when determining the front orientation information of the device presenting multimedia messages, optional
, can be in the following way:
The inertial navigation being associated with the device presenting multimedia messages is carried out attitude algorithm, is presented
The front orientation information of the device of multimedia messages.
Wherein, inertial navigation can be in gyroscope, magnetometer, accelerometer any one or appoint
Meaning combination.
In the embodiment of the present invention, the precision of observer's part interested can be improved selectively,
Further, also include operating as follows:
Determine present multimedia messages device just facing to destination multimedia information;
Just feed back to send setting of described presentation information facing to described destination multimedia information by described
Standby.
Such as, the scene corresponding to presentation information having seabeach, personage, and sailing boat, presenting if held
The eyeball fixes of the user of the device of multimedia messages personage, then using personage as destination multimedia information.
So, the device of four-dimensional spacetime model is sent when obtaining presentation information, it is possible to the only presentation of personage
Information, can not include the presentation information of sailing boat.
In the embodiment of the present invention, when determining destination multimedia information, can be based on presenting multimedia messages
" eyeball " of the photographic head of device determines.
It should be noted that the first space-time model described in the embodiment of the present invention and the second space-time model
Can be that the device presenting multimedia messages pre-builds, it is also possible to be to set up in real time, or,
Can also be that other devices pre-build, or set up in real time and be sent to present multimedia messages
Device, it is not specifically limited at this.
In the embodiment of the present invention, in some scene, can only present what four-dimensional spacetime model characterized
Presentation information, such as, in telecommuting or telecommunication scene, present the device of multimedia messages simply
Want to experience and send " the most long-range " scene that the device of four-dimensional spacetime model is sent, now, only
Present the presentation information that four-dimensional spacetime model characterizes.In some scene, when presenting the four-dimension
On the basis of the presentation information that empty model characterizes, it is also possible to present further the first space-time model or
The presentation information that second space-time model is characterized, presenting this one end of presentation information, also to add some virtual
Stage property, such as, the device presenting multimedia messages not only to experience the device sending four-dimensional spacetime model
The scene sent, also to add virtual item in this scenario, such as, wave conveniently aloft
Draw a blank, or in order to play, add some virtual item and (e.g., send out " lightning " together on hand to go
Hit the stone in scene).
In the embodiment of the present invention, further, it is also possible to by the first markup information and/or the second mark letter
Breath presents.
In the embodiment of the present invention, it is also possible to receive the four-dimensional spacetime model that multiple device sends respectively,
Such as, the presentation information receiving the first four-dimensional spacetime model sign that the first transmitting terminal sends is corresponding
Scene be the Temple of Heaven, receive second transmitting terminal send second four-dimensional spacetime model characterize presentation
Scene corresponding to information is Eiffel Tower, in now can by the Temple of Heaven and Eiffel Tower side by side in
Existing.
The present invention gives the detailed process presenting four-dimensional spacetime model, refering to shown in Fig. 1 C, by the four-dimension
Space-time model, the first space-time model and the second space-time model merge, and obtain target four-dimensional spacetime model, really
Surely front orientation information and the destination multimedia information of the device of multimedia messages are presented, and according to just facing
The presentation information that four-dimensional spacetime model characterizes is presented to information and target four-dimensional spacetime model, and by front
Orientation information and destination multimedia information feed back to send the device of four-dimensional spacetime model.
In the embodiment of the present invention, disclose a kind of method and apparatus presenting multimedia messages: obtain table
Image information is that described presentation information includes observing with the naked eye and/or can collect with equipment
For characterizing the electromagnetic field spectral information of object;Set up for characterizing according to the presentation information got
Stating the four-dimensional spacetime model of presentation information, described four-dimensional spacetime model has can table in digitized form
Levy the attribute of time dependent described presentation information;The presentation that described four-dimensional spacetime model is characterized
Information presents;In this scenario, four-dimensional spacetime model have can characterize in digitized form with
The attribute of the presentation information of time change, so, to some extent solves the presentation information presented
The problem that there is time delay, therefore, this programme to some extent solves tool present in prior art
There is the defect of time delay.
Refering to shown in Fig. 2 A, the embodiment of the present invention also proposes a kind of device presenting multimedia messages,
Including:
Acquiring unit 21, is used for obtaining presentation information, and described presentation information includes detecting by an unaided eye
To and/or the electromagnetic field spectral information for characterizing object that can collect with equipment;
Unit 22 set up by model, for setting up for characterizing described presentation according to the presentation information got
The four-dimensional spacetime model of information, described four-dimensional spacetime model has and can characterize at any time in digitized form
Between the attribute of described presentation information of change;
Display unit 23, for presenting the presentation information that described four-dimensional spacetime model characterizes.
In the embodiment of the present invention, the electromagnetic field spectral information acquired in acquiring unit 21 can be object
Launch, it is also possible to be object reflection, or can also be object refraction, do not do at this concrete
Limit.
In the embodiment of the present invention, the electromagnetic field spectral information described by acquiring unit 21 can include nothing
Line wave information, infrared ray information, visible ray information, ultraviolet information, X-ray information, and gal
At least one in horse ray information, wherein, it is seen that optical information can include laser.
In the embodiment of the present invention, object corresponding to presentation information can be indoor and/or outdoor arbitrary regard
Wild size and the object of angle.
In the embodiment of the present invention, when acquiring unit 21 obtains presentation information, per second obtain 24 frames extremely
120 frames.
In the embodiment of the present invention, the presentation information accessed by acquiring unit 21 can be different spaces
The presentation information that point and different time points get.
In the embodiment of the present invention, four-dimensional spacetime model the most at least includes as properties:
Locus attribute: can refer to that the every bit on any time object does not changes over time at one
Coordinate system under coordinate;
Appearance attribute: texture and spectral signature (such as, the face of any time body surface can be referred to
Color), the geometrical property (such as, normal direction, curvature, smoothness etc.) of body surface;
Voice attribute;
Movement properties: can refer to that the movement velocity vector of every bit on any time object, acceleration are vowed
Amount;Or, it is also possible to refer to regard the angular velocity vector of every part of rigid body on object as, angle is accelerated
Degree vector;
Other attribute: can refer to that the classification of object, identity, material, mutual relation etc. are all can be by table
At least one in the information that image information and presentation are inferred over time.
In form, four-dimensional spacetime model is to be present in storage medium with digitized data mode, this
The form of kind can be stored, and presents, retrieval, editor, transmission, encryption, and is used for higher level
Intelligent use.
In the embodiment of the present invention, model is set up after unit 22 sets up four-dimensional spacetime model, further
, it is also possible to revise, strengthen, optimize four-dimensional spacetime model.
In actual applications, presentation information is except that can include observing with the naked eye and/or can be with setting
Standby collect outside the electromagnetic field spectral information characterizing object, it is also possible to include sound field information,
Now, in the embodiment of the present invention, further, described presentation information also includes to feel with ear
To and/or the sound field information that can collect with equipment;
Described device also includes sound field information computing unit 24, for according to described presentation information calculate with
The sound field information of the object that described presentation information is corresponding;
Described model is set up unit 22 and is set up for characterizing described presentation information according to described presentation information
Four-dimensional spacetime model time, particularly as follows:
According to described presentation information and described sound field information, set up and be used for characterizing described presentation information and institute
State the four-dimensional spacetime model of sound field information.
Sound field information described in the embodiment of the present invention refers not only to audio-frequency information itself, the most implicit
Sound source spatial positional information inside, can include collectable information of acoustic wave and/or ultrasound information.
In the embodiment of the present invention, alternatively, described model is set up unit 22 and is included that the first markup information is raw
Become unit 22A, some cloud information generating unit 22B, some cloud information fusion unit 22C, visual information raw
Become unit 22D, four-dimensional spacetime model generation unit 22E, wherein:
Described first markup information signal generating unit 22A, for processing described presentation information, obtains
First markup information;
Described some cloud information generating unit 22B, for believing according to described first markup information and described presentation
Breath, obtains including first cloud information of geological information, including the second point cloud information of texture information;
Described some cloud information fusion unit 22C, for by described first cloud information and described second point cloud
Information merges, and obtains impact point cloud information;
Described visual information signal generating unit 22D, for obtaining visual information according to described impact point cloud information;
Described four-dimensional spacetime model generation unit 22E, for obtaining spatial model according to described visual information,
To merge for spatial model the most in the same time, according to fusion obtain space module, described first
Markup information and described second markup information, obtain described four-dimensional spacetime model.
Further, described presentation information also include can feeling with ear and/or equipment can be used
The sound field information collected;
Described device also includes sound field information computing unit 24, described for calculating according to described presentation information
The sound field information of the object that presentation information is corresponding;
When described four-dimensional spacetime model generation unit 22E obtains spatial model according to described visual information, tool
Body is:
Described visual information and described sound field information are merged, obtains described spatial model.
Further, described some cloud information generating unit 22B includes the second markup information signal generating unit 25,
For described impact point cloud information is processed, obtain the second markup information;
Now, described visual information signal generating unit 22D, according to described impact point cloud information, obtains vision letter
During breath, particularly as follows:
According to described second markup information and described impact point cloud information, obtain described visual information.
Further, described visual information signal generating unit 22D is additionally operable to:
Described impact point cloud information is carried out geometry vertex position optimization and normal direction calculates, obtain the first knot
Really;
Described first result is carried out surface fitting and triangle gridding processes, obtain the second result;
According to described second result, obtain described visual information.
Alternatively, described presentation information is processed by described first markup information signal generating unit 22A,
When obtaining the first markup information, particularly as follows:
Described presentation information is carried out Digital Image Processing analysis, obtains described first markup information.
In the embodiment of the present invention, described presentation information is carried out by described first markup information signal generating unit 22A
When Digital Image Processing is analyzed, split presentation information, detect, follow the tracks of, identification etc. processes.
In the embodiment of the present invention, split, detect, follow the tracks of, identify and there is no clearly between these steps
Ordering relation, for example, it is possible to presentation information is first split, then detect;Can also first detect, then
Segmentation.In order to improve the accuracy of the first markup information obtained, can circulate execution several times segmentation,
Detect, follow the tracks of and identify.Such as, after having performed once to split, detect, follow the tracks of and identify, root
According to current results, then perform at least one to take turns segmentation, detect, follow the tracks of and identify, so can improve essence
Degree.
In the embodiment of the present invention, segmentation can refer to divide the image into as prospect and background, such as, segmentation
Become sky, ground or other, detection can refer to detect pedestrian, detection car plate, tracking can refer to
The arm motion of track people, identification can refer to identify vehicle.
In the embodiment of the present invention, described some cloud information generating unit 22B is according to described first markup information
With described presentation information, when obtaining first the cloud information including geological information, particularly as follows:
According to described first markup information, described presentation information is processed, obtain and described presentation information
The coordinate information of corresponding object;
According to described coordinate information, generate first the cloud information including described geological information.
In the embodiment of the present invention, the most in the same time may with the coordinate information of the object corresponding to presentation information
Corresponding to different coordinate systems, now, in order to improve the accuracy of first the cloud information obtained, obtain
After the coordinate information of the object corresponding to presentation information, some cloud information generating unit 22B can also be by
The coordinate information of the object under different local coordinate systems the most in the same time is fused under same coordinate system, so
Afterwards according to being fused to the coordinate information under same coordinate system, generate first the cloud letter including geological information
Breath.
Alternatively, described some cloud information generating unit 22B is according to described first markup information and described table
Image information, when obtaining the second point cloud information including texture information, particularly as follows:
Use pointwise and/or the mode of image synthesis, according to described first markup information, to described presentation
Information carries out extracting information processing, obtains including the second point cloud information of texture information.
Alternatively, described visual information signal generating unit 22D is according to described second markup information and described target
Point cloud information, when obtaining visual information, particularly as follows:
According to described second markup information and described impact point cloud information, calculate body surface normal direction information;
Visual information is obtained according to described surface normal information.
In the embodiment of the present invention, alternatively, described device also includes Model Fusion unit 26, and being used for will
Described four-dimensional spacetime model and the first space-time model merge, and obtain target four-dimensional spacetime model, institute
State the first space-time model for characterizing the presentation information of the object presenting place residing for multimedia messages;
Now, the presentation information that described four-dimensional spacetime model characterizes is being entered by described display unit 23
Row in now, particularly as follows:
According to described target four-dimensional spacetime model, the presentation information that described four-dimensional spacetime model is characterized
The presentation information characterized with described first space-time model presents.
Such as, the scene that the presentation information of four-dimensional spacetime model sign is corresponding is scene by the sea, the
The scene corresponding to presentation information that one space-time model characterizes is the scene in desk office, at this
In the case of Zhong, the scene that display unit 23 presents can be to be this fusion on seashore before desk
Scene.
Further, it is also possible to human body and object detection, follow the tracks of and identify: can be by actual physical district
Territory is superimposed to virtual region, and such as, observer wears the VR helmet in the face of a grassland, his place in reality
There is wall in room, and this time " based on object detection " just can be the information of the wall of actual physical
It is superimposed to the grassland of the VR helmet, presents the translucent walls in a grassland;The most such as, staff detects,
Can the most then virtual hands superposition be entered in four dimension module by the gestures detection of real hands, say, that
Can also merge some virtual scenes, described device also includes Model Fusion unit 26, for by institute
The first space-time model, the second space-time model of stating four-dimensional spacetime model and local terminal merge, and obtain mesh
Mark four-dimensional spacetime model, described first space-time model presents place residing for multimedia messages for characterizing
The presentation information of object, described second space-time model is for characterizing the presentation information of dummy object.
Now, the presentation information that described four-dimensional spacetime model characterizes is being entered by described display unit 23
Row in now, particularly as follows:
According to described target four-dimensional spacetime model, the presentation letter that described four-dimensional spacetime model is characterized
The table that breath, the presentation information that characterized of described first space-time model and described second space-time model are characterized
Image information presents.
Such as, the scene that the presentation information of four-dimensional spacetime model sign is corresponding is scene by the sea, the
The scene corresponding to presentation information that one space-time model characterizes is the scene in desk office, at this
In the case of Zhong, the scene that display unit 23 presents can be to be this fusion on seashore before desk
Scene, further, want on the desk presented, put a basin flower, but in reality handle official business
Flower is not had, at this point it is possible to characterize flower with the second space-time model, and by the described four-dimension on table
Space-time model merges with the first space-time model, second space-time model of local terminal, obtains target four-dimensional
Space-time model, the scene that display unit 23 presents in this case can be before desk to be seashore,
And on desk, put the scene of this fusion of flower.
In the embodiment of the present invention, the scene presented is possible not only to picture, it is also possible to have sound, therefore,
Described presentation information also includes sound field that is that can feel and/or that can collect with equipment with ear
Information;
Described four-dimensional spacetime model is additionally operable to characterize the sound field letter of the object corresponding with described presentation information
Breath;
Described device also includes broadcast unit 27, for the sound field characterized by described four-dimensional spacetime model
Information plays out.
In the embodiment of the present invention, in order to improve the scene corresponding to the presentation information presented and real scene
Similarity, display unit 23, be with reference to presenting when presenting the presentation information that four-dimensional spacetime model characterizes
The front orientation information of the device of multimedia messages, therefore, further, described device also includes processing
Unit 28, for determine present multimedia messages device just facing to;
Now, the presentation information that described four-dimensional spacetime model characterizes is being carried out by described display unit 23
In now, particularly as follows:
According to described forward towards, by described four-dimensional spacetime model characterize presentation information present.
In the embodiment of the present invention, processing unit 28 determine present multimedia messages device just facing to letter
During breath, optionally, can be in the following way:
The inertial navigation being associated with the device presenting multimedia messages is carried out attitude algorithm, is presented
The front orientation information of the device of multimedia messages.
Wherein, inertial navigation can be in gyroscope, magnetometer, accelerometer any one or appoint
Meaning combination.
In the embodiment of the present invention, the precision of observer's part interested can be improved selectively,
Further, described device also includes processing unit 28, determines and is just presenting the device of multimedia messages
Facing to destination multimedia information;
Described device also includes feedback unit 29, for just believing described facing to described destination multimedia
Breath feeds back to send the equipment of described presentation information.
Such as, the scene corresponding to presentation information having seabeach, personage, and sailing boat, presenting if held
The eyeball fixes of the user of the device of multimedia messages personage, then using personage as destination multimedia information.
So, the device of four-dimensional spacetime model is sent when obtaining presentation information, it is possible to the only presentation of personage
Information, can not include the presentation information of sailing boat.
In the embodiment of the present invention, when processing unit 28 determines destination multimedia information, can be many based on presenting
" eyeball " of the photographic head of the device of media information determines.
It should be noted that the first space-time model described in the embodiment of the present invention and the second space-time model
Can be that the device presenting multimedia messages pre-builds, it is also possible to be to set up in real time, or,
Can also be that other devices pre-build, or set up in real time and be sent to present multimedia messages
Device, it is not specifically limited at this.
In the embodiment of the present invention, in some scene, display unit 23 can only present four-dimensional spacetime mould
The presentation information that type is characterized, such as, in telecommuting or telecommunication scene, presents multimedia messages
Device just hope and experience " the most long-range " field that sent of device sending four-dimensional spacetime model
Scape, now, only presents the presentation information that four-dimensional spacetime model characterizes.In some scene,
Display unit 23 is on the basis of presenting the presentation information that four-dimensional spacetime model characterizes, it is also possible to enter
The presentation information that one step presents the first space-time model or the second space-time model is characterized, presents presentation letter
Ceasing this one end and also to add some virtual item, such as, the device presenting multimedia messages not only wants body
Issue after examination and approval the scene sending the device of four-dimensional spacetime model to be sent, also to add virtual road in this scenario
Tool, such as, wave conveniently and aloft draws a blank, or in order to play, add some virtual item
(e.g., send out " lightning " together on hand and remove the stone hitting in scene).
In the embodiment of the present invention, acquiring unit 21 is when other devices obtain presentation information, it is also possible to
Receive the presentation information that multiple device sends respectively, such as, receive the table that the first transmitting terminal sends
The scene that image information is corresponding is the Temple of Heaven, receives the scene that the presentation information of the second transmitting terminal transmission is corresponding
For Eiffel Tower, in the Temple of Heaven and Eiffel Tower being presented side by side now.
In the embodiment of the present invention, acquiring unit 21 can be to include column type, cuboid, prismatic, ring
Type, spherical, and any one shape in hemispherical, including at least one photographic head, photographic head is permissible
For color photographic head, depth camera or infrared camera.
Further, acquiring unit 21 can also include at least one mike, as shown in Fig. 2 B, 2C,
Wherein, Fig. 2 D is Fig. 2 B or the top view of Fig. 2 C, and Fig. 2 E is Fig. 2 B or the side-looking of Fig. 2 C
Figure.
Alternatively, acquiring unit 21 includes 8 pairs of color photographic head and 8 mikes, wherein: top has
1 pair of color photographic head, visual angle are respectively 180 degree;Side has 6 pairs of color photographic head, visual angles to be respectively 70 degree;
Bottom has 1 pair of color photographic head, visual angle to be respectively 180 degree;1 Mike is had in the middle of every 1 pair of photographic head
Wind.
Alternatively, acquiring unit 21 can also be following form:
1 or 1 pair of color photographic head are arranged at top, and visual angle is 45~180 degree;Side have 2 or
Person 8 is to color photographic head, and visual angle is respectively 45~180 degree;1 or 1 pair of color shooting are arranged at bottom
Head, visual angle is 45~180 degree;There is 1 mike, or, have 1 in the middle of every 1 pair of photographic head
Individual mike, alternatively, the quantity of mike is between 1~8.
In the embodiment of the present invention, alternatively, the photographic head at top can also be three-dimensional camera, multifocal
One in photographic head, structure light video camera head, flight time (ToF) photographic head, light field photographic head group or
Person's combination in any.
In the embodiment of the present invention, alternatively, the photographic head of side can be that three-dimensional camera, many focal lengths are taken the photograph
As the one in head, structure light video camera head, flight time (ToF) photographic head, light field photographic head group or appoint
Meaning combination.
Such as, acquiring unit 21 is column type, has six pairs of binocular camera at cylindrical side surface, each
Camera view is 70 degree;It is respectively arranged with a pair binocular camera, each shooting at cylinder end face and bottom surface
The head visual field is 180 degree, it is possible to achieve the visual field of stereoscopic full views covers, and all photographic head are all through thing
First calibration has also obtained parameter matrix.Acquiring unit 21 can be with built-in eight mikes.
In the embodiment of the present invention, color photographic head can by optical lens, sensor devices (Image Sensor),
ISP (Image Signal Processing Unit, image signal processing chip) forms.
VPU (Vision Processing Unit, vision processor) can include that unit 22 set up by model
With processing unit 23, wherein, photographic head can be by MIPI (Mobile Industry Processor
Interface, mobile Industry Processor Interface) it is connected to VPU chip, it is right that VPU chip processes two
The data that photographic head is transmitted through, are four VPU chips inside such a cylinder.
In the embodiment of the present invention, model is set up unit 22 and can be included processor, video card, internal memory, show
Deposit, flash memory, hard disk, be wirelessly transferred, wire transmission and multiple Bus Interface Chip.
Shown below is the scene that the embodiment of the present invention is suitable for.
Scene shown in Fig. 3 A is: first is in the first scene, and second is in the second scene, and first and second are passed through
The method that present example provides makes the surrounding of first and first " remotely present " face in second in real time
Before, second can carry out interaction with first.
Further, four-dimensional spacetime model can also be first stored in storage and sets by device for processing muti-medium information
In Bei, the device that can receive and process four-dimensional spacetime model that second is held can obtain from storage device
Take four-dimensional spacetime model, as shown in Figure 3 B, in this case, scene that second institute " sees " and figure
The scene seen in the case of shown in 3A can be the same.
When four-dimensional spacetime model is stored in storage device by device for processing muti-medium information, first can also
Hold the device that can receive and process four-dimensional spacetime model, from storage device, obtain four-dimensional spacetime model,
Perceive oneself in the first scene residing for one time point of past, as shown in Figure 3 C.
Scene shown in Fig. 3 D is: first is in the first scene, and second is in the second scene, and first and second are passed through
Present example one makes the surrounding of first and first " remotely present " in face of second in real time, and second can
To carry out interaction with first;First and second realize two-way " remotely reality " in real time by present example one and " mix
Close reality ", what first was perceived is the first scene and the superposition of second at first place, and what second was perceived is
First and first scene at first place;It should be noted that first and second are all right for the scene of wanted perception
Having multiple choices, both sides can select the first scene seeing first place can also select to see second place
Second scene, or see the 3rd scene at other side place;First and second it can be seen that same true or
Person's virtual scene is it can also be seen that different true or virtual scene.
Scene shown in Fig. 3 E is: first realizes telecommuting by embodiment provided by the present invention.
Scene shown in Fig. 3 F is: first and second can realize sense by embodiment provided by the present invention
To virtual environment, further, it is also possible to realize interaction, as " on the spot in person ".
Method and apparatus is not solid with any certain computer, virtual system or miscellaneous equipment provided herein
Have relevant.Various general-purpose systems can also be used together with based on example in this.As described above,
Construct the structure required by this kind of device to be apparent from.Additionally, the present invention is also not for any specific
Programming language.It is understood that, it is possible to use various programming languages realize the content of invention described herein,
And the description done language-specific above is the preferred forms in order to disclose the present invention.
In description mentioned herein, illustrate a large amount of detail.It is to be appreciated, however, that this
Inventive embodiment can be put into practice in the case of not having these details.In some instances, not
It is shown specifically known method, structure and technology, in order to do not obscure the understanding of this description.
Similarly, it will be appreciated that in order to simplify the disclosure and help understand in each inventive aspect one
Or multiple, above in the description of the exemplary embodiment of the present invention, each feature of the present invention is sometimes
It is grouped together in single embodiment, figure or descriptions thereof.But, should be by the disclosure
Method be construed to reflect an intention that i.e. the present invention for required protection require ratio in each claim
The middle more feature of feature be expressly recited.More precisely, as the following claims reflect,
Inventive aspect is all features less than single embodiment disclosed above.Therefore, it then follows be embodied as
Claims of mode are thus expressly incorporated in this detailed description of the invention, the most each claim itself
All as the independent embodiment of the present invention.
Those skilled in the art are appreciated that and can carry out the module in the device in embodiment certainly
Change adaptively and they are arranged in one or more devices different from this embodiment.Permissible
Some block combiner in embodiment are become a module or unit or assembly, and in addition can be them
It is divided into multiple submodule or subelement or sub-component.Except in such feature and/or process or module
Outside at least some excludes each other, can use any combination that (this specification is included adjoint right
Require, summary and accompanying drawing) disclosed in all features and so disclosed any method or equipment
All processes or unit are combined.Unless expressly stated otherwise, this specification (includes adjoint right
Require, summary and accompanying drawing) disclosed in each feature can by provide identical, equivalent or similar purpose replace
Replace for feature.
Although additionally, it will be appreciated by those of skill in the art that embodiments more described herein include it
Some feature included in its embodiment rather than further feature, but the group of the feature of different embodiment
Close and mean to be within the scope of the present invention and formed different embodiments.Such as, in claim
In book, one of arbitrarily can mode using in any combination of embodiment required for protection.
Each device embodiment of the present invention can realize with hardware, or to process at one or more
The software module run on device realizes, or realizes with combinations thereof.Those skilled in the art should
Understand, microprocessor or digital signal processor (DSP) can be used in practice to realize basis
The some or all functions of the some or all modules in the device of the embodiment of the present invention.The present invention is also
Can be implemented as part or all the device program for performing method as described herein (such as,
Computer program and computer program).The program of such present invention of realization can be stored in calculating
On machine computer-readable recording medium, or can be to have the form of one or more signal.Such signal can be from
Download on internet website and obtain, or provide on carrier signal, or provide with any other form.
The present invention will be described rather than limits the invention to it should be noted above-described embodiment,
And those skilled in the art can design replacement in fact without departing from the scope of the appended claims
Execute example.In the claims, should not will be located in any reference marks between bracket to be configured to right is wanted
The restriction asked.Word " comprises " and does not excludes the presence of the element or step not arranged in the claims.It is positioned at unit
Word "a" or "an" before part does not excludes the presence of multiple such element.The present invention can be by means of
Include the hardware of some different elements and realize by means of properly programmed computer.Listing
If in the unit claim of equipment for drying, several in these devices can be by same hardware branch
Specifically embody.Word first, second and third use do not indicate that any order.Can be by this
A little word explanations are title.