CN106383655A - Interaction control method for controlling visual angle conversion in panorama playing process, and device for realizing interaction control method - Google Patents

Interaction control method for controlling visual angle conversion in panorama playing process, and device for realizing interaction control method Download PDF

Info

Publication number
CN106383655A
CN106383655A CN201610831446.8A CN201610831446A CN106383655A CN 106383655 A CN106383655 A CN 106383655A CN 201610831446 A CN201610831446 A CN 201610831446A CN 106383655 A CN106383655 A CN 106383655A
Authority
CN
China
Prior art keywords
gyroscope
user gesture
gesture
mobile device
control method
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201610831446.8A
Other languages
Chinese (zh)
Inventor
魏博
王玮
曹淼
候甲
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Small Mutual Entertainment Technology Co Ltd
Original Assignee
Beijing Small Mutual Entertainment Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Small Mutual Entertainment Technology Co Ltd filed Critical Beijing Small Mutual Entertainment Technology Co Ltd
Priority to CN201610831446.8A priority Critical patent/CN106383655A/en
Publication of CN106383655A publication Critical patent/CN106383655A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • G06F3/04883—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures for inputting data by handwriting, e.g. gesture or text
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G06F3/04815—Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G06T19/20—Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2219/00—Indexing scheme for manipulating 3D models or images for computer graphics
    • G06T2219/20—Indexing scheme for editing of 3D models
    • G06T2219/2016—Rotation, translation, scaling

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Architecture (AREA)
  • Computer Graphics (AREA)
  • Computer Hardware Design (AREA)
  • Software Systems (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

The invention discloses an interaction control method for controlling visual angle conversion in a panorama playing process, and a device for realizing the interaction control method. In the implementation way of the invention, the method comprises the following steps of: obtaining the rendering model of a player in mobile equipment; rotating the rendering model to enable the rendering model to be synchronized with the initial horizontal position of a gyroscope in mobile equipment; collecting the information of the gyroscope and a user gesture; according to the information collected from the gyroscope and the user gesture, carrying out visual angle conversion to obtain synthesis offset data corresponding to the user gesture and the gyroscope; and on the basis of the initial horizontal position and the synthesis offset data, carrying out rotation transformation on the rendering model. By use of the implementation way, on the basis of the user gesture and the gyroscope, the angle of a window is controlled so as to improve the user experience of the panorama playing.

Description

Control interaction control method and the device of visual angle effect in panorama playing process
Technical field
The present invention relates to the interaction control method in panorama playing process is and in particular to be used for controlling in panorama playing process The interaction control method of visual angle effect processed, more particularly, to complete based on gyroscope and gesture Sliding Control on mobile terminals The interaction control method of the visual angle effect during scape player plays and the device realizing the method.
Background technology
With the continuous development of virtual reality (VR, virtual reality) industry, the demand play for panoramic video Increasing.For in the narrow sense, panorama refers to 360 panoramas, and that is, visual angle exceedes the image of the normal viewing angle of people, and such as level regards Angle is the image that 360 degree and vertical angle of view are 180 degree.From broadly, panorama refers to surroundings are closed with certain geometry System carries out mapping the plane picture generating, and only just can become three-dimensional panorama by the correction process of panorama player, and not limit In specific angle.
Compared with conventional video, panoramic video can for example be obtained in three dimensions by rotating to spherical video Arbitrary angle of visual field, thus provide a user with experience comprehensive, true, on the spot in person when viewing and admiring；However, in order to aphorama The visual angle of frequency switches over, and panoramic video needs corresponding man-machine interaction mode in playing process.
At present, panorama player mainly adopts two ways on exchange method：Rely on the gyroscope that mobile device carries Interact, and/or be controlled by the slip gesture of user.
However, individually all being come with some shortcomings using one of aforesaid way.For example, when only being entered using gyroscope mode During row interactive controlling, if the user while the video pictures of rear or top are seen in expectation during viewing video, then must rotate shifting Dynamic equipment makes it point to the picture of rear or top, so that watching process underaction；When only being slided using user gesture When interacting control, the good picture that gyroscope brought can be realized and follow effect, thus affecting the viewing of user Experience.
Content of the invention
For above-mentioned deficiency, the present invention is provided and is regarded based on gyroscope and gesture slip co- controlling in panorama playing process The interaction control method of angle conversion and the device realizing the method.
In a first aspect, the present invention is provided to controlling the interaction control method of visual angle effect in panorama playing process, should Interaction control method may include following steps：Obtain the rending model of player in mobile device；Rotated rendering model make its with In mobile device, the original horizontal position of gyroscope is synchronous；Collection gyroscope and the information of user gesture；According to from gyroscope and The information execution visual angle effect of user gesture collection, to obtain synthesis offset numbers corresponding with both user gesture and gyroscope According to；And based on original horizontal position and synthesis offset data, rending model is rotated.
In the embodiment of this aspect, the information of collection gyroscope and user gesture may include：Carried using mobile device For motor control object gather mobile device in gyroscope spatial rotational data；And the gesture using mobile device offer Control object collection acts on the gesture slip data of the user gesture on the screen of mobile device.In further embodiment In, the spatial rotational data of gyroscope can be Euler angle in X-axis, Y-axis and Z-direction for the gyroscope.
In some optional embodiments, can wrap according to from the information execution visual angle effect of gyroscope and user gesture collection Include：The gesture slip data conversion of user gesture is become form angle rotation amount corresponding with user gesture；And will be with user With gyroscope, the Euler angle in X-axis, Y-axis and Z-direction carries out adduction computing to gesture corresponding form angle rotation amount, with Obtain synthesis offset data corresponding with both user gesture and gyroscope.
In some optional embodiments, the gesture slip data of user gesture may include previous in X-axis and Y direction Coordinate figure Px1And Py1And present coordinate values Px2And Py2；Form angle rotation amount corresponding with user gesture may include mapping Radian radX and radY in X-axis and Y direction afterwards；The gesture slip data conversion of user gesture is become corresponding with user gesture Form angle rotation amount can be realized by equation 1 below：
DisX=(Px2–Px1)×k1
DisY=(Py2–Py1)×k1
RadX=disX × overture/k2
RadY=disY × overture/k2
Wherein, disX and disY is respectively side-play amount in X-axis and Y direction for the user gesture, and overture is visual The angle of window, k1For amplitude constant, and k2For angle constant.
In some optional embodiments, synthesis offset data corresponding with both user gesture and gyroscope may include wash with watercolours Dye model is with respect to direction of rotation RollX, RollY of X-axis, Y-axis and Z axis and RollZ；Will form angle corresponding with user gesture With gyroscope, the Euler angle in X-axis, Y-axis and Z-direction is carried out adduction computing and can be realized by following degree rotation amount,
When mobile device is in portrait layout configuration, can be realized by equation 2 below：
RollX=roll
RollY=-pitch+radX
RollZ=-yaw+radY
When mobile device is in transverse screen configuration, can be realized by equation 3 below：
RollX=roll+radX
RollY=-pitch
RollZ=-yaw+radY
Wherein, roll, pitch and yaw are space Euler angle at original horizontal position for the gyroscope.
In some optional embodiments, amplitude constant k1Can be 0.005, angle constant k2Can be 100.
In some optional embodiments, based on original horizontal position and synthesis offset data, rending model is rotated Conversion may include：According to synthesis offset data corresponding with both user gesture and gyroscope, by rending model from initial level Position proceeds by rotation transformation.
In some optional embodiments, mobile device can be smart mobile phone, for example iPhone mobile phone.
In some optional embodiments, mobile device can be tablet PC, such as iPad.
Second aspect, the present invention is provided to controlling the device of visual angle effect in panorama playing process, this device can wrap Include model read module, model read module, visual angle effect module and rotation transform module, wherein model read module obtains and moves The rending model of player in dynamic equipment, and rotated rendering model makes its original horizontal position with gyroscope in mobile device Synchronous；Information acquisition module gathers the information of gyroscope and user gesture；Visual angle effect module is according to from gyroscope and user's handss The information execution visual angle effect of gesture collection, to obtain synthesis offset data corresponding with both user gesture and gyroscope；And Rotation transform module is based on original horizontal position and synthesis offset data carries out rotation transformation to rending model.
In some optional embodiments, information acquisition module can further include gyroscope sensing unit and gesture induction Unit, wherein, gyroscope sensing unit utilizes the motor control object that mobile device provides to gather gyroscope in mobile device Spatial rotational data；And gesture induction unit acts on mobile device using the gesture control object collection that mobile device provides Screen on user gesture gesture slip data.
In some optional embodiments, visual angle effect module can further include Date Conversion Unit and arithmetic element, Wherein, the gesture slip data conversion of user gesture is become form angle rotation corresponding with user gesture by Date Conversion Unit Amount；And arithmetic element by form angle rotation amount corresponding with user gesture with gyroscope in mobile device spatial rotational number According to carrying out adduction computing, to obtain synthesis offset data corresponding with both user gesture and gyroscope.
Disclosed by the invention based on the visual angle effect during gyroscope and gesture Sliding Control panorama player plays Interaction control method and realize the device of the method and may be adapted to hold on the mobile device of such as smart phone or panel computer OK.Using technical scheme, user being capable of free switching viewport side when using panorama player viewing panoramic video To without the direction of the equipment that is completely dependent on.User can use handss in combination or arbitraryly according to the viewing angle of oneself Gesture sliding type and/or rotate the mode of mobile phone and adjust video window to the position of suitable viewing, thus obtain desirable complete Scape video viewing experience.
Brief description
In conjunction with accompanying drawing, by the detailed description of following non-limiting embodiment, the further feature of the present invention, purpose and excellent Point will become apparent from.In the accompanying drawings：
Fig. 1 is the schematic diagram of the workflow of panorama player；
Fig. 2 is to be used for controlling the interaction control method of visual angle effect in panorama playing process according to embodiment of the present invention Indicative flowchart；
Fig. 3 is the flow chart of the panorama playing process of the specific embodiment according to Fig. 2 illustrated embodiment；
Fig. 4 is the flow chart of the visual angle effect process of the specific embodiment of panorama playing process according to Fig. 3；And
Fig. 5 is used for controlling the device of visual angle effect in panorama playing process according to embodiment of the present invention.
Specific embodiment
Describe the present invention further with reference to the accompanying drawings and examples.
It should be understood that specific embodiment described herein is only used for explaining related invention, and do not limit its scope.In order to just Only illustrate in description, accompanying drawing and invent related part.It is further to note that without departing from the scope of the present invention and In the case of not conflicting, the feature in embodiments of the invention and embodiment can be mutually combined.
Fig. 1 is the schematic diagram of the workflow of panorama player.This panorama player can be the movement based on iOS system The panorama player of equipment, such as the panorama player in iPhone or iPad, but not limited to this.For example, panorama player It can also be the player of the mobile device based on android system.
It should be noted that herein in the whole text using term " panorama " should understand from the angle of broad sense, that is, refer to by with Certain geometrical relationship by surroundings be mapped to plane graph and via panorama player carry out that correction process formed three-dimensional complete Scape, it can be 360 degree of panoramas, but is not limited to this.
As shown in figure 1, generally, the workflow 100 of panorama player is broadly divided into six steps：Video acquisition 102, conjunction Become process 104, compression storage 106, read decoding 108, model rendering 110 and interactive controlling 112.In this six steps, Video acquisition 102, synthesis process 104 and compression storage 106 composition video source Making programme, and read decoding 108, model rendering 110 and interactive controlling 112 constitute video-see flow process.
In an embodiment, above-mentioned workflow 100 can be realized especially by following：Using multi-cam equipment or super wide Angle photographic head collection video image (that is, video acquisition 102)；Row operation is entered thus closing to the video image being gathered by software Become panoramic picture (synthesizing process 104)；The panoramic picture of synthesis is compressed compiling as the common video data of a frame Code processes and is stored (that is, compressing storage 106)；This video data is decoded (that is, read solution to obtain image information Code 108)；By acquired image rendering on the model being pre-designed to show this image (that is, model rendering 110)；And The gyroscope that carried by mobile device and the slide of user gesture are controlling this model and the direction at 3D visual angle (that is, to hand over Mutually control 112), thus realizing the viewing angle control to panoramic video.
It should be noted that the step of the workflow 100 shown in above-described embodiment is only exemplary rather than limiting.In other words, entirely The step that the workflow of scape player may include in more or less of step, and workflow than shown step can not be pressed Shown order is executing.
With reference to Fig. 2 to Fig. 4, according to embodiment of the present invention, the above-mentioned workflow of panorama player is carried out in detail Description.
Fig. 2 illustrates to be used for controlling the interactive controlling side of visual angle effect in panorama playing process according to embodiment of the present invention The indicative flowchart of method；Fig. 3 is the flow chart of the panorama playing process of the specific embodiment according to Fig. 2 illustrated embodiment, And the flow chart of the visual angle effect process that Fig. 4 is the specific embodiment of panorama playing process according to Fig. 3.
With reference to Fig. 2, for controlling the interaction control method 200 of visual angle effect to include following step in panorama playing process Suddenly：
Step 202, obtains the rending model of player in mobile device.
In an alternative embodiment, mobile device can be the mobile device based on iOS system, for example, iPhone or IPad, correspondingly panorama player is the panorama player in iPhone or iPad.But this is only exemplary rather than limiting, such as Mobile device can also be the mobile device based on Android or other system.
In an alternative embodiment, specifically, execution visual angle effect device using the model ID being provided by mobile device Lai Obtain rending model.In the case of being based on the equipment of iOS system in mobile device, this model ID is, for example, GLKMatrix4Identity.
Step 204, rotating this rending model makes it synchronous with the original horizontal position of gyroscope in mobile device.
Step 206, the information of collection gyroscope and user gesture.
In an alternative embodiment, the information of collection gyroscope and user gesture includes：The gesture being provided using mobile device Control object gathers the spatial rotational data of gyroscope in mobile device；And the gesture control object using mobile device offer Collection acts on the gesture slip data of the user gesture on the screen of mobile device.Be based on iOS system mobile device and Speech, gesture control object is, for example, the CMDeviceMotion object in CMMotionManager object, and gesture control object For example, UIGestureRecognizer object.
In an alternative embodiment, the spatial rotational data of gyroscope is Euler in X-axis, Y-axis and Z-direction for the gyroscope Angle.In an alternative embodiment, the gesture slip data of user gesture includes the previous coordinate figure Px in X-axis and Y direction1With Py1And present coordinate values Px2And Py2.
Step 208, according to the information execution visual angle effect from gyroscope and user gesture collection, with acquisition and user gesture With both gyroscopes corresponding synthesis offset data.
In an alternative embodiment, including according to from the information execution visual angle effect of gyroscope and user gesture collection：Will The gesture slip data conversion of user gesture becomes form angle rotation amount corresponding with user gesture；And will be with user gesture pair With gyroscope, the Euler angle in X-axis, Y-axis and Z-direction carries out adduction computing to the form angle rotation amount answered, with obtain with Both user gesture and gyroscope corresponding synthesis offset data.
Step 210, carries out rotation transformation based on original horizontal position and synthesis offset data to rending model.
In an alternative embodiment, carrying out rotation transformation based on original horizontal position and synthesis offset data to rending model can Including：According to synthesis offset data corresponding with both user gesture and gyroscope, rending model is opened from original horizontal position Begin to carry out rotation transformation.
Referring to Fig. 3, the specific embodiment in conjunction with Fig. 2 illustrated embodiment describes in detail according to embodiment of the present invention Panorama playing process.In this embodiment, mobile device is, for example, iPhone mobile phone, but is not limited to this.
The panorama player (for example, AVPlayer player 304) of mobile device reads the video file of panoramic video 302nd, this video file 302 is decoded with obtain the source data of panoramic video and by decoded source data transmit to Renderer 306.
Afterwards, renderer 306 is according to the source data of the panoramic video receiving from AVPlayer player 304, by plane graph Shape renders on rending model 308.In this embodiment, rending model 308 is threedimensional model, specially the logic three of computer Dimension solid.
Independent of described render process, the device of visual angle effect is controlled to gather using from gyroscope 310 and user gesture 312 Information, based on rending model 308 execution visual angle effect synthesis control, to obtain the multi-view image after synthesis.In this embodiment In, the information being gathered is respectively spatial rotational data 316 corresponding with gyroscope 310 and handss corresponding with user gesture 312 Gesture slip data 318.
Finally, based on the rending model 308 after visual angle effect by the image projection after synthesizing to painting canvas (for example, GLKView painting canvas 314) on, thus drawing out three-dimensional panoramic image to be shown, and then display to the user that this panorama via screen Image.
In this embodiment, AVPlayer player 304 and GLKView painting canvas 314 are the iOS system based on Apple Given special case.It should be understood that in other embodiments, for example situation, panorama player and painting canvas based on android system Can have other respective type.
Referring to Fig. 4, in conjunction with the specific embodiment of panorama playing process shown in Fig. 3, describe in detail based on gyroscope and User gesture carries out synthesis and controls to realize the process of visual angle effect to rending model.In this embodiment, rending model is Spheroid viewport model, carries out to rending model synthesizing the process controlling by for broadcasting in panorama based on gyroscope and user gesture Control the device at visual angle realizing during putting.
Initially, spheroid viewport model is located on the inceptive direction of acquiescence, as shown in step 402.
The model read module of device obtains the inceptive direction of spheroid viewport model, and at step 404, it is first from this Beginning direction is rotated so that it is synchronous with the original horizontal position of gyroscope in mobile device.After synchronized, can draw Image inside one visual angle screen-oriented, expected angle when this visual angle is lain low with mobile device acquiescence is identical.
Afterwards, the information acquisition module in device gathers information from gyroscope 406 and user gesture 408.In this embodiment In, the information being gathered is respectively：Spatial rotational data corresponding with gyroscope, that is, mobile device is with respect to horizontal initial position Space angle change, for example, space Euler angle of X-axis, Y-axis and Z-direction；And gesture corresponding with user gesture Slip data.
In an alternative embodiment, the gesture induction unit in information acquisition module utilizes the gesture control that mobile device provides Object, basis act on the user gesture on the screen of mobile device, obtain the previous coordinate figure Px in X-axis and Y direction1With Py1And present coordinate values Px2And Py2；Gyroscope sensing unit in information acquisition module utilizes the motion that mobile device provides Control object obtains gyroscope space Euler angle roll, pitch and yaw in original horizontal position.In this embodiment, first Front coordinate figure Px1And Py1And present coordinate values Px2And Py2It is previously described gesture slip data corresponding with user gesture Concrete form, but gesture slip data is not limited to this.
Subsequently, visual angle effect module carries out visual angle effect computing based on the information from gyroscope and user gesture collection, and Result according to computing correspondingly spinning ball viewport model, thus obtain one with mobile device the same space rotation regard Angle, and then obtain the visual angle picture through gyroscope and user gesture co- controlling computing.
Specifically, the Date Conversion Unit in visual angle effect module is according to equation 1 below .1, by corresponding with user gesture Previously coordinate figure Px1And Py1And present coordinate values Px2And Py2Derive side-play amount in X-axis and Y direction for this gesture DisX and disY：
DisX=(Px2–Px1)×k1
DisY=(Py2–Py1)×k1(formula 1.1)
Wherein, k1For amplitude constant.In this embodiment, side-play amount disX and disY are previously described and user gesture The concrete form of corresponding form angle rotation amount, but this is only exemplary rather than limiting.In an alternative embodiment, amplitude constant k1Value is 0.005.
Side-play amount disX and disY are mapped as by radian in X-axis and Y direction for the user gesture according to equation 1 below .2 RadX and radY：
RadX=disX × overture/k2
RadY=disY × overture/k2 (formula 1.2)
Wherein, overture is the angle of visual windows, and k2 is angle constant.In an alternative embodiment, this angle constant K2 value is 100.
According to equation 2 below .1 and 2.2, radian radX and radY corresponding with user gesture and gyroscope will be obtained Euler angle carry out adduction computing, with draw synthesis offset data RollX, RollY and RollZ.
When mobile device is in portrait layout configuration：
RollX=roll
RollY=-pitch+radX
RollZ=-yaw+radY (formula 2.1)
When mobile device is in transverse screen configuration：
RollX=roll+radX
RollY=-pitch
RollZ=-yaw+radY (formula 2.2)
In this embodiment, synthesis offset data is the rotation with respect to X-axis, Y-axis and Z axis for the model of gained after computing Direction.
Finally, at step 410, model is entered according to the synthesis offset data of gained by the rotation transform module in device Row rotation；And the model through visual angle effect for the output at step 412, so that renderer utilizes described mould after visual angle effect Type execution form projection.
By the technical scheme described by Fig. 1 to Fig. 4, rotation (can be embodied in based on user gesture and gyroscope Change hands machine) control mode that combines is controlled to form angle so that user can be in the feelings being independent of mobile device direction Adjust video angle under condition to carry out visual angle using gyroscope to mobile device simultaneously and follow, so that the viewing of whole panoramic video Experience more hommization.
Fig. 5 is to be used for controlling the device of visual angle effect in panorama playing process according to embodiment of the present invention.Reference picture 5, for controlling the device 500 of visual angle effect to may include model read module 502, information acquisition module in panorama playing process 504th, visual angle effect module 506 and rotation transform module 508.
Model read module 502 can obtain the rending model of player in mobile device, and rotated rendering model makes it Synchronous with the original horizontal position of gyroscope in mobile device.
Information acquisition module 504 can gather the information of gyroscope and user gesture.In an alternative embodiment, information gathering mould Block 504 can further include gyroscope sensing unit 510 and gesture induction unit 512, and wherein gyroscope sensing unit 510 utilizes The motor control object that mobile device provides gathers the spatial rotational data of gyroscope in mobile device；Gesture induction unit 512 Slided using the gesture that the gesture control object collection that mobile device provides acts on the user gesture on the screen of mobile device Data.
Visual angle effect module 506 can be according to the information execution visual angle effect from gyroscope and user gesture collection, to obtain Synthesis offset data corresponding with both institute's family gesture and gyroscope.In an alternative embodiment, visual angle effect module 506 can enter one Step includes Date Conversion Unit 514 and arithmetic element 516, and the gesture of user gesture can be slided by wherein Date Conversion Unit 514 Data conversion becomes form angle rotation amount corresponding with user gesture；Arithmetic element 516 can be by form corresponding with user gesture Angle rotation amount carries out adduction computing to obtain and user gesture and gyro with the spatial rotational data of gyroscope in mobile device Both instrument corresponding synthesis offset data.
Rotation transform module 508 can be carried out rotation and become based on original horizontal position and synthesis offset data to rending model Change, so that renderer is using the rending model execution form projection after conversion.
In herein in the whole text, technical term is not limited to the implication being limited on literal, but without departing from the present invention such as power In the case that profit requires limited range, cover the different devices of statement for realizing same or similar function.
It will be understood by those skilled in the art that the technical scheme of the application can be embodied as system, method or computer program Product.Therefore, the application can behave as the embodiment of complete hardware, complete software embodiment (include firmware, resident software, Microcode etc.) or the form of embodiment that software and hardware combines, they may be referred to generally as " circuit ", " module " or " are System ".Additionally, the application can behave as the form of computer program, computer program is embedded into any tangible table Reach in medium, tangible expression medium has the computer usable program code being embedded in medium.
The flow chart of method, device (system) and computer program according to the embodiment of the present application and/or block diagram Lai Description the application.It is understood that can by each frame in computer program instructions execution flow chart and/or block diagram and The combination of the multiple frames in flow chart and/or block diagram.These computer program instructions are provided to general purpose computer, special Purpose computer or the processor of other programmable data processing unit, so that processed by computer or other programmable data The instruction of the computing device of device creates the work(indicating in a frame being used for flowchart and/or block diagram or multiple frame The device of energy/action.
These computer program instructions also can be stored in can instruct computer or other programmable data processing unit with Specific mode is realized in the computer-readable medium of function, so that the instruction being stored in computer-readable medium produces including The command device of the function/action indicating in one of flowchart and/or block diagram frame or multiple frame.
Computer program instructions also can be loaded in computer or other programmable data processing unit, is being calculated with causing A series of operating procedure is executed on machine or on other programmable device, to produce computer implemented process, so that in meter On calculation machine or other programmable device, the instruction of execution is provided for one of flowchart and/or block diagram frame or multiple frame In the process of function/action that indicates.
Flow chart in accompanying drawing and block diagram illustrate the system of multiple embodiments according to the application, method, function and operation. It should be noted that in some substitutability are implemented, in frame, the function of mark can not occur according to the order of mark in accompanying drawing.Example As, according to involved feature, two continuously illustrating frame actually can substantially simultaneously execute, or these frames sometimes with Contrary order execution.It can also be noted that can by execution specific function or action special purpose hardware based system or Special purpose hardware and computer instruction combine each frame to realize in block diagram and/or flowchart illustration and block diagram and/ Or the combination of the multiple frames in flowchart illustration.
Above description is only presently preferred embodiments of the present invention and the explanation to the technology of the present invention principle.People in the art Member it should be appreciated that the present invention in involved invention scope be not limited to above-mentioned technical characteristic particular combination technical side Case, in the case of without departing from inventive concept, the present invention also should cover and carried out arbitrarily by above-mentioned technical characteristic or its equivalent feature The other technical schemes combining and being formed.Such as features described above is entered with the technical characteristic with similar functions disclosed in the present invention The technical scheme that row is replaced mutually and formed.

Claims (12)

1. it is used for controlling the interaction control method of visual angle effect in panorama playing process, described interaction control method includes following Step：
Obtain the rending model of player in mobile device；
Rotate described rending model so as to synchronous with the original horizontal position of gyroscope in described mobile device；
Gather the information of described gyroscope and user gesture；
According to the information execution visual angle effect from described gyroscope and the collection of described user gesture, to obtain and described user gesture With both described gyroscopes corresponding synthesis offset data；And
Rotation transformation is carried out to described rending model based on described original horizontal position and described synthesis offset data.
2. interaction control method as claimed in claim 1, wherein, gathers described gyroscope and the information of user gesture includes：
The motor control object being provided using described mobile device, gathers the spatial rotational number of gyroscope in described mobile device According to；And
The gesture control object being provided using described mobile device, collection acts on the user's handss on the screen of described mobile device The gesture slip data of gesture.
3. interaction control method as claimed in claim 2, wherein, the spatial rotational data of described gyroscope is described gyroscope Euler angle in X-axis, Y-axis and Z-direction.
4. interaction control method as claimed in claim 3, wherein, according to gather from described gyroscope and described user gesture Information execution visual angle effect includes：
The gesture slip data conversion of described user gesture is become form angle rotation amount corresponding with described user gesture；And
By form angle rotation amount corresponding with described user gesture with described gyroscope the Europe in X-axis, Y-axis and Z-direction Angle is drawn to carry out adduction computing, to obtain synthesis offset data corresponding with both described user gesture and described gyroscope.
5. interaction control method as claimed in claim 4, wherein, the gesture slip data of described user gesture includes X-axis and Y Previous coordinate figure Px on direction of principal axis1And Py1And present coordinate values Px2And Py2；Form angle corresponding with described user gesture Spend radian radX and radY in X-axis and Y direction after rotation amount includes mapping；Gesture slip data by described user gesture It is converted into form angle rotation amount corresponding with described user gesture to realize by equation 1 below：
DisX=(Px2–Px1)×k1
DisY=(Py2–Py1)×k1
RadX=disX × overture/k2
RadY=disY × overture/k2
Wherein, disX and disY is respectively side-play amount in X-axis and Y direction for the described user gesture, and overture is visual The angle of window, k1For amplitude constant, k2For angle constant.
6. interaction control method as claimed in claim 5, wherein, corresponding with both described user gesture and described gyroscope Synthesis offset data includes described rending model with respect to direction of rotation RollX, RollY of X-axis, Y-axis and Z axis and RollZ；Will Form angle rotation amount corresponding with described user gesture Euler angle in X-axis, Y-axis and Z-direction with described gyroscope Carry out adduction computing to realize by following,
When described mobile device is in portrait layout configuration, realized by equation 2 below：
RollX=roll
RollY=-pitch+radX
RollZ=-yaw+radY
When described mobile device is in transverse screen configuration, realized by equation 3 below：
RollX=roll+radX
RollY=-pitch
RollZ=-yaw+radY
Wherein, roll, pitch and yaw are space Euler angle at described original horizontal position for the described gyroscope.
7. the interaction control method as described in claim 5 or 6, wherein, described amplitude constant k1For 0.005, described angle constant k2For 100.
8. the interaction control method as any one of claim 1 to 6, wherein, based on described original horizontal position and institute State synthesis offset data to carry out rotation transformation to described rending model and include：According to described user gesture and described gyroscope two Person's corresponding synthesis offset data, described rending model is proceeded by rotation transformation from described original horizontal position.
9. the interaction control method as any one of claim 1 to 6, wherein, described mobile device be iPhone or iPad.
10. it is used for controlling the device of visual angle effect in panorama playing process, described device includes：
Model read module, obtaining the rending model of player in mobile device, and rotate described rending model makes itself and institute The original horizontal position stating gyroscope in mobile device is synchronous；
Information acquisition module, gathers the information of described gyroscope and user gesture；
Visual angle effect module, according to the information execution visual angle effect from described gyroscope and the collection of described user gesture, to obtain Synthesis offset data corresponding with both described user gesture and described gyroscope；And
Rotation transform module, is rotated to described rending model based on described original horizontal position and described synthesis offset data Conversion.
11. devices as claimed in claim 10, wherein said information acquisition module further includes：
Gyroscope sensing unit, gathers gyroscope in described mobile device using the motor control object that described mobile device provides Spatial rotational data；And
Gesture induction unit, acts on the screen of described mobile device using the gesture control object collection that described mobile device provides The gesture slip data of the user gesture on curtain.
12. devices as claimed in claim 10, wherein said visual angle effect module further includes：
Date Conversion Unit, the gesture slip data conversion of described user gesture is become form angle corresponding with described user gesture Degree rotation amount；And
Arithmetic element, by form angle rotation amount corresponding with described user gesture with gyroscope in described mobile device space Rotation data carries out adduction computing, to obtain synthesis offset data corresponding with both described user gesture and described gyroscope.
CN201610831446.8A 2016-09-19 2016-09-19 Interaction control method for controlling visual angle conversion in panorama playing process, and device for realizing interaction control method Pending CN106383655A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201610831446.8A CN106383655A (en) 2016-09-19 2016-09-19 Interaction control method for controlling visual angle conversion in panorama playing process, and device for realizing interaction control method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201610831446.8A CN106383655A (en) 2016-09-19 2016-09-19 Interaction control method for controlling visual angle conversion in panorama playing process, and device for realizing interaction control method

Publications (1)

Publication Number Publication Date
CN106383655A true CN106383655A (en) 2017-02-08

Family

ID=57936645

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201610831446.8A Pending CN106383655A (en) 2016-09-19 2016-09-19 Interaction control method for controlling visual angle conversion in panorama playing process, and device for realizing interaction control method

Country Status (1)

Country Link
CN (1) CN106383655A (en)

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107728787A (en) * 2017-10-30 2018-02-23 太平洋未来科技（深圳）有限公司 Method for information display and device in panoramic video
CN108882018A (en) * 2017-05-09 2018-11-23 合信息技术（北京）有限公司 Video playing, data offering method, client and server in virtual scene
CN108965959A (en) * 2018-08-10 2018-12-07 Tcl通力电子（惠州）有限公司 Broadcasting, acquisition methods, mobile phone, PC equipment and the system of VR video
WO2019007372A1 (en) * 2017-07-05 2019-01-10 北京京东尚科信息技术有限公司 Model display method and apparatus
CN110519644A (en) * 2019-09-05 2019-11-29 青岛一舍科技有限公司 In conjunction with the panoramic video visual angle regulating method and device for recommending visual angle
CN110633030A (en) * 2019-09-26 2019-12-31 上海红星美凯龙设计云信息科技有限公司 Active touch control identifier
CN111127621A (en) * 2019-12-31 2020-05-08 歌尔科技有限公司 Picture rendering method and device and readable storage medium
CN111447462A (en) * 2020-05-20 2020-07-24 上海科技大学 Video live broadcast method, system, storage medium and terminal based on visual angle switching
CN112312227A (en) * 2020-06-15 2021-02-02 青岛一舍科技有限公司 Panoramic interaction method and device
WO2021218473A1 (en) * 2020-04-27 2021-11-04 海信视像科技股份有限公司 Display method and display device

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120307001A1 (en) * 2011-06-03 2012-12-06 Nintendo Co., Ltd. Information processing system, information processing device, storage medium storing information processing program, and moving image reproduction control method
CN105148514A (en) * 2015-09-06 2015-12-16 骆凌 Device and method for controlling game view angle
CN105208368A (en) * 2015-09-23 2015-12-30 北京奇虎科技有限公司 Method and device for displaying panoramic data
CN105828090A (en) * 2016-03-22 2016-08-03 乐视网信息技术（北京）股份有限公司 Panorama live broadcasting method and device
CN105847850A (en) * 2016-03-28 2016-08-10 乐视控股（北京）有限公司 Panorama video real time playing method and apparatus
CN105898460A (en) * 2015-12-10 2016-08-24 乐视网信息技术（北京）股份有限公司 Method and device for adjusting panorama video play visual angle of intelligent TV
CN105930054A (en) * 2016-04-01 2016-09-07 上海逗屋网络科技有限公司 Method and device for performing user operation in touch terminal

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120307001A1 (en) * 2011-06-03 2012-12-06 Nintendo Co., Ltd. Information processing system, information processing device, storage medium storing information processing program, and moving image reproduction control method
CN105148514A (en) * 2015-09-06 2015-12-16 骆凌 Device and method for controlling game view angle
CN105208368A (en) * 2015-09-23 2015-12-30 北京奇虎科技有限公司 Method and device for displaying panoramic data
CN105898460A (en) * 2015-12-10 2016-08-24 乐视网信息技术（北京）股份有限公司 Method and device for adjusting panorama video play visual angle of intelligent TV
CN105828090A (en) * 2016-03-22 2016-08-03 乐视网信息技术（北京）股份有限公司 Panorama live broadcasting method and device
CN105847850A (en) * 2016-03-28 2016-08-10 乐视控股（北京）有限公司 Panorama video real time playing method and apparatus
CN105930054A (en) * 2016-04-01 2016-09-07 上海逗屋网络科技有限公司 Method and device for performing user operation in touch terminal

Cited By (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108882018B (en) * 2017-05-09 2020-10-20 阿里巴巴（中国）有限公司 Video playing and data providing method in virtual scene, client and server
CN108882018A (en) * 2017-05-09 2018-11-23 合信息技术（北京）有限公司 Video playing, data offering method, client and server in virtual scene
WO2019007372A1 (en) * 2017-07-05 2019-01-10 北京京东尚科信息技术有限公司 Model display method and apparatus
CN107728787A (en) * 2017-10-30 2018-02-23 太平洋未来科技（深圳）有限公司 Method for information display and device in panoramic video
CN107728787B (en) * 2017-10-30 2020-07-07 太平洋未来科技（深圳）有限公司 Information display method and device in panoramic video
CN108965959A (en) * 2018-08-10 2018-12-07 Tcl通力电子（惠州）有限公司 Broadcasting, acquisition methods, mobile phone, PC equipment and the system of VR video
CN110519644A (en) * 2019-09-05 2019-11-29 青岛一舍科技有限公司 In conjunction with the panoramic video visual angle regulating method and device for recommending visual angle
CN110633030A (en) * 2019-09-26 2019-12-31 上海红星美凯龙设计云信息科技有限公司 Active touch control identifier
CN111127621A (en) * 2019-12-31 2020-05-08 歌尔科技有限公司 Picture rendering method and device and readable storage medium
CN111127621B (en) * 2019-12-31 2024-02-09 歌尔科技有限公司 Picture rendering method, device and readable storage medium
WO2021218473A1 (en) * 2020-04-27 2021-11-04 海信视像科技股份有限公司 Display method and display device
CN111447462A (en) * 2020-05-20 2020-07-24 上海科技大学 Video live broadcast method, system, storage medium and terminal based on visual angle switching
CN111447462B (en) * 2020-05-20 2022-07-05 上海科技大学 Video live broadcast method, system, storage medium and terminal based on visual angle switching
CN112312227A (en) * 2020-06-15 2021-02-02 青岛一舍科技有限公司 Panoramic interaction method and device

Similar Documents

Publication Publication Date Title
CN106383655A (en) Interaction control method for controlling visual angle conversion in panorama playing process, and device for realizing interaction control method
US10863168B2 (en) 3D user interface—360-degree visualization of 2D webpage content
WO2020186935A1 (en) Virtual object displaying method and device, electronic apparatus, and computer-readable storage medium
JP2021193599A (en) Virtual object figure synthesizing method, device, electronic apparatus, and storage medium
US11003305B2 (en) 3D user interface
JP5857946B2 (en) Image processing apparatus, image processing method, and program
CN106648098B (en) AR projection method and system for user-defined scene
CN108133454B (en) Space geometric model image switching method, device and system and interaction equipment
CN112950751A (en) Gesture action display method and device, storage medium and system
CN113781615B (en) Animation generation method, device, equipment and storage medium
JP5213500B2 (en) Image conversion program and image conversion apparatus
CN106331690A (en) 3D bullet screen realization method and device
JP2024502407A (en) Display methods, devices, devices and storage media based on augmented reality
CN104216533A (en) Head-wearing type virtual reality display based on DirectX9
JP6549764B1 (en) IMAGE PROJECTION SYSTEM, IMAGE PROJECTION METHOD, AND PROGRAM
US10623713B2 (en) 3D user interface—non-native stereoscopic image conversion
KR20230130748A (en) Image processing methods and apparatus, devices and media
CN107426601B (en) Display method and device of UI (user interface) control in smart television
CN106909263A (en) The interactive display method and system of 3-D view
CN116958344A (en) Animation generation method and device for virtual image, computer equipment and storage medium
CN109814704B (en) Video data processing method and device
CN114327174A (en) Virtual reality scene display method and cursor three-dimensional display method and device
CN103616989B (en) Method and device for automatically adjusting position of cursor and mobile terminal
WO2015146813A1 (en) Object manipulation method, object manipulation program, and information processing device
JP2014203413A (en) House computer graphics device, and program

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20170208