WO2018129792A1 - Vr播放方法、vr播放装置及vr播放系统 - Google Patents

Vr播放方法、vr播放装置及vr播放系统 Download PDF

Info

Publication number
WO2018129792A1
WO2018129792A1 PCT/CN2017/074807 CN2017074807W WO2018129792A1 WO 2018129792 A1 WO2018129792 A1 WO 2018129792A1 CN 2017074807 W CN2017074807 W CN 2017074807W WO 2018129792 A1 WO2018129792 A1 WO 2018129792A1
Authority
WO
WIPO (PCT)
Prior art keywords
module
angle
multimedia panoramic
space coordinate
axis
Prior art date
Application number
PCT/CN2017/074807
Other languages
English (en)
French (fr)
Inventor
杨帆
刘影
何永强
李勃君
Original Assignee
深圳创维－Rgb电子有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳创维－Rgb电子有限公司 filed Critical 深圳创维－Rgb电子有限公司
Priority to US15/780,549 priority Critical patent/US10977852B2/en
Priority to AU2017384696A priority patent/AU2017384696B2/en
Publication of WO2018129792A1 publication Critical patent/WO2018129792A1/zh

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
    • H04N21/4312—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G06F3/0346—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—3D [Three Dimensional] image rendering
    • G06T15/04—Texture mapping
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—3D [Three Dimensional] image rendering
    • G06T15/10—Geometric effects
    • G06T15/20—Perspective computation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G06T19/20—Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/81—Monomedia components thereof
    • H04N21/816—Monomedia components thereof involving special video data, e.g 3D video
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/16—Sound input; Sound output
    • G06F3/167—Audio in a user interface, e.g. using voice commands for navigating, audio feedback
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2200/00—Indexing scheme for image data processing or generation, in general
    • G06T2200/24—Indexing scheme for image data processing or generation, in general involving graphical user interfaces [GUIs]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20048—Transform domain processing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2215/00—Indexing scheme for image rendering
    • G06T2215/12—Shadow map, environment map
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2215/00—Indexing scheme for image rendering
    • G06T2215/16—Using real world measurements to influence rendering
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2219/00—Indexing scheme for manipulating 3D models or images for computer graphics
    • G06T2219/20—Indexing scheme for editing of 3D models
    • G06T2219/2016—Rotation, translation, scaling

Definitions

  • the present invention relates to the field of virtual reality technologies, and in particular, to a VR playing method, a VR playing device, and a VR playing system.
  • the main object of the present invention is to provide a VR playback method, which aims to solve the technical problem that a dedicated VR device is expensive, has a small visible area, and easily causes dizziness and fatigue of the wearer.
  • the VR playing method proposed by the present invention includes the following steps:
  • a virtual camera component is disposed at a center of the 3D spherical object, and by controlling an angle of the virtual camera, the content of the multimedia panoramic material is picked up and played on the media playing device.
  • the step of capturing the content of the multimedia panoramic material on the media playing device by controlling the angle of the virtual camera includes:
  • the step of creating a 3D spherical object in the development environment specifically includes:
  • the method further includes:
  • shader options include Legacy Shaders, Transparent, Cutout, and Soft. Edge Unlit.
  • Another object of the present invention is to provide a VR playback device for a media playback device, the VR playback device comprising:
  • the first creation module creates a 3D spherical body object in the development environment
  • the second creation module creates a new texture material and creates a shader option for the texture material
  • Loading a module acquiring a multimedia panoramic material, loading the texture material and the multimedia panoramic material onto a surface of the 3D spherical object;
  • the picking and playing module arranges a virtual camera component in the center of the 3D spherical object, and controls the content of the multimedia panoramic image to be played on the media playing device by controlling the angle of the virtual camera.
  • the picking and playing module is configured to track location information of the user relative to the media playing device in real time, calculate a viewing angle of the user according to the location information, adjust an angle of the virtual camera in real time according to the viewing angle, and pick up the multimedia.
  • the content of the panorama material is played on the media player.
  • the first creating module includes:
  • the Transform build unit establishes a world space coordinate Transform with the parameters of Position, Rotation, and Scale;
  • a first setting unit that sets (0, 0, 0) to a position of a center of the 3D spherical object in the world space coordinate Transform
  • a second setting unit that sets the y-axis in the world space coordinate Transform to rotate to 180 degrees
  • the third setting unit sets the zoom ratio of the x-axis, the y-axis, and the z-axis in the world space coordinate Transform to 1:10.
  • the VR playback device further includes:
  • the determining module determines whether the x-axis attribute of the parameter Scale in the world space coordinate Transform is negative, and if not, inverts the multimedia panoramic material.
  • shader options include Legacy Shaders, Transparent, Cutout, and Soft. Edge Unlit.
  • the present invention also provides a VR playback system including the VR playback device as described above, and a media playback device to which the VR playback device is applied.
  • the media playing device includes:
  • the communication module acquires a signal command sent by the smart device, and adjusts a display angle and a display area of the multimedia panoramic material according to the signal instruction;
  • a wireless receiving module receiving a radio command transmitted by the wireless transmitting device, and adjusting a picking angle, a picking speed, and a picking content of the virtual camera component according to the radio command;
  • the sensing recognition module acquires a motion gesture of the user, and controls the media playing device to respond to the motion gesture;
  • a voice recognition module receiving a voice command of the user, and controlling a play state of the media play device according to the voice command
  • the switching module is used for switching between the communication module, the wireless receiving module, the sensing recognition module, and the voice recognition module.
  • the VR playing method of the present invention is applied to a media playing device.
  • a media playing device When playing with a media playing device, first create a 3D spherical object in the development environment, then create a new texture material and create a shader option for the texture material.
  • the multimedia panoramic material is obtained from the media playing device or the external storage device, and the texture material and the multimedia panoramic material are loaded onto the surface of the 3D spherical object, and finally A virtual camera component is disposed at the center of the 3D spherical object, and the content of the multimedia panoramic material is picked up and played on the media playing device by controlling the angle of the virtual camera.
  • the VR playing method of the invention loads the multimedia panoramic material on the surface of a spherical body, and adjusts the picking angle of the virtual camera, and each time the content of a rectangular area in the multimedia panoramic material is picked up and played on the media playing device, thereby avoiding direct The curved surface and incomplete display during viewing, and by adjusting the picking angle, you can view each area of the multimedia panoramic material without restriction, increasing the viewing area and reducing the cost of the VR experience. The experiencer does not need to wear it. Causes dizziness and fatigue.
  • FIG. 1 is a flowchart of an embodiment of a VR playing method according to the present invention.
  • FIG. 2 is a schematic diagram showing the position of a user relative to a media playing device according to the present invention
  • step S10 in FIG. 1 is a specific flowchart of step S10 in FIG. 1;
  • FIG. 4 is a schematic structural diagram of an embodiment of a VR playback system according to the present invention.
  • the invention provides a VR playing method, which is applied to a media playing device.
  • FIG. 1 is a flowchart of an embodiment of a VR playing method according to the present invention.
  • the VR playback method includes the following steps:
  • S30 Acquire a multimedia panoramic material, and load the texture material and the multimedia panoramic material onto a surface of the 3D spherical object;
  • a virtual camera component is disposed at a center of the 3D spherical object, and the content of the multimedia panoramic material is picked up and played on the media playing device by controlling an angle of the virtual camera.
  • the VR playing method is applied to a media playing device, such as a television, a set top box, and a video playing box.
  • a 3D spherical object is created in a development environment of the media playing device.
  • the operation is completed in the system program of the media playing device, and the creation of the 3D spherical object can be realized by adding the program text in the system configuration, or by using a small plug-in, after creating the 3D spherical object.
  • the available forms include Legacy.
  • the playback device automatically acquires a location path of the multimedia panoramic material, and loads the multimedia panoramic material and the texture material prepared in the above step onto the surface of the 3D spherical object, and when performing the operation, the The multimedia panoramic material is attached to the surface of the 3D spherical object, and then the texture material is pasted on the outer surface of the multimedia panoramic material, or the texture material may be first attached to the 3D spherical object.
  • the multimedia panoramic material attaching the multimedia panoramic material to the surface of the texture material, and finally, when viewing, arranging a virtual camera component at the center of the 3D spherical object to pick up the surface of the 3D spherical object
  • the content of the multimedia panoramic material is played on the media playing device by controlling the virtual camera
  • the picking angle controls the content and angle displayed on the media playing device.
  • the VR playing method is used to play the panoramic video on the media playing device.
  • a plug-in is also installed on the media playing device, and then the plug-in is used in the development.
  • a spherical body object is created in the environment as a screen carrier, and the size of the screen carrier can be set according to the resolution of the panoramic video. After the spherical screen carrier is created, a new material is created and the material is colored.
  • the surface of the spherical object is simultaneously added with a mirror script carried by the insert, so that after the virtual camera assembly is disposed at the center of the spherical body, the image content picked up by the virtual camera component does not reverse, and finally
  • the center of the spherical object is arranged with a virtual camera component, and at the same time, scripts such as control, path, audio, zoom, fast forward, brightness, etc. are added, and 360 can be viewed in all directions when playing on the media playing device, and the ordinary media playing is solved. Distortion and incompleteness of the picture when the device is directly viewing the panorama material.
  • the VR playing method of the invention loads the multimedia panoramic material on the surface of a spherical body, and adjusts the picking angle of the virtual camera, and each time the content of a rectangular area in the multimedia panoramic material is picked up and played on the media playing device, thereby avoiding direct The curved surface and incomplete display during viewing, and by adjusting the picking angle, you can view each area of the multimedia panoramic material without restriction, increasing the viewing area and reducing the cost of the VR experience. The experiencer does not need to wear it. Causes dizziness and fatigue.
  • the step of capturing the content of the multimedia panoramic material on the media playing device by controlling the angle of the virtual camera includes:
  • the media playing device in order to realize the effect of playing the panoramic VR video by the traditional media playing device, can also obtain the position information and the output image of the user, so that if the user changes the position, the angle of the virtual camera also follows. Change to pick up the content of the multimedia panorama material to achieve a better experience.
  • the media playing device such as a television, tracks the position information of the user relative to the television in real time at a very high speed, and then calculates the viewing angle of the user according to the location information, and moves from the user to the system response.
  • the total time of completion is controlled within 20ms, and the frame rate is above 60Hz, so that the user can not feel the delay and the jam.
  • the range of user activity that the media player can collect and track is very high, and the effective acquisition angle is not low.
  • the recognition accuracy is above 1cm, to ensure that the user can correctly collect the position of the viewing angle of any TV screen, and obtain the user relative to the TV.
  • the position information (x, y, z) of the machine and the TV screen size information (w, h) can calculate the angle of view ( ⁇ , ⁇ ) of the user at this time, and then calculate according to the position and angle of view of the user. Mapping the corresponding panoramic video picture area (A), which is the picture that should be displayed by the local TV set. At this time, by adjusting the angle of the virtual camera, the multimedia is picked up. Screen material should be displayed for playback on a media playback device (TV).
  • TV media playback device
  • step S10 specifically includes:
  • S12 setting (0, 0, 0) is the position of the center of the 3D spherical object in the world space coordinate Transform
  • the coordinates (0, 0, 0) are set to the position of the center of the 3D spherical object in the world space coordinate Transform, in order to make the multimedia panorama loaded on the surface of the 3D spherical object.
  • the material does not reverse, and the y coordinate axis in the world space coordinate Transform is set to rotate to 180 degrees, so that when the virtual camera component is set at the center of the 3D spherical object, the captured multimedia panoramic material will not be displayed. Inversion occurs, and in order to ensure that the picture content of the multimedia panoramic material can be clearly viewed during viewing, the x-axis and y in the world space coordinate Transform are And scaling the z-axis was set to 1:10.
  • the VR playing method according to the foregoing embodiment further includes:
  • the multimedia panoramic material is acquired from the media playing device or the external storage device plugged into the media playing device, and the multimedia panoramic material and the texture material are loaded into the 3D spherical object. After the surface, it is necessary to determine whether the multimedia panoramic material has an inversion, and the method of determining is to determine whether the x-axis property of the parameter Scale in the world space coordinate Transform is negative, and if the x-axis property of the parameter Scale is negative, Then, it is determined that the multimedia panoramic material does not reverse, and the content of the multimedia panoramic material picked up by the virtual camera component and played on the media playing device is a normal unreversed inverted image, if the x-axis property of the parameter Scale is If yes, it is determined that the multimedia panoramic material is reversed, and the content of the multimedia panoramic material picked up by the virtual camera component and played on the media playing device is an inverted inverted image, and the multimedia panoramic material needs to be reversed. Processing so that the media playback device plays a normal multimedia panorama picture.
  • the invention further proposes a VR playback device.
  • the VR playback device 10 includes:
  • the first creation module 11 creates a 3D spherical object in the development environment
  • the second creating module 12 creates a new texture material and creates a shader option for the texture material
  • the picking and playing module 14 arranges a virtual camera component in the center of the 3D spherical object. By controlling the angle of the virtual camera, the content of the multimedia panoramic material is picked up and played on the media playing device 30.
  • the VR playback device 10 is applied to the media playback device 30, such as a television, a set top box, and a video play box.
  • the first creation module 11 is in the development environment of the media playback device 30. Creating a 3D spherical object in the system, the operation is completed in the system program of the media playing device 30, and the program text can be added in the system configuration, or the creation of the 3D spherical object can be realized by using a small plug-in.
  • a curtain for the 3D spherical body object that is, a new texture material is created by the second creation module 12, and the texture material is not created by the texture material, and the form is available for selection.
  • a new texture material is created by the second creation module 12
  • the texture material is not created by the texture material, and the form is available for selection.
  • different texture materials can be selected according to different playing scenes and playing content, and then the user can import the multimedia material that needs to be viewed by the VR into the media playing device 30, or import it into the media playing device 30 in the previous step.
  • the time loading module 13 automatically acquires the location path of the multimedia panoramic material, and loads the multimedia panoramic material and the texture material prepared in the above step onto the surface of the 3D spherical object, and when performing the operation, the The multimedia panoramic material is attached to the surface of the 3D spherical object, and then the texture material is pasted on the outer surface of the multimedia panoramic material, or the texture material may be first attached to the 3D spherical body. The surface of the object is then attached to the surface of the texture material, and finally, a virtual camera component is disposed in the center of the 3D spherical object by the pick and play module 14 when viewing, and the pickup is located at the surface.
  • the content of the multimedia panoramic material on the surface of the 3D spherical object is played on the media playing device 30, Controlling the pickup angle of the virtual camera, and the angle control content displayed on the media playback device 30.
  • the VR playing method is used to play the panoramic video on the media playing device 30.
  • the first creating module 11 is further required to install a plug-in on the media playing device.
  • the plug-in is used to create a spherical body object in the development environment as a screen carrier, and the size of the screen carrier can be set according to the resolution of the panoramic video.
  • the second creation module is created.
  • the loading module 13 creates the material prepared by the above steps.
  • the panoramic video file is imported into the surface of the spherical body object, and the mirror script carried by the plug-in is added, so that the virtual camera component is not displayed after the virtual camera component is disposed in the center of the spherical body.
  • the virtual camera component is arranged in the center of the spherical object by the pick and play module 14, and scripts such as control, path, audio, zoom, fast forward, brightness, etc. are added, and can be played on the media playing device 30. 360-wide viewing solves the problem of distortion and incompleteness of the screen when viewing the panoramic material directly using ordinary media playback devices.
  • the VR playback device 10 of the present invention loads the multimedia panoramic material on the surface of a spherical body.
  • the pickup angle of the virtual camera By adjusting the pickup angle of the virtual camera, each time the content of a rectangular region in the multimedia panoramic material is picked up and played on the media playing device 30, avoiding The surface and incomplete display during direct viewing, and by adjusting the picking angle, can view each area of the multimedia panoramic material without restriction, increasing the viewing area and reducing the cost of the VR experience, and the wearer does not need to wear it. Does not cause dizziness and fatigue.
  • the picking and playing module 14 is configured to track location information of the user relative to the media playing device in real time, calculate a viewing angle of the user according to the location information, adjust an angle of the virtual camera in real time according to the viewing angle, and pick up the The content of the multimedia panorama material is played on the media playback device.
  • the picking and playing module 14 can also obtain the position information and the output image of the user, so that if the user changes the position, the angle of the virtual camera is also Then change to pick up the content of the multimedia panoramic material to achieve a better experience.
  • the pick-and-play module 14, such as a local camera and a television system of a television, track the position information of the user relative to the television in real time at a very high speed, and then calculate the user according to the location information.
  • the viewing angle the total time from the user movement to the completion of the system response is controlled within 20ms, the frame rate is above 60Hz, so that the user basically does not feel the delay and the jam, in addition, the media player can collect and track the user activity.
  • the range is very high, its effective acquisition angle of view is not less than 150 degrees, preferably 180 degrees, and the effective distance is 5cm-5m, the recognition accuracy is above 1cm, to ensure that the user can be in the position of the viewing angle of any TV screen.
  • Correctly collecting, obtaining the position information (x, y, z) of the user relative to the television, and the television screen size information (w, h), can calculate the angle of view ( ⁇ , ⁇ ) of the user at this time, and then according to The position and the angle of view of the user are used to calculate the panoramic video picture area (A) corresponding to the mapping, and the area A is the picture that should be displayed by the local television.
  • the picture to be displayed by the multimedia panoramic material is played on the media playing device (television set).
  • the first creating module 11 includes:
  • the Transform establishing unit 111 establishes a world space coordinate Transform by using Position, Rotation, and Scale as parameters;
  • the first setting unit 112 sets (0, 0, 0) to the position of the center of the 3D spherical object in the world space coordinate Transform;
  • the second setting unit 113 sets the y-axis in the world space coordinate Transform to rotate to 180 degrees;
  • the third setting unit 114 sets the scaling ratio of the x-axis, the y-axis, and the z-axis in the world space coordinate Transform to 1:10.
  • the Transform establishing unit 111 is required to first use the Position ( Position), Rotation, and Scale establish a world space coordinate Transform for the parameter, and then set by the first setting unit 112, the second setting unit 113, and the third setting unit 114, respectively.
  • the position, the rotation angle, and the scaling of the 3D spherical object in the world space coordinate Transform are created for the 3D spherical object.
  • the first setting unit 112 sets the coordinates (0, 0).
  • the second setting unit 113 sets the world space coordinate Transform.
  • the y coordinate axis in the rotation is rotated to 180 degrees so that the virtual camera component is set at the center of the 3D spherical object, and the captured multimedia panoramic material does not appear.
  • a third setting unit 114 scaling the x-axis in the world-space coordinate Transform, y-axis and z-axis is set to 1:10.
  • the VR playback device 10 further includes:
  • the determining module 15 determines whether the x-axis attribute of the parameter Scale in the world space coordinate Transform is negative, and if not, performs reverse processing on the multimedia panoramic material.
  • the loading module 13 acquires the multimedia panoramic material from the media playing device 30 or the external storage device plugged into the media playing device 30, and loads the multimedia panoramic material and the texture material into the After the surface of the 3D spherical object, it is necessary to determine whether the multimedia panoramic material has an inversion, and the method of determining, that is, determining, by the determining module 15, whether the x-axis property of the parameter Scale in the world space coordinate Transform is negative, If the x-axis attribute of the parameter Scale is negative, it is determined that the multimedia panoramic material does not reverse, and the content of the multimedia panoramic material picked up by the virtual camera component and played on the media playing device 30 is normal and not inverted.
  • the multimedia panoramic material is reversed, and the content of the multimedia panoramic material picked up by the virtual camera component and played on the media playing device 30 is inverted inverted image.
  • the multimedia panoramic material needs to be reverse processed so that the media playing device 30 can play normal multimedia. Panoramic footage.
  • the invention also proposes a VR playback system.
  • FIG. 4 is a schematic structural diagram of an embodiment of a VR playback system according to the present invention.
  • the VR playback system 100 includes the VR playback device 10 as described above, and the media playback device 30 to which the VR playback device 10 is applied.
  • the VR playback system 100 includes the VR playback device 10 as described above, and the media playback device 30 that applies the VR playback device 10, and the multimedia panorama material is loaded on the surface of a spherical body by the VR playback device 10.
  • the picking angle of the virtual camera By adjusting the picking angle of the virtual camera, each time the content of a rectangular area in the multimedia panoramic material is picked up and played on the media playing device 30, the surface and the incomplete display during direct viewing are avoided, and the picking angle can be adjusted without being adjusted. Restricted viewing of each area of the multimedia panorama material increases the viewing area and reduces the cost of the VR experience. The experiencer does not need to wear it, and does not cause dizziness and fatigue.
  • the media playing device 30 includes:
  • the communication module 31 acquires a signal command sent by the smart device, and adjusts a display angle and a display area of the multimedia panoramic material according to the signal instruction;
  • the wireless receiving module 32 receives a radio command transmitted by the wireless transmitting device, and adjusts a picking angle, a picking speed, and a picking content of the virtual camera component according to the radio command;
  • the sensing recognition module 33 acquires a motion gesture of the user, and controls the media playing device to respond to the motion gesture;
  • the voice recognition module 34 receives the voice command of the user, and controls the playing state of the media playing device according to the voice command;
  • the switching module 35 is used for switching between the communication module 31, the wireless receiving module 32, the sensing recognition module 33, and the voice recognition module 34.
  • the media playing device 30 may be a television, a set top box and a video playing box matched with the television.
  • the media playing device further A communication module 31, a wireless receiving module 32, a sensing identification module 33, a voice recognition module 34, and a switching module 35 for seamless switching between the modules are provided to meet the needs of different groups of people; for example, using smart devices, such as smart The sensor, the PAD, the smart watch, and the like carry the sensor to obtain the change amount of the angle and the direction of the rotation, and establish a connection with the communication module 31 of the media playing device 30 by technical means including but not limited to wifi or Bluetooth, according to the angle change.
  • the signal, the direction of rotation, and the like command adjust the display angle and the display area of the multimedia panoramic material on the media playing device; and, if a part of the buttons of the wireless transmitting device such as a remote controller is used, transmit a radio command to the wireless receiving module 32 to adjust the virtual camera component.
  • Picking angle, picking speed, and loss of content such as menu direction keys Realizing the automatic rotation of the viewing angle and the left and right direction of the rotation, adjusting the viewing angle by the up and down direction keys, adjusting the speed of the automatic rotation by using the numeric keys, and adding the sensing recognition module 33 to the media playing device, by acquiring the click of the user, Grasping an action gesture, controlling the media playback device 30 to respond to the action gesture, thereby completing an operation on the VR system; the media playback device is further provided with a voice recognition module 34 for receiving a voice command of the user.
  • the voice recognition algorithm processes and controls the playing state of the media playing device.
  • the media playing device is further provided with a feedback device and a real-time broadcast module, so as to provide corresponding feedback prompts for each instruction of the user, and real-time Broadcast the system playback status to improve the interactivity of the VR playback system.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • Computer Graphics (AREA)
  • Signal Processing (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • General Health & Medical Sciences (AREA)
  • Computer Hardware Design (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Geometry (AREA)
  • Architecture (AREA)
  • Processing Or Creating Images (AREA)
  • Image Generation (AREA)

Abstract

本发明公开一种VR播放方法、装置及系统，在使用媒体播放设备进行播放时，首先在开发环境中创建一个3D球型体对象，然后建立新的贴图材质并为该贴图材质创建着色器选项，在为所述贴图材质选择着色后从媒体播放设备或外置存储设备中获取多媒体全景素材，将所述贴图材质及多媒体全景素材加载至所述3D球型体对象的表面，最后在所述3D球型体对象的中心布置一虚拟摄像机组件，通过控制该虚拟摄像机的角度，拾取所述多媒体全景素材的内容在媒体播放设备上进行播放。本发明的VR播放方法，避免了直接观看时的曲面和不完整显示，而且通过调整拾取角度增大可视面积，降低了VR体验的成本，体验者无需进行佩戴，不会引起眩晕和疲劳。

Description

VR播放方法、VR播放装置及VR播放系统
技术领域
本发明涉及虚拟现实技术领域，尤其涉及一种VR播放方法、VR播放装置及VR播放系统。
背景技术
近年来，随着虚拟实境技术的快速发展，有关VR（虚拟现实）的各类内容也逐渐增多，但是目前只有在专用的VR设备上，用户才能够观看和体验VR内容，并且当前的VR设备还存在以下缺点：
（1）大部分的VR设备需要单独购买，并且价格昂贵；
（2）受屏幕和设备体积所限造成可视面积小；
（3）大部分设备虽然整体重量较轻，但多数需要使用者佩戴在头部，重量全部交由头部承担，且屏幕与佩戴者的眼睛距离过近，十分容易引起佩戴者眩晕和疲劳。
发明内容
本发明的主要目的在于提供一种VR播放方法，旨在解决专用的VR设备价格昂贵、可视面积小、及容易引起佩戴者眩晕和疲劳的技术问题。
为实现上述目的，本发明提出的VR播放方法，包括以下步骤：
在开发环境中创建一个3D球型体对象；
建立新的贴图材质并为该贴图材质创建着色器选项；
获取多媒体全景素材，将所述贴图材质及多媒体全景素材加载至所述3D球型体对象的表面；
在所述3D球型体对象的中心布置一虚拟摄像机组件，通过控制该虚拟摄像机的角度，拾取所述多媒体全景素材的内容在媒体播放设备上进行播放。
进一步地，所述通过控制该虚拟摄像机的角度，拾取所述多媒体全景素材的内容在媒体播放设备上进行播放的步骤，包括：
实时跟踪用户相对于媒体播放设备的位置信息，依据所述位置信息计算用户的观看视角，依据所述观看视角实时调整虚拟摄像机的角度，拾取所述多媒体全景素材的内容在媒体播放设备上进行播放。
进一步地，所述在开发环境中创建一个3D球型体对象的步骤，具体包括：
以Position、Rotation、及Scale为参数建立世界空间坐标Transform；
设定（0,0,0）为3D球型体对象的中心在世界空间坐标Transform中的位置；
设定世界空间坐标Transform中的y轴旋转到180度；
设定世界空间坐标Transform中x轴、y轴、z轴的缩放比例为1:10。
进一步地，所述获取多媒体全景素材，将所述贴图材质及多媒体全景素材加载至所述3D球型体对象的表面的步骤之后，还包括：
判断所述世界空间坐标Transform中参数Scale的x轴属性是否为负，若否，则对所述多媒体全景素材进行反转处理。
进一步地，所述着色器选项包括Legacy Shaders、Transparent、Cutout、及Soft Edge Unlit。
本发明的另一目的在于提出一种VR播放装置，应用于媒体播放设备，该VR播放装置包括：
第一创建模块，在开发环境中创建一个3D球型体对象；
第二创建模块，建立新的贴图材质并为该贴图材质创建着色器选项；
加载模块，获取多媒体全景素材，将所述贴图材质及多媒体全景素材加载至所述3D球型体对象的表面；
拾取及播放模块，在所述3D球型体对象的中心布置一虚拟摄像机组件，通过控制该虚拟摄像机的角度，拾取所述多媒体全景素材的内容在媒体播放设备上进行播放。
进一步地，所述拾取及播放模块用于实时跟踪用户相对于媒体播放设备的位置信息，依据所述位置信息计算用户的观看视角，依据所述观看视角实时调整虚拟摄像机的角度，拾取所述多媒体全景素材的内容在媒体播放设备上进行播放。
进一步地，所述第一创建模块包括：
Transform建立单元，以Position、Rotation、及Scale为参数建立世界空间坐标Transform；
第一设定单元，设定（0,0,0）为3D球型体对象的中心在世界空间坐标Transform中的位置；
第二设定单元，设定世界空间坐标Transform中的y轴旋转到180度；
第三设定单元，设定世界空间坐标Transform中x轴、y轴、z轴的缩放比例为1:10。
进一步地，该VR播放装置还包括：
判断模块，判断所述世界空间坐标Transform中参数Scale的x轴属性是否为负，若否，则对所述多媒体全景素材进行反转处理。
进一步地，所述着色器选项包括Legacy Shaders、Transparent、Cutout、及Soft Edge Unlit。
本发明还提出一种VR播放系统，该VR播放系统包括如上所述的VR播放装置，及应用该VR播放装置的媒体播放设备。
进一步地，所述媒体播放设备包括：
通信模块，获取智能设备发送的信号指令，并依据所述信号指令调整所述多媒体全景素材的显示角度和显示区域；
无线接收模块，接收无线发射装置发射的无线电指令，并依据所述无线电指令调整所述虚拟摄像机组件的拾取角度、拾取速度、及拾取内容；
感应识别模块，获取用户的动作手势，并控制所述媒体播放设备对所述动作手势作出响应；
语音识别模块，接收用户的语音指令，并依据所述语音指令控制媒体播放设备的播放状态；及
切换模块，用于所述通信模块、无线接收模块、感应识别模块、语音识别模块之间的切换。
本发明的VR播放方法，应用于媒体播放设备，在使用媒体播放设备进行播放时，首先在开发环境中创建一个3D球型体对象，然后建立新的贴图材质并为该贴图材质创建着色器选项，在为所述贴图材质选择着色后从媒体播放设备或外置存储设备中获取多媒体全景素材，将所述贴图材质及多媒体全景素材加载至所述3D球型体对象的表面，最后在所述3D球型体对象的中心布置一虚拟摄像机组件，通过控制该虚拟摄像机的角度，拾取所述多媒体全景素材的内容在媒体播放设备上进行播放。
本发明的VR播放方法，将多媒体全景素材加载于一球型体表面，通过调整虚拟摄像机的拾取角度，每次拾取该多媒体全景素材中一矩形区域的内容在媒体播放设备上播放，避免了直接观看时的曲面和不完整显示，而且通过调整拾取角度可以不受限制地观看多媒体全景素材的每一区域，增大了可视面积，降低了VR体验的成本，体验者无需进行佩戴，不会引起眩晕和疲劳。
附图说明
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图示出的结构获得其他的附图。
图1为本发明的VR播放方法一实施例的流程图；
图2为本发明用户相对于媒体播放设备的位置示意图；
图3为图1中步骤S10的具体流程图；
图4为本发明的VR播放系统一实施例的结构示意图。
附图标号说明：
标号 名称 标号 名称
100 VR播放系统 14 拾取及播放模块
10 VR播放装置 15 判断模块
11 第一创建模块 30 媒体播放设备
111 Transform建立单元 31 通信模块
112 第一设定单元 32 无线接收模块
113 第二设定单元 34 感应识别模块
114 第三设定单元 34 语音识别模块
12 第二创建模块 35 切换模块
13 加载模块
本发明目的的实现、功能特点及优点将结合实施例，参照附图做进一步说明。
具体实施方式
应当理解，此处所描述的具体实施例仅仅用以解释本发明，并不用于限定本发明。
本发明提出一种VR播放方法，应用于媒体播放设备。
参照图1，图1为本发明VR播放方法一实施例的流程图。
在本实施例中，该VR播放方法包括以下步骤：
S10：在开发环境中创建一个3D球型体对象；
S20：建立新的贴图材质并为该贴图材质创建着色器选项；
S30：获取多媒体全景素材，将所述贴图材质及多媒体全景素材加载至所述3D球型体对象的表面；
S50：在所述3D球型体对象的中心布置一虚拟摄像机组件，通过控制该虚拟摄像机的角度，拾取所述多媒体全景素材的内容在媒体播放设备上进行播放。
在本实施例中，该VR播放方法应用于媒体播放设备，如电视、机顶盒、视频播放盒子，在用户想要进行VR体验时，在媒体播放设备的开发环境中创建一个3D球型体对象，该操作通过在媒体播放设备的系统程序中完成，可以通过在系统配置中增加程序文本，也可以采用小插件的方式实现对3D球型体对象的创建，在创建所述3D球型体对象之后还需要为该3D球型体对象创建幕布，也即建立新的贴图材质并未该贴图材质创建着色器选项，可供选择的形式包括Legacy Shaders、Transparent、Cutout、及Soft Edge Unlit，根据不同的播放场景和播放内容可以选择不同的贴图材质，接着用户可以将需要利用VR进行观看的多媒体资料导入媒体播放设备，或者在之前的步骤中将其导入媒体播放设备，此时媒体播放设备会自动获取多媒体全景素材的位置路径，将所述多媒体全景素材和上述步骤中准备就绪的贴图材质加载至所述3D球型体对象的表面，在实施该操作时，可以先将所述多媒体全景素材贴合于3D球型体对象的表面，然后再将所述贴图材质贴覆于所述多媒体全景素材的外表面，也可以先将所述贴图材质贴合于3D球型体对象的表面，然后将所述多媒体全景素材贴在贴图材质的表面，最后，也就是进行观看时在所述3D球型体对象的中心布置一虚拟摄像机组件，拾取位于该3D球型体对象的表面的多媒体全景素材的内容在媒体播放设备上进行播放，通过控制该虚拟摄像机的拾取角度，控制在所述媒体播放设备上显示的内容和角度。
在其他实施例中，如在媒体播放设备上利用该VR播放方法播放全景视频，首先，在上述实施例中的步骤S10之前，还需要在媒体播放设备上安装一插件，然后利用该插件在开发环境中创建一个球型体对象作为幕布载体，该幕布载体的大小可以根据所述全景视频的解析度进行设定，在球状的幕布载体创建完成后，建立新的材质，并为该材质将着色器选项设置成Legacy Shaders/Transparent/Cutout/Soft Edge Unlit，然后选择DUMMY纹理，使媒体播放设备未播放时的状态呈现为黑屏，以便于给用于提供一定的视觉缓冲，避免造成眩晕和疲劳，接着将上述步骤创建好的材质及全景视频文件导入所述球型体对象的表面，同时添加插件所带的镜面脚本，以便在该球型体的中心布置虚拟摄像机组件之后，所述虚拟摄像机组件拾取到的画面内容不会出现反转，最后在该球型体对象的中心布置虚拟摄像机组件，同时增加控制、路径、音频、缩放、快进、亮度等脚本，在媒体播放设备上进行播放时可以360全方位观看，解决了利用普通的媒体播放设备直接对全景素材进行观看时出现的画面扭曲和不完整的问题。
本发明的VR播放方法，将多媒体全景素材加载于一球型体表面，通过调整虚拟摄像机的拾取角度，每次拾取该多媒体全景素材中一矩形区域的内容在媒体播放设备上播放，避免了直接观看时的曲面和不完整显示，而且通过调整拾取角度可以不受限制地观看多媒体全景素材的每一区域，增大了可视面积，降低了VR体验的成本，体验者无需进行佩戴，不会引起眩晕和疲劳。
进一步地，所述通过控制该虚拟摄像机的角度，拾取所述多媒体全景素材的内容在媒体播放设备上进行播放的步骤，包括：
实时跟踪用户相对于媒体播放设备的位置信息，依据所述位置信息计算用户的观看视角，依据所述观看视角实时调整虚拟摄像机的角度，拾取所述多媒体全景素材的内容在媒体播放设备上进行播放。
在本实施例中，为了实现传统媒体播放设备播放全景VR视频的效果，所述媒体播放设备还能够通过获取用户的位置信息和输出图像，做到如果用户改变位置，虚拟摄像机的角度也随之改变，以拾取所述多媒体全景素材的内容，达到更好的体验效果。具体如图2所示，所述媒体播放设备，如电视机以极高的速度实时跟踪用户相对于电视机的位置信息，然后依据所述位置信息计算用户的观看视角，从用户移动到系统响应完成的总时间控制在20ms以内，帧率在60Hz以上，使用户基本感受不到延时及卡顿，此外，媒体播放设备所能采集和追踪用户活动的范围非常高，其有效采集视角不低于150度，优选为180度，且有效距离在5cm-5m，识别精度在1cm以上，以保证用户在任何电视屏幕的可视角度的位置都能够被系统正确采集，在获取到用户相对于电视机的位置信息（x，y，z）、电视屏幕尺寸信息（w，h），可以计算出用户此时的视场角（α，β），然后根据用户的位置及视场角，计算出映射对应的全景视频画面区域（A），区域A即是本地电视机所应显示的画面，此时通过调整虚拟摄像机的角度，拾取所述多媒体全景素材应显示的画面在媒体播放设备（电视机）上进行播放。
进一步地，参照图3，基于上述实施例的VR播放方法，步骤S10，具体包括：
S11：以Position、Rotation、及Scale为参数建立世界空间坐标Transform；
S12：设定（0,0,0）为3D球型体对象的中心在世界空间坐标Transform中的位置；
S13：设定世界空间坐标Transform中的y轴旋转到180度；
S14：设定世界空间坐标Transform中x轴、y轴、z轴的缩放比例为1:10。
在本实施例中，无论是直接在媒体播放设备的开发环境中创建3D球型体对象，还是借助于第三方插件在开发环境中创建球型体对象，都需要先以Position（位置）、Rotation（旋转）、及Scale（缩放）为参数建立世界空间坐标Transform，然后通过设定所述3D球型体对象在世界空间坐标Transform中的位置、旋转角度、及缩放比例对所述3D球型体对象进行创建，在本实施例中，设定坐标（0,0,0）为3D球型体对象的中心在世界空间坐标Transform中的位置，为了使加载于3D球型体对象表面的多媒体全景素材未发生反转，设定世界空间坐标Transform中的y坐标轴旋转到180度，以使得所述虚拟摄像机组件设置于3D球型体对象的中心时，拾取到的多媒体全景素材的画面不会出现反转，同时为了保证观看时能够清晰地观看到多媒体全景素材的画面内容，将世界空间坐标Transform中的x轴、y轴及z轴的缩放比例设定为1:10。
进一步地，参照图1，基于上述实施例的VR播放方法，步骤S30之后，还包括：
S40：判断所述世界空间坐标Transform中参数Scale的x轴属性是否为负，若否，则对所述多媒体全景素材进行反转处理。
在本实施例中，在从媒体播放设备或插接于所述媒体播放设备的外置存储设备中获取多媒体全景素材，并将所述多媒体全景素材及贴图材质加载于所述3D球型体对象的表面之后，需要确定该多媒体全景素材有没有出现反转，判定的方法也即，判断所述世界空间坐标Transform中参数Scale的x轴属性是否为负，若参数Scale的x轴属性为负，则判断所述多媒体全景素材没有出现反转，经虚拟摄像机组件拾取到并在媒体播放设备上进行播放的多媒体全景素材的内容是正常的未反转倒立的画面，若参数Scale的x轴属性为正，则判断所述多媒体全景素材出现反转，经虚拟摄像机组件拾取到并在媒体播放设备上进行播放的多媒体全景素材的内容是反转倒立的画面，需要对所述多媒体全景素材进行反转处理，以便媒体播放设备播放正常的多媒体全景素材画面。
本发明进一步提出一种VR播放装置。
参照图4，在本实施例中，该VR播放装置10包括：
第一创建模块11，在开发环境中创建一个3D球型体对象；
第二创建模块12，建立新的贴图材质并为该贴图材质创建着色器选项；
加载模块13，获取多媒体全景素材，将所述贴图材质及多媒体全景素材加载至所述3D球型体对象的表面；
拾取及播放模块14，在所述3D球型体对象的中心布置一虚拟摄像机组件，通过控制该虚拟摄像机的角度，拾取所述多媒体全景素材的内容在媒体播放设备30上进行播放。
在本实施例中，该VR播放装置10应用于媒体播放设备30，如电视、机顶盒、视频播放盒子，在用户想要进行VR体验时，经第一创建模块11在媒体播放设备30的开发环境中创建一个3D球型体对象，该操作通过在媒体播放设备30的系统程序中完成，可以通过在系统配置中增加程序文本，也可以采用小插件的方式实现对3D球型体对象的创建，在创建所述3D球型体对象之后还需要为该3D球型体对象创建幕布，也即由第二创建模块12建立新的贴图材质并未该贴图材质创建着色器选项，可供选择的形式包括Legacy Shaders、Transparent、Cutout、及Soft Edge Unlit，根据不同的播放场景和播放内容可以选择不同的贴图材质，接着用户可以将需要利用VR进行观看的多媒体资料导入媒体播放设备30，或者在之前的步骤中将其导入媒体播放设备30，此时加载模块13会自动获取多媒体全景素材的位置路径，将所述多媒体全景素材和上述步骤中准备就绪的贴图材质加载至所述3D球型体对象的表面，在实施该操作时，可以先将所述多媒体全景素材贴合于3D球型体对象的表面，然后再将所述贴图材质贴覆于所述多媒体全景素材的外表面，也可以先将所述贴图材质贴合于3D球型体对象的表面，然后将所述多媒体全景素材贴在贴图材质的表面，最后，也就是进行观看时由拾取及播放模块14在所述3D球型体对象的中心布置一虚拟摄像机组件，拾取位于该3D球型体对象的表面的多媒体全景素材的内容在媒体播放设备30上进行播放，通过控制该虚拟摄像机的拾取角度，控制在所述媒体播放设备30上显示的内容和角度。
在其他实施例中，如在媒体播放设备30上利用该VR播放方法播放全景视频，首先，在上述实施例中的步骤S10之前，还需要第一创建模块11在媒体播放设备上安装一插件，然后利用该插件在开发环境中创建一个球型体对象作为幕布载体，该幕布载体的大小可以根据所述全景视频的解析度进行设定，在球状的幕布载体创建完成后，由第二创建模块12建立新的材质，并为该材质将着色器选项设置成Legacy Shaders/Transparent/Cutout/Soft Edge Unlit，然后选择DUMMY纹理，使媒体播放设备30未播放时的状态呈现为黑屏，以便于给用于提供一定的视觉缓冲，避免造成眩晕和疲劳，接着由加载模块13将上述步骤创建好的材质及全景视频文件导入所述球型体对象的表面，同时添加插件所带的镜面脚本，以便在该球型体的中心布置虚拟摄像机组件之后，所述虚拟摄像机组件拾取到的画面内容不会出现反转，最后由拾取及播放模块14在该球型体对象的中心布置虚拟摄像机组件，同时增加控制、路径、音频、缩放、快进、亮度等脚本，在媒体播放设备30上进行播放时可以360全方位观看，解决了利用普通的媒体播放设备直接对全景素材进行观看时出现的画面扭曲和不完整的问题。
本发明的VR播放装置10，将多媒体全景素材加载于一球型体表面，通过调整虚拟摄像机的拾取角度，每次拾取该多媒体全景素材中一矩形区域的内容在媒体播放设备30上播放，避免了直接观看时的曲面和不完整显示，而且通过调整拾取角度可以不受限制地观看多媒体全景素材的每一区域，增大了可视面积，降低了VR体验的成本，体验者无需进行佩戴，不会引起眩晕和疲劳。
进一步地，所述拾取及播放模块14用于实时跟踪用户相对于媒体播放设备的位置信息，依据所述位置信息计算用户的观看视角，依据所述观看视角实时调整虚拟摄像机的角度，拾取所述多媒体全景素材的内容在媒体播放设备上进行播放。
在本实施例中，为了实现传统媒体播放设备播放全景VR视频的效果，所述拾取及播放模块14还能够通过获取用户的位置信息和输出图像，做到如果用户改变位置，虚拟摄像机的角度也随之改变，以拾取所述多媒体全景素材的内容，达到更好的体验效果。具体如图2所示，所述拾取及播放模块14，如电视机的本地摄像装置及电视机系统以极高的速度实时跟踪用户相对于电视机的位置信息，然后依据所述位置信息计算用户的观看视角，从用户移动到系统响应完成的总时间控制在20ms以内，帧率在60Hz以上，使用户基本感受不到延时及卡顿，此外，媒体播放设备所能采集和追踪用户活动的范围非常高，其有效采集视角不低于150度，优选为180度，且有效距离在5cm-5m，识别精度在1cm以上，以保证用户在任何电视屏幕的可视角度的位置都能够被系统正确采集，在获取到用户相对于电视机的位置信息（x，y，z）、电视屏幕尺寸信息（w，h），可以计算出用户此时的视场角（α，β），然后根据用户的位置及视场角，计算出映射对应的全景视频画面区域（A），区域A即是本地电视机所应显示的画面，此时通过调整虚拟摄像机的角度，拾取所述多媒体全景素材应显示的画面在媒体播放设备（电视机）上进行播放。
进一步地，参照图4，所述第一创建模块11包括：
Transform建立单元111，以Position、Rotation、及Scale为参数建立世界空间坐标Transform；
第一设定单元112，设定（0,0,0）为3D球型体对象的中心在世界空间坐标Transform中的位置；
第二设定单元113，设定世界空间坐标Transform中的y轴旋转到180度；
第三设定单元114，设定世界空间坐标Transform中x轴、y轴、z轴的缩放比例为1:10。
在本实施例中，无论是直接在媒体播放设备的开发环境中创建3D球型体对象，还是借助于第三方插件在开发环境中创建球型体对象，都需要Transform建立单元111先以Position（位置）、Rotation（旋转）、及Scale（缩放）为参数建立世界空间坐标Transform，然后分别由第一设定单元112、第二设定单元113、级第三设定单元114通过设定所述3D球型体对象在世界空间坐标Transform中的位置、旋转角度、及缩放比例对所述3D球型体对象进行创建，在本实施例中，第一设定单元112设定坐标（0,0,0）为3D球型体对象的中心在世界空间坐标Transform中的位置，为了使加载于3D球型体对象表面的多媒体全景素材未发生反转，二设定单元113设定世界空间坐标Transform中的y坐标轴旋转到180度，以使得所述虚拟摄像机组件设置于3D球型体对象的中心时，拾取到的多媒体全景素材的画面不会出现反转，同时为了保证观看时能够清晰地观看到多媒体全景素材的画面内容，第三设定单元114将世界空间坐标Transform中的x轴、y轴及z轴的缩放比例设定为1:10。
进一步地，参照图4，该VR播放装置10还包括：
判断模块15，判断所述世界空间坐标Transform中参数Scale的x轴属性是否为负，若否，则对所述多媒体全景素材进行反转处理。
在本实施例中，加载模块13在从媒体播放设备30或插接于所述媒体播放设备30的外置存储设备中获取多媒体全景素材，并将所述多媒体全景素材及贴图材质加载于所述3D球型体对象的表面之后，需要确定该多媒体全景素材有没有出现反转，判定的方法也即，由判断模块15判断所述世界空间坐标Transform中参数Scale的x轴属性是否为负，若参数Scale的x轴属性为负，则判断所述多媒体全景素材没有出现反转，经虚拟摄像机组件拾取到并在媒体播放设备30上进行播放的多媒体全景素材的内容是正常的未反转倒立的画面，若参数Scale的x轴属性为正，则判断所述多媒体全景素材出现反转，经虚拟摄像机组件拾取到并在媒体播放设备30上进行播放的多媒体全景素材的内容是反转倒立的画面，需要对所述多媒体全景素材进行反转处理，以便媒体播放设备30播放正常的多媒体全景素材画面。
本发明还提出一种VR播放系统。
参照图4，图4为本发明的VR播放系统一实施例的结构示意图。
在本实施例中，该VR播放系统100包括如上所述的VR播放装置10，及应用该VR播放装置10的媒体播放设备30。
在本实施例中，该VR播放系统100包括如上所述的VR播放装置10、及应用该VR播放装置10的媒体播放设备30，通过VR播放装置10将多媒体全景素材加载于一球型体表面，通过调整虚拟摄像机的拾取角度，每次拾取该多媒体全景素材中一矩形区域的内容在媒体播放设备30上播放，避免了直接观看时的曲面和不完整显示，而且通过调整拾取角度可以不受限制地观看多媒体全景素材的每一区域，增大了可视面积，降低了VR体验的成本，体验者无需进行佩戴，不会引起眩晕和疲劳。
进一步地，参照图4，所述媒体播放设备30包括：
通信模块31，获取智能设备发送的信号指令，并依据所述信号指令调整所述多媒体全景素材的显示角度和显示区域；
无线接收模块32，接收无线发射装置发射的无线电指令，并依据所述无线电指令调整所述虚拟摄像机组件的拾取角度、拾取速度、及拾取内容；
感应识别模块33，获取用户的动作手势，并控制所述媒体播放设备对所述动作手势作出响应；
语音识别模块34，接收用户的语音指令，并依据所述语音指令控制媒体播放设备的播放状态；及
切换模块35，用于所述通信模块31、无线接收模块32、感应识别模块33、语音识别模块34之间的切换。
在本实施例中，所述媒体播放设备30可以是电视、与电视配合的机顶盒、视频播放盒子，为了增加用户在该VR播放系统中进行体验的沉浸感和交互性，所述媒体播放设备还设置有通信模块31、无线接收模块32、感应识别模块33、语音识别模块34、及用于上述模块之间无缝切换的切换模块35，以满足不同人群的需求；如利用智能设备，如智能手机、PAD、智能手表等本身携带的传感器获取角度的变化量、及转动的方向，通过包括但不限于wifi、蓝牙的技术手段与媒体播放设备30的通信模块31建立连接，依据所述角度变化量、转动方向等信号指令调整所述多媒体全景素材在媒体播放设备上的显示角度和显示区域；再如利用遥控器等无线发射装置的部分按键发射无线电指令给无线接收模块32，调整虚拟摄像机组件的拾取角度、拾取速度、及失去内容，如菜单方向键，实现视角的自动旋转及旋转的左右方向，通过上下方向键调整观看的角度，利用数字键调节自动旋转的速度等；又如在媒体播放设备上加装感应识别模块33，通过获取用户的点击、抓取等动作手势，控制所述媒体播放设备30对所述动作手势作出响应，进而完成对VR系统的操作；该媒体播放设备还装设了语音识别模块34，以接收用户的语音指令，经过语音识别算法处理，控制媒体播放设备的播放状态；在其他实施例中，该媒体播放设备还装设有反馈装置和实时播报模块，以便对用户的每一条指令，作出相应的反馈提示，并实时播报系统播放状态，提高该VR播放系统的交互性。
以上仅为本发明的优选实施例，并非因此限制本发明的专利范围，凡是利用本发明说明书及附图内容所作的等效结构或等效流程变换，或直接或间接运用在其他相关的技术领域，均同理包括在本发明的专利保护范围内。

Claims (15)

  1. 一种VR播放方法，应用于媒体播放设备，其特征在于，该VR播放方法包括以下步骤：
    在开发环境中创建一个3D球型体对象；
    建立新的贴图材质并为该贴图材质创建着色器选项；
    获取多媒体全景素材，将所述贴图材质及多媒体全景素材加载至所述3D球型体对象的表面；
    在所述3D球型体对象的中心布置一虚拟摄像机组件，通过控制该虚拟摄像机的角度，拾取所述多媒体全景素材的内容在媒体播放设备上进行播放。
  2. 根据权利要求1所述的VR播放方法，其特征在于，所述通过控制该虚拟摄像机的角度，拾取所述多媒体全景素材的内容在媒体播放设备上进行播放的步骤，包括：
    实时跟踪用户相对于媒体播放设备的位置信息，依据所述位置信息计算用户的观看视角，依据所述观看视角实时调整虚拟摄像机的角度，拾取所述多媒体全景素材的内容在媒体播放设备上进行播放。
  3. 根据权利要求1所述的VR播放方法，其特征在于，所述在开发环境中创建一个3D球型体对象的步骤，具体包括：
    以Position、Rotation、及Scale为参数建立世界空间坐标Transform；
    设定（0,0,0）为3D球型体对象的中心在世界空间坐标Transform中的位置；
    设定世界空间坐标Transform中的y轴旋转到180度；
    设定世界空间坐标Transform中x轴、y轴、z轴的缩放比例为1:10。
  4. 根据权利要求3所述的VR播放方法，其特征在于，所述获取多媒体全景素材，将所述贴图材质及多媒体全景素材加载至所述3D球型体对象的表面的步骤之后，还包括：
    判断所述世界空间坐标Transform中参数Scale的x轴属性是否为负，若否，则对所述多媒体全景素材进行反转处理。
  5. 根据权利要求1所述的VR播放方法，其特征在于，所述着色器选项包括Legacy Shaders、Transparent、Cutout、及Soft Edge Unlit。
  6. 一种VR播放装置，应用于媒体播放设备，其特征在于，该VR播放装置包括：
    第一创建模块，在开发环境中创建一个3D球型体对象；
    第二创建模块，建立新的贴图材质并为该贴图材质创建着色器选项；
    加载模块，获取多媒体全景素材，将所述贴图材质及多媒体全景素材加载至所述3D球型体对象的表面；
    拾取及播放模块，在所述3D球型体对象的中心布置一虚拟摄像机组件，通过控制该虚拟摄像机的角度，拾取所述多媒体全景素材的内容在媒体播放设备上进行播放。
  7. 根据权利要求6所述的VR播放装置，其特征在于，所述拾取及播放模块用于实时跟踪用户相对于媒体播放设备的位置信息，依据所述位置信息计算用户的观看视角，依据所述观看视角实时调整虚拟摄像机的角度，拾取所述多媒体全景素材的内容在媒体播放设备上进行播放。
  8. 根据权利要求6所述的VR播放装置，其特征在于，所述第一创建模块包括：
    Transform建立单元，以Position、Rotation、及Scale为参数建立世界空间坐标Transform；
    第一设定单元，设定（0,0,0）为3D球型体对象的中心在世界空间坐标Transform中的位置；
    第二设定单元，设定世界空间坐标Transform中的y轴旋转到180度；
    第三设定单元，设定世界空间坐标Transform中x轴、y轴、z轴的缩放比例为1:10。
  9. 根据权利要求8所述的VR播放装置，其特征在于，该VR播放装置还包括：
    判断模块，判断所述世界空间坐标Transform中参数Scale的x轴属性是否为负，若否，则对所述多媒体全景素材进行反转处理。
  10. 根据权利要求6所述的VR播放装置，其特征在于，所述着色器选项包括Legacy Shaders、Transparent、Cutout、及Soft Edge Unlit。
  11. 一种VR播放系统，其特征在于，该VR播放系统包括一VR播放装置，及应用该VR播放装置的媒体播放设备，所述VR播放装置包括：
    第一创建模块，在开发环境中创建一个3D球型体对象；
    第二创建模块，建立新的贴图材质并为该贴图材质创建着色器选项；
    加载模块，获取多媒体全景素材，将所述贴图材质及多媒体全景素材加载至所述3D球型体对象的表面；
    拾取及播放模块，在所述3D球型体对象的中心布置一虚拟摄像机组件，通过控制该虚拟摄像机的角度，拾取所述多媒体全景素材的内容在媒体播放设备上进行播放。
  12. 根据权利要求11所述的VR播放系统，其特征在于，所述拾取及播放模块用于实时跟踪用户相对于媒体播放设备的位置信息，依据所述位置信息计算用户的观看视角，依据所述观看视角实时调整虚拟摄像机的角度，拾取所述多媒体全景素材的内容在媒体播放设备上进行播放。
  13. 根据权利要求11所述的VR播放系统，其特征在于，所述第一创建模块包括：
    Transform建立单元，以Position、Rotation、及Scale为参数建立世界空间坐标Transform；
    第一设定单元，设定（0,0,0）为3D球型体对象的中心在世界空间坐标Transform中的位置；
    第二设定单元，设定世界空间坐标Transform中的y轴旋转到180度；
    第三设定单元，设定世界空间坐标Transform中x轴、y轴、z轴的缩放比例为1:10。
  14. 根据权利要求13所述的VR播放系统，其特征在于，所述VR播放装置还包括：
    判断模块，判断所述世界空间坐标Transform中参数Scale的x轴属性是否为负，若否，则对所述多媒体全景素材进行反转处理。
  15. 根据权利要求11所述的VR播放系统，其特征在于，所述媒体播放设备包括：
    通信模块，获取智能设备发送的信号指令，并依据所述信号指令调整所述多媒体全景素材的显示角度和显示区域；
    无线接收模块，接收无线发射装置发射的无线电指令，并依据所述无线电指令调整所述虚拟摄像机组件的拾取角度、拾取速度、及拾取内容；
    感应识别模块，获取用户的动作手势，并控制所述媒体播放设备对所述动作手势作出响应；
    语音识别模块，接收用户的语音指令，并依据所述语音指令控制媒体播放设备的播放状态；及
    切换模块，用于所述通信模块、无线接收模块、感应识别模块、语音识别模块之间的切换。
PCT/CN2017/074807 2017-01-16 2017-02-24 Vr播放方法、vr播放装置及vr播放系统 WO2018129792A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US15/780,549 US10977852B2 (en) 2017-01-16 2017-02-24 VR playing method, VR playing device, and VR playing system
AU2017384696A AU2017384696B2 (en) 2017-01-16 2017-02-24 Vr playing method, vr playing apparatus and vr playing system

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201710028145.6 2017-01-16
CN201710028145.6A CN106658148B (zh) 2017-01-16 2017-01-16 Vr播放方法、vr播放装置及vr播放系统

Publications (1)

Publication Number Publication Date
WO2018129792A1 true WO2018129792A1 (zh) 2018-07-19

Family

ID=58842968

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/074807 WO2018129792A1 (zh) 2017-01-16 2017-02-24 Vr播放方法、vr播放装置及vr播放系统

Country Status (4)

Country Link
US (1) US10977852B2 (zh)
CN (1) CN106658148B (zh)
AU (1) AU2017384696B2 (zh)
WO (1) WO2018129792A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109872400A (zh) * 2019-02-18 2019-06-11 上海电气集团股份有限公司 一种全景虚拟现实场景的生成方法
CN115379277A (zh) * 2022-08-22 2022-11-22 武汉烽火技术服务有限公司 一种基于iptv业务的vr全景视频播放方法和系统

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109462775A (zh) * 2017-09-05 2019-03-12 中兴通讯股份有限公司 全景视频播放方法、机顶盒、计算机可读存储介质
CN110384921B (zh) * 2018-04-23 2023-03-28 北京五一视界数字孪生科技股份有限公司 一种基于自适应边缘视野遮挡的vr应用防眩晕技术
CN110809148A (zh) * 2019-11-26 2020-02-18 大连海事大学 一种海域搜索系统及三维环境沉浸式体验vr智能眼镜
CN112843693B (zh) * 2020-12-31 2023-12-29 上海米哈游天命科技有限公司 拍摄图像的方法、装置、电子设备及存储介质
CN114168101A (zh) * 2021-12-01 2022-03-11 北京凯视达科技股份有限公司 一种将球形全景视频贴到立方体屏幕上显示的方法及装置
CN116616045B (zh) * 2023-06-07 2023-11-24 山东农业工程学院 一种基于植物生长的采摘方法及采摘系统

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090058850A1 (en) * 2007-09-04 2009-03-05 Wey Fun System and method for intuitive interactive navigational control in virtual environments
CN102486872A (zh) * 2010-12-06 2012-06-06 上海杉达学院 三维对象的建模方法
CN102568029A (zh) * 2010-12-07 2012-07-11 上海杉达学院 基于vrml的虚拟现实构建方法
US20140176607A1 (en) * 2012-12-24 2014-06-26 Electronics & Telecommunications Research Institute Simulation system for mixed reality content
CN104268939A (zh) * 2014-09-28 2015-01-07 国家电网公司 基于三维全景视图的变电站虚拟现实管理系统及其实现方法
CN105654539A (zh) * 2015-12-31 2016-06-08 中国南方电网有限责任公司 变电站运行监控的三维可视化方法
CN106296819A (zh) * 2016-08-12 2017-01-04 北京航空航天大学 一种基于智能机顶盒的全景视频播放器

Family Cites Families (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5940140A (en) * 1996-09-10 1999-08-17 Ultimatte Corporation Backing luminance non-uniformity compensation in real-time compositing systems
WO1999063488A1 (fr) * 1998-06-02 1999-12-09 Sony Corporation Dispositif et procede de traitement d'images
US7523411B2 (en) * 2000-08-22 2009-04-21 Bruce Carlin Network-linked interactive three-dimensional composition and display of saleable objects in situ in viewer-selected scenes for purposes of object promotion and procurement, and generation of object advertisements
US7684641B1 (en) * 2005-12-13 2010-03-23 Nvidia Corporation Inside testing for paths using a derivative mask
CN100594519C (zh) * 2008-03-03 2010-03-17 北京航空航天大学 用球面全景摄像机实时生成增强现实环境光照模型的方法
US8810590B2 (en) * 2008-07-11 2014-08-19 Advanced Micro Devices, Inc. Method and apparatus for spatial binning on a GPU and global path planning to avoid spatially binned objects
US20120010995A1 (en) * 2008-10-23 2012-01-12 Savnor Technologies Web content capturing, packaging, distribution
US8363928B1 (en) * 2010-12-24 2013-01-29 Trimble Navigation Ltd. General orientation positioning system
EP2608153A1 (en) * 2011-12-21 2013-06-26 Harman Becker Automotive Systems GmbH Method and system for playing an augmented reality game in a motor vehicle
CN102982194A (zh) * 2012-10-26 2013-03-20 李宝林 一种3d产品在线体验系统
CN103402106B (zh) * 2013-07-25 2016-01-06 青岛海信电器股份有限公司 三维图像显示方法及装置
US20150358539A1 (en) * 2014-06-06 2015-12-10 Jacob Catt Mobile Virtual Reality Camera, Method, And System
JP6548998B2 (ja) * 2015-09-07 2019-07-24 株式会社東芝 人物検索システムおよび人物検索方法
CN109804297B (zh) * 2016-08-03 2021-09-07 米拉维兹公司 逆反射屏幕、显示系统和显示图像的方法
CN108629850B (zh) * 2017-03-24 2021-06-22 长沙眸瑞网络科技有限公司 3d模型的移动端展示交互实现方法

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090058850A1 (en) * 2007-09-04 2009-03-05 Wey Fun System and method for intuitive interactive navigational control in virtual environments
CN102486872A (zh) * 2010-12-06 2012-06-06 上海杉达学院 三维对象的建模方法
CN102568029A (zh) * 2010-12-07 2012-07-11 上海杉达学院 基于vrml的虚拟现实构建方法
US20140176607A1 (en) * 2012-12-24 2014-06-26 Electronics & Telecommunications Research Institute Simulation system for mixed reality content
CN104268939A (zh) * 2014-09-28 2015-01-07 国家电网公司 基于三维全景视图的变电站虚拟现实管理系统及其实现方法
CN105654539A (zh) * 2015-12-31 2016-06-08 中国南方电网有限责任公司 变电站运行监控的三维可视化方法
CN106296819A (zh) * 2016-08-12 2017-01-04 北京航空航天大学 一种基于智能机顶盒的全景视频播放器

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109872400A (zh) * 2019-02-18 2019-06-11 上海电气集团股份有限公司 一种全景虚拟现实场景的生成方法
CN109872400B (zh) * 2019-02-18 2023-07-28 上海电气集团股份有限公司 一种全景虚拟现实场景的生成方法
CN115379277A (zh) * 2022-08-22 2022-11-22 武汉烽火技术服务有限公司 一种基于iptv业务的vr全景视频播放方法和系统
CN115379277B (zh) * 2022-08-22 2023-07-25 烽火通信科技股份有限公司 一种基于iptv业务的vr全景视频播放方法和系统

Also Published As

Publication number Publication date
AU2017384696B2 (en) 2019-10-03
US20210027518A1 (en) 2021-01-28
CN106658148B (zh) 2020-04-10
US10977852B2 (en) 2021-04-13
CN106658148A (zh) 2017-05-10
AU2017384696A1 (en) 2018-08-02

Similar Documents

Publication Publication Date Title
WO2018129792A1 (zh) Vr播放方法、vr播放装置及vr播放系统
WO2018128472A1 (en) Virtual reality experience sharing
WO2015186964A1 (ko) 촬영 장치 및 촬영 장치의 비디오 생성방법
WO2014182112A1 (en) Display apparatus and control method thereof
WO2017124792A1 (zh) 动态捕捉人脸摄像的方法、系统及移动终端
EP3603054A1 (en) Method and apparatus for rendering timed text and graphics in virtual reality video
WO2020162673A1 (en) Electronic device for providing avatar animation and method thereof
WO2019035581A1 (ko) 서버, 디스플레이장치 및 그 제어방법
WO2018088730A1 (en) Display apparatus and control method thereof
WO2018021707A1 (ko) Vr영상광고 시스템 및 vr광고제작시스템
WO2010114337A2 (ko) 3차원 멀티뷰 장치, 제공방법 및 기록매체
WO2023277253A1 (en) Automatic representation switching based on depth camera field of view
WO2019037542A1 (zh) 电视信源的预览方法、装置及计算机可读存储介质
WO2019035551A1 (en) APPARATUS FOR COMPOSING OBJECTS USING DEPTH CARD AND ASSOCIATED METHOD
WO2019071548A1 (zh) 终端拍照控制方法、移动终端及可读储存介质
WO2014107025A1 (en) Method and apparatus for controlling contents in electronic device
WO2019114587A1 (zh) 虚拟现实终端的信息处理方法、装置及可读存储介质
EP2596625A2 (en) Method and apparatus for reproducing 3d content
WO2014193093A1 (en) Display apparatus and control method thereof
WO2021080290A1 (en) Electronic apparatus and control method thereof
WO2017164545A1 (ko) 디스플레이 장치, 및 디스플레이 장치의 제어방법
WO2020147161A1 (zh) 一种交互显示装置及方法
WO2018164317A1 (ko) 전방향 영상의 방향 정보를 생성하는 방법 및 이러한 방법을 수행하는 장치
WO2020075925A1 (ko) 모바일 장치 및 모바일 장치의 제어 방법
WO2020213783A1 (ko) 가상 인터렉티브 컨텐츠의 사용자 인터페이스 제공 시스템, 방법 및 이를 위한 컴퓨터 프로그램이 저장된 기록매체

Legal Events

Date Code Title Description
ENP Entry into the national phase

Ref document number: 2017384696

Country of ref document: AU

Date of ref document: 20170224

Kind code of ref document: A

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17891615

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17891615

Country of ref document: EP

Kind code of ref document: A1