WO2019080792A1 - Panoramic video image playing method and device, storage medium and electronic device - Google Patents
Panoramic video image playing method and device, storage medium and electronic deviceInfo
- Publication number
- WO2019080792A1 WO2019080792A1 PCT/CN2018/111185 CN2018111185W WO2019080792A1 WO 2019080792 A1 WO2019080792 A1 WO 2019080792A1 CN 2018111185 W CN2018111185 W CN 2018111185W WO 2019080792 A1 WO2019080792 A1 WO 2019080792A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- region
- area
- interest
- image
- pixel width
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 65
- 230000015654 memory Effects 0.000 claims description 18
- 238000004590 computer program Methods 0.000 claims description 4
- 229910003460 diamond Inorganic materials 0.000 description 24
- 239000010432 diamond Substances 0.000 description 24
- 230000008569 process Effects 0.000 description 15
- 238000010586 diagram Methods 0.000 description 8
- 238000010009 beating Methods 0.000 description 6
- 239000011521 glass Substances 0.000 description 6
- 230000009471 action Effects 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000009877 rendering Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 1
- 230000009191 jumping Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
- H04N21/4312—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/20—Image signal generators
- H04N13/275—Image signal generators from 3D object models, e.g. computer-generated stereoscopic image signals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/30—Image reproducers
- H04N13/363—Image reproducers using image projection screens
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/433—Content storage operation, e.g. storage operation in response to a pause request, caching operations
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/433—Content storage operation, e.g. storage operation in response to a pause request, caching operations
- H04N21/4331—Caching operations, e.g. of an advertisement for later insertion during playback
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/44004—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving video buffer management, e.g. video decoder buffer or video display buffer
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
- H04N21/4722—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for requesting additional data associated with the content
- H04N21/4725—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for requesting additional data associated with the content using interactive regions of the image, e.g. hot spots
Definitions
- the present application relates to the field of computers, and in particular, to a method, an apparatus, a storage medium, and an electronic device for playing a panoramic video image.
- the method of playing panoramic video is to convert the video data stream into a FOV video stream.
- the field of view (FOV) image is an image between the edge of the display device and the angle of the line connecting the observation point (eye). It is said that the user can clearly see the content of the picture and the residual light, representing the angle image that the user sees.
- the video data stream is converted into a FOV video stream.
- the bottom of the model is converted to the maximum extent of the picture quality.
- the four sides of the conversion model are generated by projection, and there is a certain image distortion.
- the scheme moves in the head (ie, the observation point). When the FOV video is switched, there will be a certain time delay due to the current video stream i switching to the next video stream j.
- the picture is still presented. Is the image on the side of the video i, due to the difference in the angle of view of the video i side image and the video of the bottom positive angle image on the projection, which causes the user to feel the jitter of the video when viewing the video, and also because of the pixel density and pyramid at the bottom of the conversion model.
- the difference in pixel spatial density on the side is too large, and the sharpness also varies greatly.
- the embodiment of the present invention provides a method, a device, a storage medium, and an electronic device for playing a panoramic video image, so as to at least solve the technical problem that the video image may be beat when the video stream is switched in the related art.
- a method for playing a panoramic video image including: acquiring a first image image that matches a first viewing angle from a panoramic video image frame in video stream data to be played, and Obtaining, from the panoramic video image frame, a second image frame that matches a second viewing angle, where the first viewing angle is a current playback perspective, and the second perspective is a perspective adjacent to the first perspective; Determining a first region of interest included in the first image frame and determining a second region of interest included in the second image frame; according to the first region of interest and the second region of interest And determining, by the region, the view switching buffer region included in the first image frame, wherein the view switching buffer region is a region corresponding to the second region of interest in a corresponding region on the first image frame a partial region adjacent to the first region of interest, the first region of interest and a view switching buffer region constituting a first region in the first image frame, the first image An area other than the first area in the picture is a
- a playback apparatus for a panoramic video image including: an acquisition module, configured to: when playing video data in a virtual reality scene, from video stream data to be played Obtaining a first image frame matching the first viewing angle in the panoramic video image frame, and acquiring a second image frame matching the second viewing angle from the panoramic video image frame, wherein the first viewing angle is current Playing a viewing angle, the second viewing angle is a viewing angle adjacent to the first viewing angle; the first determining module is configured to determine a first region of interest included in the first image frame, and determine the first a second region of interest included in the two image frames; a second determining module configured to determine a viewing angle switch included in the first image frame according to the first region of interest and the second region of interest a buffer area, wherein the view switching buffer area is a picture corresponding to the second area of interest in a corresponding area on the first image picture and the first area of interest An adjacent partial region, the first region of interest and a view switching buffer
- a storage medium comprising a stored program, wherein the program is executed to perform the method described in any of the above.
- an electronic device comprising a memory, a processor, and a computer program stored on the memory and operable on the processor, the processor
- the computer program performs the method described in any of the above.
- a first image image matching the first viewing angle is acquired from the panoramic video image frame in the video stream data to be played, and the second viewing angle is matched from the panoramic video image frame.
- a second image frame wherein the first view angle is a current play view angle, the second view angle is a view angle adjacent to the first view angle; determining a first region of interest included in the first image frame, and determining a second image frame a second region of interest included; determining a view switching buffer region included in the first image frame according to the first region of interest and the second region of interest, wherein the view switching buffer region is a picture corresponding to the second region of interest a partial region adjacent to the first region of interest in the corresponding region on the first image frame, the first region of interest and the view switching buffer region constitute a first region in the first image frame, the first image image is divided by the first image The area outside the area is the second area; the image on the first area is played at the resolution corresponding to the first area, and the second is played in the resolution
- the first image frame matched by the first angle of view in the panoramic video image frame includes a first area and a second area, wherein the first area includes a first area of interest and a view switching buffer area, and the view switching buffer area is
- the picture corresponding to the second region of interest is a partial region adjacent to the first region of interest in the corresponding region on the first image frame, and the images on the first region and the second region are respectively played at different resolutions, wherein
- the resolution corresponding to an area is higher than the resolution corresponding to the second area, so that during the playback of the panoramic video image, during the process of switching the video picture from the first view to the second view, the user's line of sight will sweep through the first area.
- the middle view switchover screen has a higher resolution on the buffer area, so that when the view angle is switched, the bounce of the picture can be avoided, so that the image of the adjacent view is more smoothly connected, thereby overcoming the video picture generated when the video stream is switched in the related art.
- the problem of beating is a higher resolution on the buffer area, so that when the view angle is switched, the bounce of the picture can be avoided, so that the image of the adjacent view is more smoothly connected, thereby overcoming the video picture generated when the video stream is switched in the related art.
- the problem of beating is a higher resolution on the buffer area, so that when the view angle is switched, the bounce of the picture can be avoided, so that the image of the adjacent view is more smoothly connected, thereby overcoming the video picture generated when the video stream is switched in the related art.
- FIG. 1 is a schematic diagram of an application environment of an optional method for playing a panoramic video image according to an embodiment of the present application
- FIG. 2 is a flowchart of an optional method for playing a panoramic video image according to an embodiment of the present application
- FIG. 3 is a schematic diagram of an optional method for playing a panoramic video image according to an embodiment of the present application.
- FIG. 4 is a schematic diagram of another optional method for playing a panoramic video image according to an embodiment of the present application.
- FIG. 5 is a schematic diagram of still another optional method for playing a panoramic video image according to an embodiment of the present application.
- FIG. 6 is a schematic diagram of an optional playback device for panoramic video images according to an embodiment of the present application.
- FIG. 7 is a schematic diagram of an application scenario of an optional method for playing a panoramic video image according to an embodiment of the present application
- FIG. 8 is a schematic diagram of an application scenario of another optional method for playing a panoramic video image according to an embodiment of the present application.
- FIG. 9 is a schematic diagram of an alternative electronic device in accordance with an embodiment of the present application.
- an embodiment of a method for playing a panoramic video image is provided.
- the method for playing the panoramic video image may be, but is not limited to, being applied to an application environment as shown in FIG. 1.
- the terminal 102 records the panoramic video content in the video stream data to be broadcasted in the field.
- the network 104 transmits the panoramic video image frame in the video stream data to the playback device 106 (such as Virtual Reality (VR) glasses), and the playback device 106 is in the panoramic video image frame in the video stream data to be played.
- the playback device 106 such as Virtual Reality (VR) glasses
- the area is the second area; the image corresponding to the first area is played at a resolution corresponding to the first area, and the image corresponding to the second area is played at a resolution corresponding to the second area, wherein the
- the first image frame matched by the first angle of view in the panoramic video image frame includes a first area and a second area, wherein the first area includes a first area of interest and a view switching buffer area, and the view switching buffer
- the area is a partial area of the picture corresponding to the second area of interest corresponding to the first area of interest in the corresponding area on the first image picture, and the images on the first area and the second area are respectively played at different resolutions,
- the resolution corresponding to the first area is higher than the resolution corresponding to the second area, so that during the playback of the panoramic video image, the user's line of sight will be swept during the process of switching the video picture from the first perspective to the second perspective.
- the view angle switches the picture with higher resolution on the buffer area, so that when the view angle is switched, the bounce of the picture can be avoided, so that the image of the adjacent view is more smoothly connected, thereby overcoming the video picture when the video stream is switched in the related art.
- beating There will be a problem with beating.
- the foregoing playback device may include, but is not limited to, at least one of the following: a mobile phone, a tablet computer, a notebook computer, a desktop PC, a digital television, a VR glasses, and the like for displaying live video stream data.
- Panoramic video image of hardware device may include, but is not limited to, at least one of the following: a wide area network, a metropolitan area network, and a local area network. The above is only an example, and the embodiment does not limit this.
- a method for playing a panoramic video image includes:
- the playback device acquires a first image frame that matches the first view from the panoramic video image frame in the video stream data to be played, and acquires a second image frame that matches the second view from the panoramic video image frame.
- the first perspective is a current playback perspective
- the second perspective is a perspective adjacent to the first perspective
- the playback device determines a first region of interest included in the first image frame, and determines a second region of interest included in the second image frame.
- the playback device determines, according to the first region of interest and the second region of interest, a view switching buffer region included in the first image frame, where the view switch buffer region is a picture corresponding to the second region of interest in the first image frame.
- a partial region adjacent to the first region of interest in the corresponding region, the first region of interest and the view switching buffer region constitute a first region in the first image frame, except for the first region in the first image frame
- the area is the second area;
- the playback device plays the image corresponding to the first region at a resolution corresponding to the first region, and plays the image corresponding to the second region at a resolution corresponding to the second region, where the resolution corresponding to the first region is higher than the second region. Corresponding resolution.
- the foregoing image encoding method may be, but is not limited to, being applied to a scenario in which a panoramic video image is played, such as a virtual reality (VR) video playback process, and a panoramic video playback process.
- the above play may include but is not limited to: video on demand, live video.
- the above video may be, but is not limited to, applied to a terminal application such as a video playing application, a game application, a shopping application, and the like. The above is only an example, and is not limited in this embodiment.
- the panoramic video image frame may be, but is not limited to, a 360-degree spherical panoramic image, wherein the 360-degree spherical panoramic image may be, but is not limited to, divided into multiple views (FOV).
- the image image corresponding to each view will generate a video stream data for playback by the playback device.
- the projection model of the video image provided by the method for playing the panoramic video image is similar to a diamond in shape, and may be referred to as a diamond model.
- the playback device is illustrated by taking VR glasses as an example.
- the left side of FIG. 3 shows that a 360-degree spherical panoramic image in a video can be viewed using VR glasses, and the right side of FIG. 3 shows a 360-degree spherical panoramic image.
- the built-in diamond model (each vertex is located on the spherical surface) is mapped and projected. Further, each view will get a corresponding image (including the projection of the diamond model on each face), as shown in FIG. 4, and the left side of FIG. 4 shows the perspective image of the diamond model after being projected on each face.
- Image processing such as stretching will result in an image picture matching the angle of view shown on the right side of Fig.
- the first region of interest is a Region of Interest (ROI) in the first image frame.
- ROI Region of Interest
- the first region may be a bottom region of the diamond model corresponding to the current playback angle, the bottom region including the first region of interest and the view switching buffer region around the first region of interest, and the other regions of the diamond model (eg, the side region) correspond to the second region region.
- the region of interest in each of the above views may be, but is not limited to, a rectangular region, a diamond region, a regular hexagon region, or other polygonal regions of any shape.
- the case where the region of interest is a rectangular region is taken as an example, and other situations are similar thereto, and details are not described herein again.
- the second viewing angle may be, but is not limited to, a plurality of viewing angles adjacent to the first viewing angle, and the second viewing angle may be, but not limited to, the first viewing angle.
- there are four second viewing angles adjacent to the first viewing angle which are a second viewing angle A, a second viewing angle B, a second viewing angle C, and a second viewing angle D, respectively.
- the corresponding region of the second region of interest on the first image frame is region A, region B, region C, and region D, respectively, in each of the above regions.
- the partial area is used as the view switching buffer area, then the first area in the first image picture includes the view switching buffer area A, the view switching buffer area B, the view switching buffer area C, the view switching buffer area D, and the first region of interest. .
- the other area of the first image picture other than the above area is the second area.
- the first image frame matched by the first view in the panoramic video image frame includes a first region and a second region, wherein the first region includes the first region of interest and the view switching buffer region, and the view switching buffer
- the area is a partial area of the picture corresponding to the second area of interest corresponding to the first area of interest in the corresponding area on the first image picture, and the images on the first area and the second area are respectively played at different resolutions,
- the resolution corresponding to the first area is higher than the resolution corresponding to the second area, so that during the playback of the panoramic video image, the user's line of sight will be swept during the process of switching the video picture from the first perspective to the second perspective.
- the view angle switches the picture with higher resolution on the buffer area, so that when the view angle is switched, the bounce of the picture can be avoided, so that the image of the adjacent view is more smoothly connected, thereby overcoming the video picture when the video stream is switched in the related art.
- beating There will be a problem with beating.
- determining, according to the first region of interest and the second region of interest, the view switching buffer region included in the first image frame includes:
- the playback device determines an area corresponding to the picture corresponding to the second region of interest on the first image frame.
- the playback device determines that a region of the corresponding region that is adjacent to the first region of interest has a pixel width of the target pixel width as a view switching buffer region.
- the regions of interest of two adjacent viewing angles may be adjacent, and thus may, but are not limited to, determine a region in which the pixel width adjacent to the first region of interest is the target pixel width. Switch the buffer area for the angle of view.
- the target pixel width may be, but is not limited to, being set according to an actual application scenario of the playback method of the panoramic video image.
- the method before determining that the area of the target pixel width adjacent to the first region of interest is the view switching buffer area, the method further includes:
- the playback device obtains a pixel width corresponding to a jump angle, where the jump angle is an angle that is rotated when the video data corresponding to the first view angle is converted to the video data corresponding to the second view angle.
- the playback device determines the target pixel width according to the pixel width corresponding to the jump angle.
- the video sequence currently viewed by the user is a video sequence corresponding to the first perspective.
- the second perspective can be started when the head posture crosses the bottom angle position.
- the video sequence so that the target pixel width can be configured according to the angle of rotation when the angle of view is rotated, and the angle of rotation when the video data corresponding to the first view angle is switched to the video data corresponding to the second view angle can be obtained, and corresponding to the angle according to the angle
- the pixel width determines the target pixel width.
- the target pixel width may be determined by one of the following methods:
- the pixel width corresponding to the jump angle is determined as the target pixel width.
- the pixel width corresponding to the jump angle can be used as the target pixel width to ensure that when the viewing point of the viewing angle is rotated from the first viewing angle to the second viewing angle, the second viewing angle is loaded while the line of sight sweeps through the viewing angle switching buffer area. Corresponding video data so that the user does not feel the beating of the picture.
- one pixel width is selected from a pixel width larger than a pixel width corresponding to the jump angle and smaller than a pixel width corresponding to the second region of interest, and one pixel width is determined as the target pixel width.
- one pixel width may be selected as the target pixel width in a pixel width corresponding to the jump angle, and the target pixel width is selected such that the view switching buffer area does not exceed the range of the second region of interest.
- playing the image corresponding to the first area at a resolution corresponding to the first area includes:
- the playback device plays the image on the first region of interest at a first resolution, and plays the image on the buffer region at a second resolution.
- the first resolution is higher than or equal to the second resolution.
- the second resolution is higher than the resolution corresponding to the second region.
- the region of interest is equivalent to the region directly opposite the user, and the viewing angle switching buffer region and the second region are equivalent to the region covered by the user's residual light and the user does not In the area seen, the user is often not very sensitive to the image scanned in the residual light. Therefore, the image of the viewing angle switching buffer area can be played at a resolution slightly lower than the resolution corresponding to the first region of interest, thereby It is possible, but not limited to, to play images on different areas with different resolutions.
- the resolution corresponding to the first region of interest may be, but is not limited to, a resolution corresponding to the view switching buffer region, and the resolution corresponding to the view switching buffer region may be, but not limited to, higher than the resolution corresponding to the second region.
- the occupancy of the system is reduced, thereby improving the playback efficiency of the video.
- the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, by hardware, but in many cases, the former is A better implementation.
- the technical solution of the present application which is essential or contributes to the related art, may be embodied in the form of a software product stored in a storage medium (such as ROM/RAM, disk, CD-ROM).
- the instructions include a number of instructions for causing a terminal device (which may be a cell phone, computer, server, or network device, etc.) to perform the methods of various embodiments of the present application.
- a playback device for implementing a panoramic video image of the above-described method for playing a panoramic video image, which is applied to a playback device.
- the device includes:
- the obtaining module 62 is configured to acquire a first image frame that matches the first viewing angle from the panoramic video image frame in the video stream data to be played, and obtain a matching with the second viewing angle from the panoramic video image frame. a second image frame, wherein the first view angle is a current view angle of view, and the second view angle is a view angle adjacent to the first view angle;
- the first determining module 64 is configured to determine a first region of interest included in the first image frame, and determine a second region of interest included in the second image frame;
- the second determining module 66 is configured to determine, according to the first region of interest and the second region of interest, a view switching buffer region included in the first image frame, where the view switching buffer region is corresponding to the second region of interest
- the first image region and the view switching buffer region form a first region in the first image frame, in the first image frame, in the corresponding region on the first image frame adjacent to the first region of interest
- the area other than the first area is the second area;
- the playback module 68 is configured to play the image corresponding to the first area at a resolution corresponding to the first area, and play the image corresponding to the second area at a resolution corresponding to the second area, where the resolution corresponding to the first area Higher than the resolution corresponding to the second region.
- the foregoing image encoding apparatus may be, but is not limited to, being applied to a scene for playing a panoramic video image, such as a virtual reality (VR) video playing process, and a panoramic video playing process.
- a panoramic video image such as a virtual reality (VR) video playing process
- the above play may include but is not limited to: video on demand, live video.
- the above video may be, but is not limited to, applied to a terminal application such as a video playing application, a game application, a shopping application, and the like.
- a terminal application such as a video playing application, a game application, a shopping application, and the like.
- the above is only an example, and is not limited in this embodiment.
- the panoramic video image frame may be, but is not limited to, a 360-degree spherical panoramic image, wherein the 360-degree spherical panoramic image may be, but is not limited to, divided into multiple views (FOV).
- the image image corresponding to each view will generate a video stream data for playback by the playback device.
- the projection model of the video image provided by the playback device of the panoramic video image is similar to a diamond in shape, and may be referred to as a diamond model.
- the playback device is illustrated by taking VR glasses as an example.
- the left side of FIG. 3 shows that a 360-degree spherical panoramic image in a video can be viewed using VR glasses, and the right side of FIG. 3 shows a 360-degree spherical panoramic image.
- the built-in diamond model (each vertex is located on the spherical surface) is mapped and projected. Further, each view will get a corresponding image (including the projection of the diamond model on each face), as shown in FIG. 4, and the left side of FIG. 4 shows the perspective image of the diamond model after being projected on each face.
- Image processing such as stretching will result in an image picture matching the angle of view shown on the right side of Fig.
- the first region of interest is a Region of Interest (ROI) in the first image frame.
- ROI Region of Interest
- the first region may be a bottom region of the diamond model corresponding to the current playback angle, the bottom region including the first region of interest and the view switching buffer region around the first region of interest, and the other regions of the diamond model (eg, the side region) correspond to the second region region.
- the region of interest in each of the above views may be, but is not limited to, a rectangular region, a diamond region, a regular hexagon region, or other polygonal regions of any shape.
- the case where the region of interest is a rectangular region is taken as an example, and other situations are similar thereto, and details are not described herein again.
- the second viewing angle may be, but is not limited to, a plurality of viewing angles adjacent to the first viewing angle, and the second viewing angle may be, but not limited to, the first viewing angle.
- there are four second viewing angles adjacent to the first viewing angle which are a second viewing angle A, a second viewing angle B, a second viewing angle C, and a second viewing angle D, respectively.
- the corresponding region of the second region of interest on the first image frame (the region of the shaded portion in the figure) is region A, region B, region C, region D, respectively, and a partial region in each of the above regions
- the first area in the first image picture includes the view switching buffer area A, the view switching buffer area B, the view switching buffer area C, the view switching buffer area D, and the first region of interest.
- the other area of the first image picture other than the above area is the second area.
- the first image frame matched by the first view in the panoramic video image frame includes a first region and a second region, wherein the first region includes the first region of interest and the view switching buffer region, and the view switching buffer
- the area is a partial area of the picture corresponding to the second area of interest corresponding to the first area of interest in the corresponding area on the first image picture, and the images on the first area and the second area are respectively played at different resolutions,
- the resolution corresponding to the first area is higher than the resolution corresponding to the second area, so that during the playback of the panoramic video image, the user's line of sight will be swept during the process of switching the video picture from the first perspective to the second perspective.
- the view angle switches the picture with higher resolution on the buffer area, so that when the view angle is switched, the bounce of the picture can be avoided, so that the image of the adjacent view is more smoothly connected, thereby overcoming the video picture when the video stream is switched in the related art.
- beating There will be a problem with beating.
- the second determining module includes:
- a first determining unit configured to determine an area corresponding to a picture corresponding to the second region of interest on the first image frame
- the second determining unit is configured to determine that a region of the corresponding region adjacent to the first region of interest has a pixel width of the target pixel width as a view switching buffer region.
- the regions of interest of two adjacent viewing angles may be adjacent, and thus may, but are not limited to, determine a region in which the pixel width adjacent to the first region of interest is the target pixel width. Switch the buffer area for the angle of view.
- the target pixel width may be, but is not limited to, being set according to an actual application scenario of the playback method of the panoramic video image.
- the above device further includes:
- an obtaining module configured to obtain a pixel width corresponding to a jump angle, wherein the jump angle is an angle that is rotated when the video data corresponding to the first viewing angle is converted to the video data corresponding to the second viewing angle;
- the third determining module is configured to determine the target pixel width according to the pixel width corresponding to the jump angle.
- the video sequence currently viewed by the user is a video sequence corresponding to the first perspective.
- the second perspective can be started when the head posture crosses the bottom angle position.
- the video sequence so that the target pixel width can be configured according to the angle of rotation when the angle of view is rotated, and the angle of rotation when the video data corresponding to the first view angle is switched to the video data corresponding to the second view angle can be obtained, and corresponding to the angle according to the angle
- the pixel width determines the target pixel width.
- the second determining module includes:
- a third determining unit configured to determine a pixel width corresponding to the jump angle as the target pixel width
- the fourth determining unit is configured to select one pixel width from a pixel width larger than a pixel width corresponding to the jump angle and smaller than a pixel width corresponding to the second region of interest, and determine one pixel width as the target pixel width.
- the target pixel width may be determined by one of the following methods:
- the pixel width corresponding to the jump angle may be used as the target pixel width to ensure that when the viewing point of the viewing angle is rotated from the first viewing angle to the second viewing angle, the second viewing angle is loaded while the line of sight sweeps through the viewing angle switching buffer area.
- the video data so that the user does not feel the beating of the picture.
- one pixel width may be selected as the target pixel width in a pixel width corresponding to the jump angle, and the target pixel width is selected such that the view switching buffer area does not exceed the range of the second region of interest.
- the play module includes:
- a playback unit configured to play an image on the first region of interest at a first resolution and to play an image on the buffer region at a second resolution, wherein the first resolution is higher than or equal to the second The resolution, the second resolution is higher than the resolution corresponding to the second region.
- the region of interest is equivalent to the region directly opposite the user, and the viewing angle switching buffer region and the second region are equivalent to the region covered by the user's residual light and the user does not In the area seen, the user is often not very sensitive to the image scanned in the residual light. Therefore, the image of the viewing angle switching buffer area can be played at a resolution slightly lower than the resolution corresponding to the first region of interest, thereby It is possible, but not limited to, to play images on different areas with different resolutions.
- the resolution corresponding to the first region of interest may be, but is not limited to, a resolution corresponding to the view switching buffer region, and the resolution corresponding to the view switching buffer region may be, but not limited to, higher than the resolution corresponding to the second region.
- the occupancy of the system is reduced, thereby improving the playback efficiency of the video.
- the application environment of the embodiment of the present application may be, but is not limited to, the reference to the application environment in the foregoing embodiments.
- An embodiment of the present application provides an optional application example for implementing the above-described method for playing a panoramic video image.
- the above method for playing a panoramic video image may be, but is not limited to, applied to a process of playing a video image as shown in FIG. 7.
- the process can include the following steps:
- Step S702 monitoring the gyro sensor of the mobile phone device, calculating the current posture position according to the current position information, and notifying the FovPlayer if the posture position is changed.
- Step S704 the FOV player (FovPlayer module) transmits information for indicating the FOV view switching to the perspective receiving (FovPv) module of the Fov Push Server (FovPush).
- the FOV player (FovPlayer module) has one and only one network connection with the Fov Push Server (FovPush), that is, the FovPlayer module is connected to the Fov Recv module, and is set to transmit FOV video stream information and control information for switching the position of the FOV attitude view.
- Step S706 the FovPlayer module receives the FOV video stream information transmitted by the Fov Recv module, and decodes the video frame to determine whether the current video frame FOV index changes.
- Step S708 the posture position information of the new video frame changes, and the logic of the VR Rendering Module (VR Render) is updated, so that the video rendering position is consistent with the posture position of the frame.
- VR Render VR Render
- the FOV video is switched from view A (FOV A) video to view B (FOV B) video, as shown in FIG. 8, and the left side of FIG. 8 shows the FOV video switching process using the pyramid model.
- FOV B video sequence that is, the process of passing through the FOV buffer area
- the image of the A sequence is still the side of the pyramid.
- the image of the area is more deformed than the bottom of the image, and the quality of the video quality is also general.
- due to image distortion on the side of the diamond it may also cause the user to watch the VR video to produce a sense of jumping.
- the right side of Figure 8 shows the FOV video switching process using the diamond model.
- the bottom of the model consists of two areas, the positive viewing angle area, and the viewing angle switching buffer area, when the VR helmet attitude moves to the angle of the FOV buffer area.
- the quality of the picture viewed by the user is still relatively clear relative to the bottom area.
- the FOV B video sequence is loaded, and when viewing the B video sequence, the viewing angle viewing area is the bottom area of the FOV B; therefore, when the video loading speed is sufficient, the FOV video sequence using the diamond model can always provide a stable high for the user. Clear quality picture.
- an electronic device for implementing the above-described method for playing a panoramic video image is further provided.
- the electronic device may include one or more (only shown in the figure)
- the processor 902, the memory 904, the sensor 906, the display 908, the encoder 910, and the transmission device 912 are provided.
- the memory 904 can be configured to store a software program and a module, such as a method and a device for playing a panoramic video image in the embodiment of the present application.
- processor 902 executes various functional applications and data processing, ie, image encoding methods, by running software programs and modules stored in memory 904.
- Memory 904 can include high speed random access memory and can also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid state memory.
- memory 904 can further include memory remotely located relative to processor 902, which can be connected to the terminal over a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.
- the transmission device 912 described above is arranged to receive or transmit data via a network. Examples of network options described above may include wired networks and wireless networks.
- the transmission device 912 includes a Network Interface Controller (NIC) that can be connected to other network devices and routers via a network cable to communicate with the Internet or a local area network.
- NIC Network Interface Controller
- transmission device 912 is a radio frequency (RF) module that is configured to communicate with the Internet wirelessly.
- RF radio frequency
- the memory 904 is configured to store preset action conditions and information of the preset rights user, and an application.
- FIG. 9 is only schematic, and the electronic device can also be a smart phone (such as an Android mobile phone, an iOS mobile phone, etc.), a tablet computer, an applause computer, and a mobile Internet device (Mobile Internet Devices, MID). ), PAD and other terminal devices.
- FIG. 9 does not limit the structure of the above electronic device.
- the electronic device may also include more or fewer components (such as a network interface, display device, etc.) than shown in FIG. 9, or have a different configuration than that shown in FIG.
- Embodiments of the present application also provide a storage medium.
- the foregoing storage medium may be located in at least one of the plurality of network devices in the network.
- the storage medium is arranged to store program code for performing the following steps:
- S3. Determine, according to the first region of interest and the second region of interest, a view switching buffer region included in the first image frame, where the view switching buffer region corresponds to the image corresponding to the second region of interest on the first image frame.
- a partial region adjacent to the first region of interest in the region, the first region of interest and the view switching buffer region constitute a first region in the first image frame, and the region other than the first region in the first image frame is Second area;
- the foregoing storage medium may include, but not limited to, a USB flash drive, a Read-Only Memory (ROM), a Random Access Memory (RAM), a mobile hard disk, and a magnetic memory.
- ROM Read-Only Memory
- RAM Random Access Memory
- a mobile hard disk e.g., a hard disk
- magnetic memory e.g., a hard disk
- the integrated unit in the above embodiment if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in the above-described computer readable storage medium.
- the technical solution of the present application may be embodied in the form of a software product, or the whole or part of the technical solution, which is stored in the storage medium, including
- the instructions are used to cause one or more computer devices (which may be a personal computer, server or network device, etc.) to perform all or part of the steps of the methods described in the various embodiments of the present application.
- the disclosed client may be implemented in other manners.
- the device embodiments described above are merely illustrative.
- the division of the unit is only a logical function division.
- multiple units or components may be combined or may be Integrate into another system, or some features can be ignored or not executed.
- the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, unit or module, and may be electrical or otherwise.
- the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.
- each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
- the above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.
- the first image frame matched by the first view in the panoramic video image frame of the present application includes a first region and a second region, wherein the first region includes the first region of interest and the view switching buffer region
- the view switching buffer area is a partial area of the picture corresponding to the second region of interest corresponding to the first region of interest in the corresponding region on the first image frame, and the first region and the second region are respectively played at different resolutions.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Databases & Information Systems (AREA)
- Human Computer Interaction (AREA)
- Television Signal Processing For Recording (AREA)
- Controls And Circuits For Display Device (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
Disclosed are a panoramic video image playing method and device, a storage medium and an electronic device. The method comprises: acquiring, from a panoramic video image frame in the to-be-played video streaming data, a first screen image matched with a first view angle, and acquiring, from the panoramic video image frame, a second screen image matched with a second view angle; determining a first area-of-interest included in the first screen image and a second area-of-interest included in the second screen image; determining, according to the first area-of-interest and the second area-of-interest, a view angle switching buffer region included in the first screen image; and playing images in the first area at a resolution ratio corresponding to the first area, and playing images in the second area at a resolution ratio corresponding to the second area. The present application solves the technical problem that video images will jitter when video streaming is switched in the related art.
Description
本申请要求于2017年10月23日提交中国专利局、申请号为2017109978988、发明名称“全景视频图像的播放方法、装置、存储介质及电子装置”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。This application claims the priority of the Chinese patent application filed on October 23, 2017, the Chinese Patent Office, the application number is 2017109978988, the invention name "playing method, device, storage medium and electronic device for panoramic video images", the entire contents of which are The citations are incorporated herein by reference.
本申请涉及计算机领域,具体而言,涉及一种全景视频图像的播放方法、装置、存储介质及电子装置。The present application relates to the field of computers, and in particular, to a method, an apparatus, a storage medium, and an electronic device for playing a panoramic video image.
目前播放全景视频的方式是将视频数据流转换成FOV视频流,视角(Field of View,简称为FOV)图像是显示设备边缘与观察点(眼睛)连线的夹角之间的图像,简单来说就是用户能清晰看见画面和余光扫到的内容,代表着用户所看到的角度图像。目前将视频数据流转换成FOV视频流的方式转换模型底部最大程度保留了画面质量,转换模型四个边通过投影方式来生成,会有一定的图像变形。该方案在头部(即观察点)运动,FOV视频切换的时候,由于当前的视频流i切换到下一个视频流j会存在一定时间的延时,在呈现视频j画面之前,画面呈现的依然是视频i侧边的图像,由于视频i侧面图像和视频j底部正视角图像在投影上的视角角度差别,会导致用户观看视频时感受到视频画面的跳动,另外由于转换模型底部像素密度和金字塔侧面的像素空间密度差值过大,清晰度也有很大的变动。Currently, the method of playing panoramic video is to convert the video data stream into a FOV video stream. The field of view (FOV) image is an image between the edge of the display device and the angle of the line connecting the observation point (eye). It is said that the user can clearly see the content of the picture and the residual light, representing the angle image that the user sees. At present, the video data stream is converted into a FOV video stream. The bottom of the model is converted to the maximum extent of the picture quality. The four sides of the conversion model are generated by projection, and there is a certain image distortion. The scheme moves in the head (ie, the observation point). When the FOV video is switched, there will be a certain time delay due to the current video stream i switching to the next video stream j. Before the video j picture is presented, the picture is still presented. Is the image on the side of the video i, due to the difference in the angle of view of the video i side image and the video of the bottom positive angle image on the projection, which causes the user to feel the jitter of the video when viewing the video, and also because of the pixel density and pyramid at the bottom of the conversion model. The difference in pixel spatial density on the side is too large, and the sharpness also varies greatly.
针对上述的问题,目前尚未提出有效的解决方案。In response to the above problems, no effective solution has been proposed yet.
发明内容Summary of the invention
本申请实施例提供了一种全景视频图像的播放方法、装置、存储介质及电子装置,以至少解决相关技术中视频流切换时视频画面会产生跳动的技术问题。The embodiment of the present invention provides a method, a device, a storage medium, and an electronic device for playing a panoramic video image, so as to at least solve the technical problem that the video image may be beat when the video stream is switched in the related art.
根据本申请实施例的一个方面,提供了一种全景视频图像的播放方法,包括:从待播放的视频流数据中的全景视频图像帧中获取与第一视角相匹配的第一图像画面,并从所述全景视频图像帧中获取与第二视角相匹配的第二图像画面,其中,所述第一视角为当前播放视角,所述第二视角为与所述第一视角相邻的视角;确定所述第一图像画面中所包含的第一感兴趣区域,并确定所述第二图像画面中所包含的第二感兴趣区域;根据所述第一感兴趣区域和所述第二感兴趣区域确定所述第一图像画面中所包含的视角切换缓冲区域,其中,所述视角切换缓冲区域为所述第二感兴趣区域对应的画面在所述第一图像画面上对应的区域中与所述第一感兴趣区域相邻的部分区域,所述第一感兴趣区域和视角切换缓冲区域组成所述第一图像画面中的第一区域,所述第一图像画面中除所述第一区域之外的区域为第二区域;以所述第一区域对应的分辨率播放所述第一区域上的图像,以所述第二区域对应的分辨率播放所述第二区域上的图像,其中,所述第一区域对应的分辨率高于所述第二区域对应的分辨率。According to an aspect of the embodiments of the present application, a method for playing a panoramic video image is provided, including: acquiring a first image image that matches a first viewing angle from a panoramic video image frame in video stream data to be played, and Obtaining, from the panoramic video image frame, a second image frame that matches a second viewing angle, where the first viewing angle is a current playback perspective, and the second perspective is a perspective adjacent to the first perspective; Determining a first region of interest included in the first image frame and determining a second region of interest included in the second image frame; according to the first region of interest and the second region of interest And determining, by the region, the view switching buffer region included in the first image frame, wherein the view switching buffer region is a region corresponding to the second region of interest in a corresponding region on the first image frame a partial region adjacent to the first region of interest, the first region of interest and a view switching buffer region constituting a first region in the first image frame, the first image An area other than the first area in the picture is a second area; playing an image on the first area at a resolution corresponding to the first area, and playing the resolution in a resolution corresponding to the second area An image on the second area, wherein the first area corresponds to a higher resolution than the second area.
根据本申请实施例的另一方面,还提供了一种全景视频图像的播放装置,包括:获取模块,被设置为在播放虚拟实境场景下的视频数据时,从待播放的视频流数据中的全景视频图像帧中获取与第一视角相匹配的第一图像画面,并从所述全景视频图像帧中获取与第二视角相匹配的第二图像画面,其中,所述第一视角为当前播放视角,所述第二视角为与所述第一视角相邻的视角;第一确定模块,被设置为确定所述第一图像画面中所包含的第一感兴趣区域,并确定所述第二图像画面中所包含的第二感兴趣区域;第二确定模块,被设置为根据所述第一感兴趣区域和所述第二感兴趣区域确定所述第一图像画面中所包含的视角切换缓冲区域,其中,所述视角切换缓冲区域为所述第二感兴趣区域对应的画面在所述第一图像画面上对应的区域中与所述第一感兴趣区域相邻的部分区域,所述第一感兴 趣区域和视角切换缓冲区域组成所述第一图像画面中的第一区域,所述第一图像画面中除所述第一区域之外的区域为第二区域;播放模块,被设置为以所述第一区域对应的分辨率播放所述第一区域上的图像,以所述第二区域对应的分辨率播放所述第二区域上的图像,其中,所述第一区域对应的分辨率高于所述第二区域对应的分辨率。According to another aspect of the embodiments of the present application, a playback apparatus for a panoramic video image is further provided, including: an acquisition module, configured to: when playing video data in a virtual reality scene, from video stream data to be played Obtaining a first image frame matching the first viewing angle in the panoramic video image frame, and acquiring a second image frame matching the second viewing angle from the panoramic video image frame, wherein the first viewing angle is current Playing a viewing angle, the second viewing angle is a viewing angle adjacent to the first viewing angle; the first determining module is configured to determine a first region of interest included in the first image frame, and determine the first a second region of interest included in the two image frames; a second determining module configured to determine a viewing angle switch included in the first image frame according to the first region of interest and the second region of interest a buffer area, wherein the view switching buffer area is a picture corresponding to the second area of interest in a corresponding area on the first image picture and the first area of interest An adjacent partial region, the first region of interest and a view switching buffer region constitute a first region in the first image frame, and an area of the first image frame other than the first region is a second area; a playing module, configured to play an image on the first area at a resolution corresponding to the first area, and play an image on the second area in a resolution corresponding to the second area, where The resolution corresponding to the first area is higher than the resolution corresponding to the second area.
根据本申请实施例的另一方面,还提供了一种存储介质,所述存储介质包括存储的程序,其中,所述程序运行时执行上述任一项中所述的方法。According to another aspect of an embodiment of the present application, there is also provided a storage medium comprising a stored program, wherein the program is executed to perform the method described in any of the above.
根据本申请实施例的另一方面,还提供了一种电子装置,包括存储器、处理器及存储在所述存储器上并可在所述处理器上运行的计算机程序,所述处理器通过所述计算机程序执行上述任一项中所述的方法。According to another aspect of an embodiment of the present application, there is also provided an electronic device comprising a memory, a processor, and a computer program stored on the memory and operable on the processor, the processor The computer program performs the method described in any of the above.
在本申请实施例中,从待播放的视频流数据中的全景视频图像帧中获取与第一视角相匹配的第一图像画面,并从全景视频图像帧中获取与第二视角相匹配的第二图像画面,其中,第一视角为当前播放视角,第二视角为与第一视角相邻的视角;确定第一图像画面中所包含的第一感兴趣区域,并确定第二图像画面中所包含的第二感兴趣区域;根据第一感兴趣区域和第二感兴趣区域确定第一图像画面中所包含的视角切换缓冲区域,其中,视角切换缓冲区域为第二感兴趣区域对应的画面在第一图像画面上对应的区域中与第一感兴趣区域相邻的部分区域,第一感兴趣区域和视角切换缓冲区域组成第一图像画面中的第一区域,第一图像画面中除第一区域之外的区域为第二区域;以第一区域对应的分辨率播放第一区域上的图像,以第二区域对应的分辨率播放第二区域上的图像,其中,第一区域对应的分辨率高于第二区域对应的分辨率。也就是说,全景视频图像帧中第一视角匹配的第一图像画面包括第一区域和第二区域,其中的第一区域包括第一感兴趣区域和视角切换缓冲区域,该视角切换缓冲区域是第二感兴趣区域对应的画面在第一图像画面上对应的区域中与第一感兴趣区域相邻的部分区域,分别以不同的分辨率播放第一区域和第二区域上的图像,其中第一区域对应的分辨率高于第二区域对应的分辨率,这样在全景视频图像 的播放过程中,视频画面从第一视角切换到第二视角的过程中,用户的视线将扫过第一区域中视角切换缓冲区域上分辨率较高的画面,从而在视角切换时,可避免画面的跳动,使得相邻视角的图像的衔接更为平滑,进而克服相关技术中视频流切换时视频画面会产生跳动的问题。In the embodiment of the present application, a first image image matching the first viewing angle is acquired from the panoramic video image frame in the video stream data to be played, and the second viewing angle is matched from the panoramic video image frame. a second image frame, wherein the first view angle is a current play view angle, the second view angle is a view angle adjacent to the first view angle; determining a first region of interest included in the first image frame, and determining a second image frame a second region of interest included; determining a view switching buffer region included in the first image frame according to the first region of interest and the second region of interest, wherein the view switching buffer region is a picture corresponding to the second region of interest a partial region adjacent to the first region of interest in the corresponding region on the first image frame, the first region of interest and the view switching buffer region constitute a first region in the first image frame, the first image image is divided by the first image The area outside the area is the second area; the image on the first area is played at the resolution corresponding to the first area, and the second is played in the resolution corresponding to the second area. The image on the domain, wherein the first region is higher than the second resolution corresponding to a resolution corresponding to the region. That is, the first image frame matched by the first angle of view in the panoramic video image frame includes a first area and a second area, wherein the first area includes a first area of interest and a view switching buffer area, and the view switching buffer area is The picture corresponding to the second region of interest is a partial region adjacent to the first region of interest in the corresponding region on the first image frame, and the images on the first region and the second region are respectively played at different resolutions, wherein The resolution corresponding to an area is higher than the resolution corresponding to the second area, so that during the playback of the panoramic video image, during the process of switching the video picture from the first view to the second view, the user's line of sight will sweep through the first area. The middle view switchover screen has a higher resolution on the buffer area, so that when the view angle is switched, the bounce of the picture can be avoided, so that the image of the adjacent view is more smoothly connected, thereby overcoming the video picture generated when the video stream is switched in the related art. The problem of beating.
此处所说明的附图用来提供对本申请的进一步理解,构成本申请的一部分,本申请的示意性实施例及其说明用于解释本申请,并不构成对本申请的不当限定。在附图中:The drawings described herein are intended to provide a further understanding of the present application, and are intended to be a part of this application. In the drawing:
图1是根据本申请实施例的一种可选的全景视频图像的播放方法的应用环境示意图;1 is a schematic diagram of an application environment of an optional method for playing a panoramic video image according to an embodiment of the present application;
图2是根据本申请实施例的一种可选的全景视频图像的播放方法的流程图;2 is a flowchart of an optional method for playing a panoramic video image according to an embodiment of the present application;
图3是根据本申请实施例的一种可选的全景视频图像的播放方法的示意图;FIG. 3 is a schematic diagram of an optional method for playing a panoramic video image according to an embodiment of the present application; FIG.
图4是根据本申请实施例的另一种可选的全景视频图像的播放方法的示意图;4 is a schematic diagram of another optional method for playing a panoramic video image according to an embodiment of the present application;
图5是根据本申请实施例的又一种可选的全景视频图像的播放方法的示意图;FIG. 5 is a schematic diagram of still another optional method for playing a panoramic video image according to an embodiment of the present application; FIG.
图6是根据本申请实施例的一种可选的全景视频图像的播放装置的示意图;6 is a schematic diagram of an optional playback device for panoramic video images according to an embodiment of the present application;
图7是根据本申请实施例的一种可选的全景视频图像的播放方法的应用场景示意图;FIG. 7 is a schematic diagram of an application scenario of an optional method for playing a panoramic video image according to an embodiment of the present application;
图8是根据本申请实施例的另一种可选的全景视频图像的播放方法的应用场景示意图;以及FIG. 8 is a schematic diagram of an application scenario of another optional method for playing a panoramic video image according to an embodiment of the present application;
图9是根据本申请实施例的一种可选的电子装置的示意图。9 is a schematic diagram of an alternative electronic device in accordance with an embodiment of the present application.
为了使本技术领域的人员更好地理解本申请方案,下面将结合本申请实施例中的附图,对本申请实施例中的技术方案进行清楚、完整地描述,显然,所描述的实施例仅仅是本申请一部分的实施例,而不是全部的实施例。基于本申请中的实施例,本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例,都应当属于本申请保护的范围。The technical solutions in the embodiments of the present application are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present application. It is an embodiment of the present application, and not all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present application without departing from the inventive scope shall fall within the scope of the application.
需要说明的是,本申请的说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别类似的对象,而不必用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换,以便这里描述的本申请的实施例能够以除了在这里图示或描述的那些以外的顺序实施。此外,术语“包括”和“具有”以及他们的任何变形,意图在于覆盖不排他的包含,例如,包含了一系列步骤或单元的过程、方法、系统、产品或设备不必限于清楚地列出的那些步骤或单元,而是可包括没有清楚地列出的或对于这些过程、方法、产品或设备固有的其它步骤或单元。It should be noted that the terms "first", "second" and the like in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or order. It is to be understood that the data so used may be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in a sequence other than those illustrated or described herein. In addition, the terms "comprises" and "comprises" and "the" and "the" are intended to cover a non-exclusive inclusion, for example, a process, method, system, product, or device that comprises a series of steps or units is not necessarily limited to Those steps or units may include other steps or units not explicitly listed or inherent to such processes, methods, products or devices.
在本申请实施例中,提供了一种上述全景视频图像的播放方法的实施例。作为一种可选的实施方式,该全景视频图像的播放方法可以但不限于应用于如图1所示的应用环境中,终端102在现场录制所要直播的视频流数据中的全景视频内容,通过网络104将上述视频流数据中的全景视频图像帧传输给播放设备106(如虚拟实境(Virtual Reality,简称VR)眼镜),播放设备106从待播放的视频流数据中的全景视频图像帧中获取与第一视角相匹配的第一图像画面,并从全景视频图像帧中获取与第二视角相匹配的第二图像画面,其中,第一视角为当前播放视角,第二视角为与第一视角相邻的视角;确定第一图像画面中所包含的第一感兴趣区域,并确定第二图像画面中所包含的第二感兴趣区域;根据第一感兴趣区域和第二感兴趣区域确定第一图像画面中所包含的视角切换缓冲区域,其中,视角切换缓冲区域为第二感兴趣区域对应的画面在第一图像画面上对应的区域中 与第一感兴趣区域相邻的部分区域,第一感兴趣区域和视角切换缓冲区域组成第一图像画面中的第一区域,第一图像画面中除第一区域之外的区域为第二区域;以第一区域对应的分辨率播放第一区域对应的图像,以第二区域对应的分辨率播放第二区域对应的图像,其中,第一区域对应的分辨率高于第二区域对应的分辨率。In the embodiment of the present application, an embodiment of a method for playing a panoramic video image is provided. As an optional implementation manner, the method for playing the panoramic video image may be, but is not limited to, being applied to an application environment as shown in FIG. 1. The terminal 102 records the panoramic video content in the video stream data to be broadcasted in the field. The network 104 transmits the panoramic video image frame in the video stream data to the playback device 106 (such as Virtual Reality (VR) glasses), and the playback device 106 is in the panoramic video image frame in the video stream data to be played. Obtaining a first image frame that matches the first viewing angle, and acquiring a second image image that matches the second viewing angle from the panoramic video image frame, where the first viewing angle is the current playback viewing angle, and the second viewing angle is the first a view angle adjacent to the view; determining a first region of interest included in the first image frame, and determining a second region of interest included in the second image frame; determining according to the first region of interest and the second region of interest a view switching buffer region included in the first image frame, wherein the view switching buffer region is a picture corresponding to the second region of interest in the first image a partial region adjacent to the first region of interest in the corresponding region on the surface, the first region of interest and the view switching buffer region constitute a first region in the first image frame, except for the first region in the first image frame The area is the second area; the image corresponding to the first area is played at a resolution corresponding to the first area, and the image corresponding to the second area is played at a resolution corresponding to the second area, wherein the resolution corresponding to the first area is higher than The resolution corresponding to the second region.
在本实施例中,全景视频图像帧中第一视角匹配的第一图像画面包括第一区域和第二区域,其中的第一区域包括第一感兴趣区域和视角切换缓冲区域,该视角切换缓冲区域是第二感兴趣区域对应的画面在第一图像画面上对应的区域中与第一感兴趣区域相邻的部分区域,分别以不同的分辨率播放第一区域和第二区域上的图像,其中第一区域对应的分辨率高于第二区域对应的分辨率,这样在全景视频图像的播放过程中,视频画面从第一视角切换到第二视角的过程中,用户的视线将扫过第一区域中视角切换缓冲区域上分辨率较高的画面,从而在视角切换时,可避免画面的跳动,使得相邻视角的图像的衔接更为平滑,进而克服相关技术中视频流切换时视频画面会产生跳动的问题。In this embodiment, the first image frame matched by the first angle of view in the panoramic video image frame includes a first area and a second area, wherein the first area includes a first area of interest and a view switching buffer area, and the view switching buffer The area is a partial area of the picture corresponding to the second area of interest corresponding to the first area of interest in the corresponding area on the first image picture, and the images on the first area and the second area are respectively played at different resolutions, The resolution corresponding to the first area is higher than the resolution corresponding to the second area, so that during the playback of the panoramic video image, the user's line of sight will be swept during the process of switching the video picture from the first perspective to the second perspective. In a region, the view angle switches the picture with higher resolution on the buffer area, so that when the view angle is switched, the bounce of the picture can be avoided, so that the image of the adjacent view is more smoothly connected, thereby overcoming the video picture when the video stream is switched in the related art. There will be a problem with beating.
可选地,在本实施例中,上述播放设备可以包括但不限于以下至少之一:手机、平板电脑、笔记本电脑、台式PC机、数字电视、VR眼镜及其他用于显示直播视频流数据中的全景视频图像的硬件设备。上述网络可以包括但不限于以下至少之一:广域网、城域网、局域网。上述只是一种示例,本实施例对此不做任何限定。Optionally, in this embodiment, the foregoing playback device may include, but is not limited to, at least one of the following: a mobile phone, a tablet computer, a notebook computer, a desktop PC, a digital television, a VR glasses, and the like for displaying live video stream data. Panoramic video image of hardware device. The above network may include, but is not limited to, at least one of the following: a wide area network, a metropolitan area network, and a local area network. The above is only an example, and the embodiment does not limit this.
根据本申请实施例,提供了一种全景视频图像的播放方法,如图2所示,该方法包括:According to an embodiment of the present application, a method for playing a panoramic video image is provided. As shown in FIG. 2, the method includes:
S202,播放设备从待播放的视频流数据中的全景视频图像帧中获取与第一视角相匹配的第一图像画面,并从全景视频图像帧中获取与第二视角相匹配的第二图像画面,其中,第一视角为当前播放视角,第二视角为与第一视角相邻的视角;S202. The playback device acquires a first image frame that matches the first view from the panoramic video image frame in the video stream data to be played, and acquires a second image frame that matches the second view from the panoramic video image frame. Wherein the first perspective is a current playback perspective, and the second perspective is a perspective adjacent to the first perspective;
S204,播放设备确定第一图像画面中所包含的第一感兴趣区域,并确 定第二图像画面中所包含的第二感兴趣区域;S204. The playback device determines a first region of interest included in the first image frame, and determines a second region of interest included in the second image frame.
S206,播放设备根据第一感兴趣区域和第二感兴趣区域确定第一图像画面中所包含的视角切换缓冲区域,其中,视角切换缓冲区域为第二感兴趣区域对应的画面在第一图像画面上对应的区域中与第一感兴趣区域相邻的部分区域,第一感兴趣区域和视角切换缓冲区域组成第一图像画面中的第一区域,第一图像画面中除第一区域之外的区域为第二区域;S206. The playback device determines, according to the first region of interest and the second region of interest, a view switching buffer region included in the first image frame, where the view switch buffer region is a picture corresponding to the second region of interest in the first image frame. a partial region adjacent to the first region of interest in the corresponding region, the first region of interest and the view switching buffer region constitute a first region in the first image frame, except for the first region in the first image frame The area is the second area;
S208,播放设备以第一区域对应的分辨率播放第一区域对应的图像,以第二区域对应的分辨率播放第二区域对应的图像,其中,第一区域对应的分辨率高于第二区域对应的分辨率。S208: The playback device plays the image corresponding to the first region at a resolution corresponding to the first region, and plays the image corresponding to the second region at a resolution corresponding to the second region, where the resolution corresponding to the first region is higher than the second region. Corresponding resolution.
可选地,在本实施例中,上述图像编码方法可以但不限于应用于播放全景视频图像的场景中,如虚拟实境(Virtual Reality,简称VR)视频播放过程中、全景视频播放过程中,上述播放可以包括但不限于:视频点播、视频直播。其中,上述视频可以但不限于应用于:视频播放应用、游戏应用、购物应用等等终端应用中。上述仅是一种示例,本实施例中对此不做任何限定。Optionally, in this embodiment, the foregoing image encoding method may be, but is not limited to, being applied to a scenario in which a panoramic video image is played, such as a virtual reality (VR) video playback process, and a panoramic video playback process. The above play may include but is not limited to: video on demand, live video. The above video may be, but is not limited to, applied to a terminal application such as a video playing application, a game application, a shopping application, and the like. The above is only an example, and is not limited in this embodiment.
可选地,在本实施例中,全景视频图像帧可以但不限于为360度球形全景图像,其中,该360度球形全景图像可以但不限于划分为多个视角(Field of View,简称FOV),每个视角对应的图像画面将生成一路视角视频流数据供播放设备进行播放。Optionally, in this embodiment, the panoramic video image frame may be, but is not limited to, a 360-degree spherical panoramic image, wherein the 360-degree spherical panoramic image may be, but is not limited to, divided into multiple views (FOV). The image image corresponding to each view will generate a video stream data for playback by the playback device.
可选地,在本实施例中,上述全景视频图像的播放方法提供的视频图像的投影模型从形状上来看类似钻石,可以称为钻石模型。Optionally, in this embodiment, the projection model of the video image provided by the method for playing the panoramic video image is similar to a diamond in shape, and may be referred to as a diamond model.
例如,如图3所示,播放设备以VR眼镜为例进行说明,图3左侧示出使用VR眼镜可以观看视频中的360度球形全景图像,图3右侧示出在360度球形全景图像中,内置钻石模型(各个顶点均位于球面)进行映射投影。进一步,每个视角将得到对应的图像画面(包含钻石模型在各个面上的投影),如图4所示,图4左侧示出钻石模型在各个面上的投影展开 后的视角图像,经过拉伸等图像处理,将得到图4右侧所示与该视角相匹配的图像画面,其中,阴影部分表示模型底部对应画面。可以看出,使用上述钻石投影模型对全景视频图像进行投影,模型底部的画面相对于金字塔模型面积更大,因此该模型相比金字塔模型具有更大范围的高清图像。钻石投影模型底部区域使用的像素点数量也比金字塔模型更多,图像质量更优。For example, as shown in FIG. 3, the playback device is illustrated by taking VR glasses as an example. The left side of FIG. 3 shows that a 360-degree spherical panoramic image in a video can be viewed using VR glasses, and the right side of FIG. 3 shows a 360-degree spherical panoramic image. In the middle, the built-in diamond model (each vertex is located on the spherical surface) is mapped and projected. Further, each view will get a corresponding image (including the projection of the diamond model on each face), as shown in FIG. 4, and the left side of FIG. 4 shows the perspective image of the diamond model after being projected on each face. Image processing such as stretching will result in an image picture matching the angle of view shown on the right side of Fig. 4, wherein the shaded portion indicates the corresponding picture at the bottom of the model. It can be seen that the panoramic video image is projected using the above diamond projection model, and the picture at the bottom of the model is larger than the pyramid model, so the model has a larger range of high definition images than the pyramid model. The bottom area of the diamond projection model also uses more pixels than the pyramid model, and the image quality is better.
可选地,在本实施例中,上述第一感兴趣区域为第一图像画面中的感兴趣区域(Region of Interest,简称ROI),例如,如图3右侧所示,第一区域可以为当前播放视角所对应的钻石模型的底部区域,该底部区域包括第一感兴趣区域以及第一感兴趣区域周围的视角切换缓冲区域,而钻石模型的其他区域(如侧面区域)则对应为第二区域。Optionally, in the embodiment, the first region of interest is a Region of Interest (ROI) in the first image frame. For example, as shown in the right side of FIG. 3, the first region may be a bottom region of the diamond model corresponding to the current playback angle, the bottom region including the first region of interest and the view switching buffer region around the first region of interest, and the other regions of the diamond model (eg, the side region) correspond to the second region region.
可选地,在本实施例中,上述每个视角中的感兴趣区域可以但不限于为矩形区域、菱形区域、正六边形区域或者其他任意形状的多边形区域。在本实施例中,以感兴趣区域为矩形区域为例,其他情况与此类似,在此不再赘述。Optionally, in this embodiment, the region of interest in each of the above views may be, but is not limited to, a rectangular region, a diamond region, a regular hexagon region, or other polygonal regions of any shape. In this embodiment, the case where the region of interest is a rectangular region is taken as an example, and other situations are similar thereto, and details are not described herein again.
可选地,在本实施例中,上述第二视角可以但不限于为与第一视角相邻的多个视角,以感兴趣区域为矩形区域为例,第二视角可以但不限于为与第一视角相邻的上、下、左、右四个视角。例如:如图5所示,与第一视角相邻的第二视角有四个,分别为第二视角A、第二视角B、第二视角C、第二视角D,这四个第二视角中的第二感兴趣区域对应的画面在第一图像画面上对应的区域(图中阴影部分的区域)分别为区域A、区域B、区域C、区域D,将每个上述每个区域中的部分区域作为视角切换缓冲区域,那么,第一图像画面中的第一区域就包括视角切换缓冲区域A、视角切换缓冲区域B、视角切换缓冲区域C、视角切换缓冲区域D和第一感兴趣区域。第一图像画面中除上述区域外的其它区域为第二区域。Optionally, in this embodiment, the second viewing angle may be, but is not limited to, a plurality of viewing angles adjacent to the first viewing angle, and the second viewing angle may be, but not limited to, the first viewing angle. Four perspectives of up, down, left, and right adjacent to each other. For example, as shown in FIG. 5, there are four second viewing angles adjacent to the first viewing angle, which are a second viewing angle A, a second viewing angle B, a second viewing angle C, and a second viewing angle D, respectively. The corresponding region of the second region of interest on the first image frame (the region of the shaded portion in the figure) is region A, region B, region C, and region D, respectively, in each of the above regions. The partial area is used as the view switching buffer area, then the first area in the first image picture includes the view switching buffer area A, the view switching buffer area B, the view switching buffer area C, the view switching buffer area D, and the first region of interest. . The other area of the first image picture other than the above area is the second area.
可见,通过上述步骤,全景视频图像帧中第一视角匹配的第一图像画面包括第一区域和第二区域,其中的第一区域包括第一感兴趣区域和视角 切换缓冲区域,该视角切换缓冲区域是第二感兴趣区域对应的画面在第一图像画面上对应的区域中与第一感兴趣区域相邻的部分区域,分别以不同的分辨率播放第一区域和第二区域上的图像,其中第一区域对应的分辨率高于第二区域对应的分辨率,这样在全景视频图像的播放过程中,视频画面从第一视角切换到第二视角的过程中,用户的视线将扫过第一区域中视角切换缓冲区域上分辨率较高的画面,从而在视角切换时,可避免画面的跳动,使得相邻视角的图像的衔接更为平滑,进而克服相关技术中视频流切换时视频画面会产生跳动的问题。It can be seen that, by using the above steps, the first image frame matched by the first view in the panoramic video image frame includes a first region and a second region, wherein the first region includes the first region of interest and the view switching buffer region, and the view switching buffer The area is a partial area of the picture corresponding to the second area of interest corresponding to the first area of interest in the corresponding area on the first image picture, and the images on the first area and the second area are respectively played at different resolutions, The resolution corresponding to the first area is higher than the resolution corresponding to the second area, so that during the playback of the panoramic video image, the user's line of sight will be swept during the process of switching the video picture from the first perspective to the second perspective. In a region, the view angle switches the picture with higher resolution on the buffer area, so that when the view angle is switched, the bounce of the picture can be avoided, so that the image of the adjacent view is more smoothly connected, thereby overcoming the video picture when the video stream is switched in the related art. There will be a problem with beating.
作为一种可选的方案,根据第一感兴趣区域和第二感兴趣区域确定第一图像画面中所包含的视角切换缓冲区域包括:As an optional solution, determining, according to the first region of interest and the second region of interest, the view switching buffer region included in the first image frame includes:
S1,播放设备确定第二感兴趣区域对应的画面在第一图像画面上对应的区域;S1. The playback device determines an area corresponding to the picture corresponding to the second region of interest on the first image frame.
S2,播放设备确定对应的区域中与第一感兴趣区域相邻的像素宽度为目标像素宽度的区域为视角切换缓冲区域。S2. The playback device determines that a region of the corresponding region that is adjacent to the first region of interest has a pixel width of the target pixel width as a view switching buffer region.
可选地,在本实施例中,两个相邻视角的感兴趣区域可以是相邻的,因此,可以但不限于将与第一感兴趣区域相邻的像素宽度为目标像素宽度的区域确定为视角切换缓冲区域。Optionally, in this embodiment, the regions of interest of two adjacent viewing angles may be adjacent, and thus may, but are not limited to, determine a region in which the pixel width adjacent to the first region of interest is the target pixel width. Switch the buffer area for the angle of view.
可选地,在本实施例中,上述目标像素宽度可以但不限于是根据该全景视频图像的播放方法的实际应用场景设置的。Optionally, in this embodiment, the target pixel width may be, but is not limited to, being set according to an actual application scenario of the playback method of the panoramic video image.
作为一种可选的方案,在确定第一感兴趣区域相邻的目标像素宽度的区域为视角切换缓冲区域之前,还包括:As an optional solution, before determining that the area of the target pixel width adjacent to the first region of interest is the view switching buffer area, the method further includes:
S1,播放设备获取跳转角度对应的像素宽度,其中,跳转角度为从播放第一视角对应的视频数据转换至播放第二视角对应的视频数据时转动的角度;S1. The playback device obtains a pixel width corresponding to a jump angle, where the jump angle is an angle that is rotated when the video data corresponding to the first view angle is converted to the video data corresponding to the second view angle.
S2,播放设备根据跳转角度对应的像素宽度确定目标像素宽度。S2. The playback device determines the target pixel width according to the pixel width corresponding to the jump angle.
可选地,在本实施例中,用户当前观看的视频序列为第一视角对应的视频序列,在FOV视频切换的时候,可以在头部姿态越过底部一定角度位置时,开始加载第二视角对应的视频序列,因此可以根据视角转换时转过的角度配置目标像素宽度,可以获取从播放第一视角对应的视频数据转换至播放第二视角对应的视频数据时转动的角度,并根据该角度对应的像素宽度确定目标像素宽度。Optionally, in this embodiment, the video sequence currently viewed by the user is a video sequence corresponding to the first perspective. When the FOV video is switched, the second perspective can be started when the head posture crosses the bottom angle position. The video sequence, so that the target pixel width can be configured according to the angle of rotation when the angle of view is rotated, and the angle of rotation when the video data corresponding to the first view angle is switched to the video data corresponding to the second view angle can be obtained, and corresponding to the angle according to the angle The pixel width determines the target pixel width.
可选地,在本实施例中,可以但不限于通过以下方式之一确定目标像素宽度:Optionally, in this embodiment, the target pixel width may be determined by one of the following methods:
方式一,将跳转角度对应的像素宽度确定为目标像素宽度。在该方式中可以将跳转角度对应的像素宽度作为目标像素宽度,以确保在视角的观察点从第一视角转动到第二视角时,在视线扫过视角切换缓冲区域的同时加载第二视角对应的视频数据,从而使得用户不会感觉到画面的跳动。In the first method, the pixel width corresponding to the jump angle is determined as the target pixel width. In this manner, the pixel width corresponding to the jump angle can be used as the target pixel width to ensure that when the viewing point of the viewing angle is rotated from the first viewing angle to the second viewing angle, the second viewing angle is loaded while the line of sight sweeps through the viewing angle switching buffer area. Corresponding video data so that the user does not feel the beating of the picture.
方式二,从大于跳转角度对应的像素宽度且小于第二感兴趣区域对应的像素宽度的像素宽度中选取一个像素宽度,并将一个像素宽度确定为目标像素宽度。在该方式中可以在大于跳转角度对应的像素宽度中选取一个像素宽度作为目标像素宽度,并且该目标像素宽度的选取使得视角切换缓冲区域不超过第二感兴趣区域的范围。In the second manner, one pixel width is selected from a pixel width larger than a pixel width corresponding to the jump angle and smaller than a pixel width corresponding to the second region of interest, and one pixel width is determined as the target pixel width. In this manner, one pixel width may be selected as the target pixel width in a pixel width corresponding to the jump angle, and the target pixel width is selected such that the view switching buffer area does not exceed the range of the second region of interest.
作为一种可选的方案,以第一区域对应的分辨率播放第一区域对应的图像包括:As an optional solution, playing the image corresponding to the first area at a resolution corresponding to the first area includes:
S1,播放设备以第一分辨率播放第一感兴趣区域上的图像,并以第二分辨率播放视角切换缓冲区域上的图像,其中,第一分辨率高于或者等于第二分辨率,第二分辨率高于第二区域对应的分辨率。S1. The playback device plays the image on the first region of interest at a first resolution, and plays the image on the buffer region at a second resolution. The first resolution is higher than or equal to the second resolution. The second resolution is higher than the resolution corresponding to the second region.
可选地,在本实施例中,在用户观看全景视频时,感兴趣区域相当于是用户正对面的区域,视角切换缓冲区域和第二区域相当于是用户的余光覆盖到的区域以及用户不会看到的区域,用户往往对余光中扫到的图像不会很敏感,因此,对于视角切换缓冲区域的图像可以使用比第一感兴趣区 域对应的分辨率稍低的分辨率进行播放,由此,可以但不限于使用不同的分辨率播放不同区域上的图像。例如:第一感兴趣区域对应的分辨率可以但不限于高于视角切换缓冲区域对应的分辨率,视角切换缓冲区域对应的分辨率可以但不限于高于第二区域对应的分辨率。以使得在确保用户视角进行转动时不会明显感觉到画面跳动的同时,降低系统的占用率,从而也提高了视频的播放效率。Optionally, in this embodiment, when the user views the panoramic video, the region of interest is equivalent to the region directly opposite the user, and the viewing angle switching buffer region and the second region are equivalent to the region covered by the user's residual light and the user does not In the area seen, the user is often not very sensitive to the image scanned in the residual light. Therefore, the image of the viewing angle switching buffer area can be played at a resolution slightly lower than the resolution corresponding to the first region of interest, thereby It is possible, but not limited to, to play images on different areas with different resolutions. For example, the resolution corresponding to the first region of interest may be, but is not limited to, a resolution corresponding to the view switching buffer region, and the resolution corresponding to the view switching buffer region may be, but not limited to, higher than the resolution corresponding to the second region. In order to ensure that the screen is not obviously perceived when the user's perspective is rotated, the occupancy of the system is reduced, thereby improving the playback efficiency of the video.
需要说明的是,对于前述的各方法实施例,为了简单描述,故将其都表述为一系列的动作组合,但是本领域技术人员应该知悉,本申请并不受所描述的动作顺序的限制,因为依据本申请,某些步骤可以采用其他顺序或者同时进行。其次,本领域技术人员也应该知悉,说明书中所描述的实施例均属于可选实施例,所涉及的动作和模块并不一定是本申请所必须的。It should be noted that, for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations, but those skilled in the art should understand that the present application is not limited by the described action sequence. Because certain steps may be performed in other sequences or concurrently in accordance with the present application. In the following, those skilled in the art should also understand that the embodiments described in the specification are optional embodiments, and the actions and modules involved are not necessarily required by the present application.
通过以上的实施方式的描述,本领域的技术人员可以清楚地了解到根据上述实施例的方法可借助软件加必需的通用硬件平台的方式来实现,当然也可以通过硬件,但很多情况下前者是更佳的实施方式。基于这样的理解,本申请的技术方案本质上或者说对相关技术做出贡献的部分可以以软件产品的形式体现出来,该计算机软件产品存储在一个存储介质(如ROM/RAM、磁碟、光盘)中,包括若干指令用以使得一台终端设备(可以是手机,计算机,服务器,或者网络设备等)执行本申请各个实施例的方法。Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, by hardware, but in many cases, the former is A better implementation. Based on such understanding, the technical solution of the present application, which is essential or contributes to the related art, may be embodied in the form of a software product stored in a storage medium (such as ROM/RAM, disk, CD-ROM). The instructions include a number of instructions for causing a terminal device (which may be a cell phone, computer, server, or network device, etc.) to perform the methods of various embodiments of the present application.
根据本申请实施例,还提供了一种用于实施上述全景视频图像的播放方法的全景视频图像的播放装置,应用于播放设备,如图6所示,该装置包括:According to an embodiment of the present application, there is also provided a playback device for implementing a panoramic video image of the above-described method for playing a panoramic video image, which is applied to a playback device. As shown in FIG. 6, the device includes:
1)获取模块62,被设置为从待播放的视频流数据中的全景视频图像帧中获取与第一视角相匹配的第一图像画面,并从全景视频图像帧中获取与第二视角相匹配的第二图像画面,其中,第一视角为当前播放视角,第二视角为与第一视角相邻的视角;1) The obtaining module 62 is configured to acquire a first image frame that matches the first viewing angle from the panoramic video image frame in the video stream data to be played, and obtain a matching with the second viewing angle from the panoramic video image frame. a second image frame, wherein the first view angle is a current view angle of view, and the second view angle is a view angle adjacent to the first view angle;
2)第一确定模块64,被设置为确定第一图像画面中所包含的第一感 兴趣区域,并确定第二图像画面中所包含的第二感兴趣区域;2) The first determining module 64 is configured to determine a first region of interest included in the first image frame, and determine a second region of interest included in the second image frame;
3)第二确定模块66,被设置为根据第一感兴趣区域和第二感兴趣区域确定第一图像画面中所包含的视角切换缓冲区域,其中,视角切换缓冲区域为第二感兴趣区域对应的画面在第一图像画面上对应的区域中与第一感兴趣区域相邻的部分区域,第一感兴趣区域和视角切换缓冲区域组成第一图像画面中的第一区域,第一图像画面中除第一区域之外的区域为第二区域;3) The second determining module 66 is configured to determine, according to the first region of interest and the second region of interest, a view switching buffer region included in the first image frame, where the view switching buffer region is corresponding to the second region of interest The first image region and the view switching buffer region form a first region in the first image frame, in the first image frame, in the corresponding region on the first image frame adjacent to the first region of interest The area other than the first area is the second area;
4)播放模块68,被设置为以第一区域对应的分辨率播放第一区域对应的图像,以第二区域对应的分辨率播放第二区域对应的图像,其中,第一区域对应的分辨率高于第二区域对应的分辨率。The playback module 68 is configured to play the image corresponding to the first area at a resolution corresponding to the first area, and play the image corresponding to the second area at a resolution corresponding to the second area, where the resolution corresponding to the first area Higher than the resolution corresponding to the second region.
可选地,在本实施例中,上述图像编码装置可以但不限于应用于播放全景视频图像的场景中,如虚拟实境(Virtual Reality,简称VR)视频播放过程中、全景视频播放过程中,上述播放可以包括但不限于:视频点播、视频直播。其中,上述视频可以但不限于应用于:视频播放应用、游戏应用、购物应用等等终端应用中。上述仅是一种示例,本实施例中对此不做任何限定。Optionally, in this embodiment, the foregoing image encoding apparatus may be, but is not limited to, being applied to a scene for playing a panoramic video image, such as a virtual reality (VR) video playing process, and a panoramic video playing process. The above play may include but is not limited to: video on demand, live video. The above video may be, but is not limited to, applied to a terminal application such as a video playing application, a game application, a shopping application, and the like. The above is only an example, and is not limited in this embodiment.
可选地,在本实施例中,全景视频图像帧可以但不限于为360度球形全景图像,其中,该360度球形全景图像可以但不限于划分为多个视角(Field of View,简称FOV),每个视角对应的图像画面将生成一路视角视频流数据供播放设备进行播放。Optionally, in this embodiment, the panoramic video image frame may be, but is not limited to, a 360-degree spherical panoramic image, wherein the 360-degree spherical panoramic image may be, but is not limited to, divided into multiple views (FOV). The image image corresponding to each view will generate a video stream data for playback by the playback device.
可选地,在本实施例中,上述全景视频图像的播放装置提供的视频图像的投影模型从形状上来看类似钻石,可以称为钻石模型。Optionally, in this embodiment, the projection model of the video image provided by the playback device of the panoramic video image is similar to a diamond in shape, and may be referred to as a diamond model.
例如,如图3所示,播放设备以VR眼镜为例进行说明,图3左侧示出使用VR眼镜可以观看视频中的360度球形全景图像,图3右侧示出在360度球形全景图像中,内置钻石模型(各个顶点均位于球面)进行映射投影。进一步,每个视角将得到对应的图像画面(包含钻石模型在各个面 上的投影),如图4所示,图4左侧示出钻石模型在各个面上的投影展开后的视角图像,经过拉伸等图像处理,将得到图4右侧所示与该视角相匹配的图像画面,其中,阴影部分表示模型底部对应画面。可以看出,使用上述钻石投影模型对全景视频图像进行投影,模型底部的画面相对于金字塔模型面积更大,因此该模型相比金字塔模型具有更大范围的高清图像。钻石投影模型底部区域使用的像素点数量也比金字塔模型更多,图像质量更优。For example, as shown in FIG. 3, the playback device is illustrated by taking VR glasses as an example. The left side of FIG. 3 shows that a 360-degree spherical panoramic image in a video can be viewed using VR glasses, and the right side of FIG. 3 shows a 360-degree spherical panoramic image. In the middle, the built-in diamond model (each vertex is located on the spherical surface) is mapped and projected. Further, each view will get a corresponding image (including the projection of the diamond model on each face), as shown in FIG. 4, and the left side of FIG. 4 shows the perspective image of the diamond model after being projected on each face. Image processing such as stretching will result in an image picture matching the angle of view shown on the right side of Fig. 4, wherein the shaded portion indicates the corresponding picture at the bottom of the model. It can be seen that the panoramic video image is projected using the above diamond projection model, and the picture at the bottom of the model is larger than the pyramid model, so the model has a larger range of high definition images than the pyramid model. The bottom area of the diamond projection model also uses more pixels than the pyramid model, and the image quality is better.
可选地,在本实施例中,上述第一感兴趣区域为第一图像画面中的感兴趣区域(Region of Interest,简称ROI),例如,如图3右侧所示,第一区域可以为当前播放视角所对应的钻石模型的底部区域,该底部区域包括第一感兴趣区域以及第一感兴趣区域周围的视角切换缓冲区域,而钻石模型的其他区域(如侧面区域)则对应为第二区域。Optionally, in the embodiment, the first region of interest is a Region of Interest (ROI) in the first image frame. For example, as shown in the right side of FIG. 3, the first region may be a bottom region of the diamond model corresponding to the current playback angle, the bottom region including the first region of interest and the view switching buffer region around the first region of interest, and the other regions of the diamond model (eg, the side region) correspond to the second region region.
可选地,在本实施例中,上述每个视角中的感兴趣区域可以但不限于为矩形区域、菱形区域、正六边形区域或者其他任意形状的多边形区域。在本实施例中,以感兴趣区域为矩形区域为例,其他情况与此类似,在此不再赘述。Optionally, in this embodiment, the region of interest in each of the above views may be, but is not limited to, a rectangular region, a diamond region, a regular hexagon region, or other polygonal regions of any shape. In this embodiment, the case where the region of interest is a rectangular region is taken as an example, and other situations are similar thereto, and details are not described herein again.
可选地,在本实施例中,上述第二视角可以但不限于为与第一视角相邻的多个视角,以感兴趣区域为矩形区域为例,第二视角可以但不限于为与第一视角相邻的上、下、左、右四个视角。例如:如图5所示,与第一视角相邻的第二视角有四个,分别为第二视角A、第二视角B、第二视角C、第二视角D,这四个第二视角中的第二感兴趣区域对应的画面在第一图像画面上对应的区域(图中阴影部分的区域)分别为区域A、区域B、区域C、区域D,将上述每个区域中的部分区域作为视角切换缓冲区域,那么,第一图像画面中的第一区域就包括视角切换缓冲区域A、视角切换缓冲区域B、视角切换缓冲区域C、视角切换缓冲区域D和第一感兴趣区域。第一图像画面中除上述区域外的其它区域为第二区域。Optionally, in this embodiment, the second viewing angle may be, but is not limited to, a plurality of viewing angles adjacent to the first viewing angle, and the second viewing angle may be, but not limited to, the first viewing angle. Four perspectives of up, down, left, and right adjacent to each other. For example, as shown in FIG. 5, there are four second viewing angles adjacent to the first viewing angle, which are a second viewing angle A, a second viewing angle B, a second viewing angle C, and a second viewing angle D, respectively. The corresponding region of the second region of interest on the first image frame (the region of the shaded portion in the figure) is region A, region B, region C, region D, respectively, and a partial region in each of the above regions As the view switching buffer area, the first area in the first image picture includes the view switching buffer area A, the view switching buffer area B, the view switching buffer area C, the view switching buffer area D, and the first region of interest. The other area of the first image picture other than the above area is the second area.
可见,通过上述装置,全景视频图像帧中第一视角匹配的第一图像画 面包括第一区域和第二区域,其中的第一区域包括第一感兴趣区域和视角切换缓冲区域,该视角切换缓冲区域是第二感兴趣区域对应的画面在第一图像画面上对应的区域中与第一感兴趣区域相邻的部分区域,分别以不同的分辨率播放第一区域和第二区域上的图像,其中第一区域对应的分辨率高于第二区域对应的分辨率,这样在全景视频图像的播放过程中,视频画面从第一视角切换到第二视角的过程中,用户的视线将扫过第一区域中视角切换缓冲区域上分辨率较高的画面,从而在视角切换时,可避免画面的跳动,使得相邻视角的图像的衔接更为平滑,进而克服相关技术中视频流切换时视频画面会产生跳动的问题。It can be seen that, by using the foregoing apparatus, the first image frame matched by the first view in the panoramic video image frame includes a first region and a second region, wherein the first region includes the first region of interest and the view switching buffer region, and the view switching buffer The area is a partial area of the picture corresponding to the second area of interest corresponding to the first area of interest in the corresponding area on the first image picture, and the images on the first area and the second area are respectively played at different resolutions, The resolution corresponding to the first area is higher than the resolution corresponding to the second area, so that during the playback of the panoramic video image, the user's line of sight will be swept during the process of switching the video picture from the first perspective to the second perspective. In a region, the view angle switches the picture with higher resolution on the buffer area, so that when the view angle is switched, the bounce of the picture can be avoided, so that the image of the adjacent view is more smoothly connected, thereby overcoming the video picture when the video stream is switched in the related art. There will be a problem with beating.
作为一种可选的方案,第二确定模块包括:As an optional solution, the second determining module includes:
1)第一确定单元,被设置为确定第二感兴趣区域对应的画面在第一图像画面上对应的区域;1) a first determining unit, configured to determine an area corresponding to a picture corresponding to the second region of interest on the first image frame;
2)第二确定单元,被设置为确定对应的区域中与第一感兴趣区域相邻的像素宽度为目标像素宽度的区域为视角切换缓冲区域。2) The second determining unit is configured to determine that a region of the corresponding region adjacent to the first region of interest has a pixel width of the target pixel width as a view switching buffer region.
可选地,在本实施例中,两个相邻视角的感兴趣区域可以是相邻的,因此,可以但不限于将与第一感兴趣区域相邻的像素宽度为目标像素宽度的区域确定为视角切换缓冲区域。Optionally, in this embodiment, the regions of interest of two adjacent viewing angles may be adjacent, and thus may, but are not limited to, determine a region in which the pixel width adjacent to the first region of interest is the target pixel width. Switch the buffer area for the angle of view.
可选地,在本实施例中,上述目标像素宽度可以但不限于是根据该全景视频图像的播放方法的实际应用场景设置的。Optionally, in this embodiment, the target pixel width may be, but is not limited to, being set according to an actual application scenario of the playback method of the panoramic video image.
作为一种可选的方案,上述装置还包括:As an alternative, the above device further includes:
1)获取模块,被设置为获取跳转角度对应的像素宽度,其中,跳转角度为从播放第一视角对应的视频数据转换至播放第二视角对应的视频数据时转动的角度;1) an obtaining module, configured to obtain a pixel width corresponding to a jump angle, wherein the jump angle is an angle that is rotated when the video data corresponding to the first viewing angle is converted to the video data corresponding to the second viewing angle;
2)第三确定模块,被设置为根据跳转角度对应的像素宽度确定目标像素宽度。2) The third determining module is configured to determine the target pixel width according to the pixel width corresponding to the jump angle.
可选地,在本实施例中,用户当前观看的视频序列为第一视角对应的视频序列,在FOV视频切换的时候,可以在头部姿态越过底部一定角度位置时,开始加载第二视角对应的视频序列,因此可以根据视角转换时转过的角度配置目标像素宽度,可以获取从播放第一视角对应的视频数据转换至播放第二视角对应的视频数据时转动的角度,并根据该角度对应的像素宽度确定目标像素宽度。Optionally, in this embodiment, the video sequence currently viewed by the user is a video sequence corresponding to the first perspective. When the FOV video is switched, the second perspective can be started when the head posture crosses the bottom angle position. The video sequence, so that the target pixel width can be configured according to the angle of rotation when the angle of view is rotated, and the angle of rotation when the video data corresponding to the first view angle is switched to the video data corresponding to the second view angle can be obtained, and corresponding to the angle according to the angle The pixel width determines the target pixel width.
作为一种可选的方案,第二确定模块包括:As an optional solution, the second determining module includes:
1)第三确定单元,被设置为将跳转角度对应的像素宽度确定为目标像素宽度;或者,1) a third determining unit configured to determine a pixel width corresponding to the jump angle as the target pixel width; or
2)第四确定单元,被设置为从大于跳转角度对应的像素宽度且小于第二感兴趣区域对应的像素宽度的像素宽度中选取一个像素宽度,并将一个像素宽度确定为目标像素宽度。2) The fourth determining unit is configured to select one pixel width from a pixel width larger than a pixel width corresponding to the jump angle and smaller than a pixel width corresponding to the second region of interest, and determine one pixel width as the target pixel width.
可选地,在本实施例中,可以但不限于通过以下方式之一确定目标像素宽度:Optionally, in this embodiment, the target pixel width may be determined by one of the following methods:
方式一,可以将跳转角度对应的像素宽度作为目标像素宽度,以确保在视角的观察点从第一视角转动到第二视角时,在视线扫过视角切换缓冲区域的同时加载第二视角对应的视频数据,从而使得用户不会感觉到画面的跳动。In the first manner, the pixel width corresponding to the jump angle may be used as the target pixel width to ensure that when the viewing point of the viewing angle is rotated from the first viewing angle to the second viewing angle, the second viewing angle is loaded while the line of sight sweeps through the viewing angle switching buffer area. The video data so that the user does not feel the beating of the picture.
方式二,可以在大于跳转角度对应的像素宽度中选取一个像素宽度作为目标像素宽度,并且该目标像素宽度的选取使得视角切换缓冲区域不超过第二感兴趣区域的范围。In the second manner, one pixel width may be selected as the target pixel width in a pixel width corresponding to the jump angle, and the target pixel width is selected such that the view switching buffer area does not exceed the range of the second region of interest.
作为一种可选的方案,播放模块包括:As an alternative, the play module includes:
1)播放单元,被设置为以第一分辨率播放第一感兴趣区域上的图像,并以第二分辨率播放视角切换缓冲区域上的图像,其中,第一分辨率高于或者等于第二分辨率,第二分辨率高于第二区域对应的分辨率。1) a playback unit configured to play an image on the first region of interest at a first resolution and to play an image on the buffer region at a second resolution, wherein the first resolution is higher than or equal to the second The resolution, the second resolution is higher than the resolution corresponding to the second region.
可选地,在本实施例中,在用户观看全景视频时,感兴趣区域相当于是用户正对面的区域,视角切换缓冲区域和第二区域相当于是用户的余光覆盖到的区域以及用户不会看到的区域,用户往往对余光中扫到的图像不会很敏感,因此,对于视角切换缓冲区域的图像可以使用比第一感兴趣区域对应的分辨率稍低的分辨率进行播放,由此,可以但不限于使用不同的分辨率播放不同区域上的图像。例如:第一感兴趣区域对应的分辨率可以但不限于高于视角切换缓冲区域对应的分辨率,视角切换缓冲区域对应的分辨率可以但不限于高于第二区域对应的分辨率。以使得在确保用户视角进行转动时不会明显感觉到画面跳动的同时,降低系统的占用率,从而也提高了视频的播放效率。Optionally, in this embodiment, when the user views the panoramic video, the region of interest is equivalent to the region directly opposite the user, and the viewing angle switching buffer region and the second region are equivalent to the region covered by the user's residual light and the user does not In the area seen, the user is often not very sensitive to the image scanned in the residual light. Therefore, the image of the viewing angle switching buffer area can be played at a resolution slightly lower than the resolution corresponding to the first region of interest, thereby It is possible, but not limited to, to play images on different areas with different resolutions. For example, the resolution corresponding to the first region of interest may be, but is not limited to, a resolution corresponding to the view switching buffer region, and the resolution corresponding to the view switching buffer region may be, but not limited to, higher than the resolution corresponding to the second region. In order to ensure that the screen is not obviously perceived when the user's perspective is rotated, the occupancy of the system is reduced, thereby improving the playback efficiency of the video.
本申请实施例的应用环境可以但不限于参照上述实施例中的应用环境,本实施例中对此不再赘述。本申请实施例提供了用于实施上述全景视频图像的播放方法的一种可选的应用示例。The application environment of the embodiment of the present application may be, but is not limited to, the reference to the application environment in the foregoing embodiments. An embodiment of the present application provides an optional application example for implementing the above-described method for playing a panoramic video image.
作为一种可选的实施例,上述全景视频图像的播放方法可以但不限于应用于如图7所示的对视频图像进行播放的流程中。该流程可以包括如下步骤:As an optional embodiment, the above method for playing a panoramic video image may be, but is not limited to, applied to a process of playing a video image as shown in FIG. 7. The process can include the following steps:
步骤S702,监听手机设备的陀螺仪传感器,依据当前的位置信息计算当前的姿态位置,如果姿态位置发送变化,则通知FovPlayer。Step S702, monitoring the gyro sensor of the mobile phone device, calculating the current posture position according to the current position information, and notifying the FovPlayer if the posture position is changed.
步骤S704,FOV播放器(FovPlayer模块)将用于指示FOV视角切换的信息传输给Fov推送服务端(FovPush)的视角接收(Fov Recv)模块。FOV播放器(FovPlayer模块)与Fov推送服务端(FovPush)有且仅有一个网络连接,即FovPlayer模块与Fov Recv模块连接,被设置为传输FOV视频流信息和切换FOV姿态视角位置的控制信息。Step S704, the FOV player (FovPlayer module) transmits information for indicating the FOV view switching to the perspective receiving (FovPv) module of the Fov Push Server (FovPush). The FOV player (FovPlayer module) has one and only one network connection with the Fov Push Server (FovPush), that is, the FovPlayer module is connected to the Fov Recv module, and is set to transmit FOV video stream information and control information for switching the position of the FOV attitude view.
步骤S706,FovPlayer模块接收Fov Recv模块传输的FOV视频流信息,并解码视频帧,判断当前的视频帧FOV索引(FOV Index)是否发生 变化。Step S706, the FovPlayer module receives the FOV video stream information transmitted by the Fov Recv module, and decodes the video frame to determine whether the current video frame FOV index changes.
步骤S708,新视频帧的姿态位置信息发生变化,更新VR视频渲染模块(VR Render)的逻辑,使得视频渲染位置与帧的姿态位置保持一致。Step S708, the posture position information of the new video frame changes, and the logic of the VR Rendering Module (VR Render) is updated, so that the video rendering position is consistent with the posture position of the frame.
在一个可选的实施方式中,FOV视频从视角A(FOV A)视频切换到视角B(FOV B)视频,如图8所示,图8左侧示出使用金字塔模型的FOV视频切换过程,在金字塔模型中进行FOV视频切换的时候,一般在头部姿态越过底部一定角度位置时,开始加FOV B视频序列(即经过FOV缓冲区域的过程),此时观看A序列的图像依然为金字塔侧面区域位置图像,画质相对底部存在更大的变形,视频画质质量也一般。另外由于钻石侧面的图像畸变原因,也可能导致用户观看VR视频产生跳动感。In an alternative embodiment, the FOV video is switched from view A (FOV A) video to view B (FOV B) video, as shown in FIG. 8, and the left side of FIG. 8 shows the FOV video switching process using the pyramid model. When the FOV video is switched in the pyramid model, the FOV B video sequence (that is, the process of passing through the FOV buffer area) is generally started when the head posture crosses the bottom angle position. At this time, the image of the A sequence is still the side of the pyramid. The image of the area is more deformed than the bottom of the image, and the quality of the video quality is also general. In addition, due to image distortion on the side of the diamond, it may also cause the user to watch the VR video to produce a sense of jumping.
图8右侧示出使用钻石模型的FOV视频切换过程,在钻石投影模型之中,模型底部包括两块区域,正视角区域,视角切换缓冲区域,在VR头盔姿态移动到FOV缓冲区域的角度时,这时用户观看的画质相对底部区域依然为比较清楚的区域。此时加载FOV B视频序列,开始观看B视频序列时,视角观看区域为FOV B底部区域;所以在视频加载速度足够的情况下,采用钻石模型的FOV视频序列,可以给用户一直提供稳定的高质量清晰画面。The right side of Figure 8 shows the FOV video switching process using the diamond model. In the diamond projection model, the bottom of the model consists of two areas, the positive viewing angle area, and the viewing angle switching buffer area, when the VR helmet attitude moves to the angle of the FOV buffer area. At this time, the quality of the picture viewed by the user is still relatively clear relative to the bottom area. At this time, the FOV B video sequence is loaded, and when viewing the B video sequence, the viewing angle viewing area is the bottom area of the FOV B; therefore, when the video loading speed is sufficient, the FOV video sequence using the diamond model can always provide a stable high for the user. Clear quality picture.
根据本申请实施例的又一个方面,还提供了一种用于实施上述全景视频图像的播放方法的电子装置,如图9所示,该电子装置可以包括:一个或多个(图中仅示出一个)处理器902、存储器904、传感器906、显示器908、编码器910以及传输装置912。According to still another aspect of the embodiments of the present application, an electronic device for implementing the above-described method for playing a panoramic video image is further provided. As shown in FIG. 9, the electronic device may include one or more (only shown in the figure) The processor 902, the memory 904, the sensor 906, the display 908, the encoder 910, and the transmission device 912 are provided.
其中,存储器904可被设置为存储软件程序以及模块,如本申请实施例中的全景视频图像的播放方法及装置。The memory 904 can be configured to store a software program and a module, such as a method and a device for playing a panoramic video image in the embodiment of the present application.
对应的程序指令/模块,处理器902通过运行存储在存储器904内的软件程序以及模块,从而执行各种功能应用以及数据处理,即图像编码方法。存储器904可包括高速随机存储器,还可以包括非易失性存储器,如一个 或者多个磁性存储装置、闪存、或者其他非易失性固态存储器。在一些实例中,存储器904可进一步包括相对于处理器902远程设置的存储器,这些远程存储器可以通过网络连接至终端。上述网络的实例包括但不限于互联网、企业内部网、局域网、移动通信网及其组合。Corresponding program instructions/modules, processor 902 executes various functional applications and data processing, ie, image encoding methods, by running software programs and modules stored in memory 904. Memory 904 can include high speed random access memory and can also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid state memory. In some examples, memory 904 can further include memory remotely located relative to processor 902, which can be connected to the terminal over a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.
上述的传输装置912被设置为经由一个网络接收或者发送数据。上述的网络可选实例可包括有线网络及无线网络。在一个实例中,传输装置912包括一个网络适配器(Network Interface Controller,NIC),其可通过网线与其他网络设备与路由器相连从而可与互联网或局域网进行通讯。在一个实例中,传输装置912为射频(Radio Frequency,RF)模块,其被设置为通过无线方式与互联网进行通讯。The transmission device 912 described above is arranged to receive or transmit data via a network. Examples of network options described above may include wired networks and wireless networks. In one example, the transmission device 912 includes a Network Interface Controller (NIC) that can be connected to other network devices and routers via a network cable to communicate with the Internet or a local area network. In one example, transmission device 912 is a radio frequency (RF) module that is configured to communicate with the Internet wirelessly.
其中,可选地,存储器904被设置为存储预设动作条件和预设权限用户的信息、以及应用程序。Optionally, the memory 904 is configured to store preset action conditions and information of the preset rights user, and an application.
可选地,本实施例中的可选示例可以参考上述实施例中所描述的示例,本实施例在此不再赘述。For an alternative example in this embodiment, reference may be made to the example described in the foregoing embodiment, and details are not described herein again.
本领域普通技术人员可以理解,图9所示的结构仅为示意,电子装置也可以是智能手机(如Android手机、iOS手机等)、平板电脑、掌声电脑以及移动互联网设备(Mobile Internet Devices,MID)、PAD等终端设备。图9其并不对上述电子装置的结构造成限定。例如,电子装置还可包括比图9中所示更多或者更少的组件(如网络接口、显示装置等),或者具有与图9所示不同的配置。A person skilled in the art can understand that the structure shown in FIG. 9 is only schematic, and the electronic device can also be a smart phone (such as an Android mobile phone, an iOS mobile phone, etc.), a tablet computer, an applause computer, and a mobile Internet device (Mobile Internet Devices, MID). ), PAD and other terminal devices. FIG. 9 does not limit the structure of the above electronic device. For example, the electronic device may also include more or fewer components (such as a network interface, display device, etc.) than shown in FIG. 9, or have a different configuration than that shown in FIG.
本领域普通技术人员可以理解上述实施例的各种方法中的全部或部分步骤是可以通过程序来指令终端设备相关的硬件来完成,该程序可以存储于一计算机可读存储介质中,存储介质可以包括:闪存盘、只读存储器(Read-Only Memory,ROM)、随机存取器(Random Access Memory,RAM)、磁盘或光盘等。A person of ordinary skill in the art may understand that all or part of the steps of the foregoing embodiments may be completed by a program to instruct terminal device related hardware, and the program may be stored in a computer readable storage medium, and the storage medium may be Including: flash disk, read-only memory (ROM), random access memory (RAM), disk or optical disk.
本申请的实施例还提供了一种存储介质。可选地,在本实施例中,上 述存储介质可以位于网络中的多个网络设备中的至少一个网络设备。Embodiments of the present application also provide a storage medium. Optionally, in this embodiment, the foregoing storage medium may be located in at least one of the plurality of network devices in the network.
可选地,在本实施例中,存储介质被设置为存储用于执行以下步骤的程序代码:Optionally, in the present embodiment, the storage medium is arranged to store program code for performing the following steps:
S1,从待播放的视频流数据中的全景视频图像帧中获取与第一视角相匹配的第一图像画面,并从全景视频图像帧中获取与第二视角相匹配的第二图像画面,其中,第一视角为当前播放视角,第二视角为与第一视角相邻的视角;S1. Acquire a first image frame that matches the first view from the panoramic video image frame in the video stream data to be played, and obtain a second image frame that matches the second view from the panoramic video image frame, where The first viewing angle is a current viewing angle, and the second viewing angle is a viewing angle adjacent to the first viewing angle;
S2,确定第一图像画面中所包含的第一感兴趣区域,并确定第二图像画面中所包含的第二感兴趣区域;S2, determining a first region of interest included in the first image frame, and determining a second region of interest included in the second image frame;
S3,根据第一感兴趣区域和第二感兴趣区域确定第一图像画面中所包含的视角切换缓冲区域,其中,视角切换缓冲区域为第二感兴趣区域对应的画面在第一图像画面上对应的区域中与第一感兴趣区域相邻的部分区域,第一感兴趣区域和视角切换缓冲区域组成第一图像画面中的第一区域,第一图像画面中除第一区域之外的区域为第二区域;S3. Determine, according to the first region of interest and the second region of interest, a view switching buffer region included in the first image frame, where the view switching buffer region corresponds to the image corresponding to the second region of interest on the first image frame. a partial region adjacent to the first region of interest in the region, the first region of interest and the view switching buffer region constitute a first region in the first image frame, and the region other than the first region in the first image frame is Second area;
S4,以第一区域对应的分辨率播放第一区域对应的图像,以第二区域对应的分辨率播放第二区域对应的图像,其中,第一区域对应的分辨率高于第二区域对应的分辨率。S4: The image corresponding to the first area is played at a resolution corresponding to the first area, and the image corresponding to the second area is played at a resolution corresponding to the second area, where the resolution corresponding to the first area is higher than that of the second area. Resolution.
可选地,在本实施例中,上述存储介质可以包括但不限于:U盘、只读存储器(ROM,Read-Only Memory)、随机存取存储器(RAM,Random Access Memory)、移动硬盘、磁碟或者光盘等各种可以存储程序代码的介质。Optionally, in this embodiment, the foregoing storage medium may include, but not limited to, a USB flash drive, a Read-Only Memory (ROM), a Random Access Memory (RAM), a mobile hard disk, and a magnetic memory. A variety of media that can store program code, such as a disc or a disc.
可选地,本实施例中的可选示例可以参考上述实施例中所描述的示例,本实施例在此不再赘述。For an alternative example in this embodiment, reference may be made to the example described in the foregoing embodiment, and details are not described herein again.
上述本申请实施例序号仅仅为了描述,不代表实施例的优劣。The serial numbers of the embodiments of the present application are merely for the description, and do not represent the advantages and disadvantages of the embodiments.
上述实施例中的集成的单元如果以软件功能单元的形式实现并作为 独立的产品销售或使用时,可以存储在上述计算机可读取的存储介质中。基于这样的理解,本申请的技术方案本质上或者说对相关技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来,该计算机软件产品存储在存储介质中,包括若干指令用以使得一台或多台计算机设备(可为个人计算机、服务器或者网络设备等)执行本申请各个实施例所述方法的全部或部分步骤。The integrated unit in the above embodiment, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in the above-described computer readable storage medium. Based on such understanding, the technical solution of the present application may be embodied in the form of a software product, or the whole or part of the technical solution, which is stored in the storage medium, including The instructions are used to cause one or more computer devices (which may be a personal computer, server or network device, etc.) to perform all or part of the steps of the methods described in the various embodiments of the present application.
在本申请的上述实施例中,对各个实施例的描述都各有侧重,某个实施例中没有详述的部分,可以参见其他实施例的相关描述。In the above-mentioned embodiments of the present application, the descriptions of the various embodiments are different, and the parts that are not detailed in a certain embodiment can be referred to the related descriptions of other embodiments.
在本申请所提供的几个实施例中,应该理解到,所揭露的客户端,可通过其它的方式实现。其中,以上所描述的装置实施例仅仅是示意性的,例如所述单元的划分,仅仅为一种逻辑功能划分,实际实现时可以有另外的划分方式,例如多个单元或组件可以结合或者可以集成到另一个系统,或一些特征可以忽略,或不执行。另一点,所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口,单元或模块的间接耦合或通信连接,可以是电性或其它的形式。In the several embodiments provided by the present application, it should be understood that the disclosed client may be implemented in other manners. The device embodiments described above are merely illustrative. For example, the division of the unit is only a logical function division. In actual implementation, there may be another division manner. For example, multiple units or components may be combined or may be Integrate into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, unit or module, and may be electrical or otherwise.
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的,作为单元显示的部件可以是或者也可以不是物理单元,即可以位于一个地方,或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.
另外,在本申请各个实施例中的各功能单元可以集成在一个处理单元中,也可以是各个单元单独物理存在,也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现,也可以采用软件功能单元的形式实现。In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.
以上所述仅是本申请的可选实施方式,应当指出,对于本技术领域的普通技术人员来说,在不脱离本申请原理的前提下,还可以做出若干改进和润饰,这些改进和润饰也应视为本申请的保护范围。The above description is only an alternative embodiment of the present application, and it should be noted that those skilled in the art can make several improvements and retouchings without departing from the principles of the present application. It should also be considered as the scope of protection of this application.
工业实用性:通过上述描述可知,本申请全景视频图像帧中第一视角匹配的第一图像画面包括第一区域和第二区域,其中的第一区域包括第一感兴趣区域和视角切换缓冲区域,该视角切换缓冲区域是第二感兴趣区域对应的画面在第一图像画面上对应的区域中与第一感兴趣区域相邻的部分区域,分别以不同的分辨率播放第一区域和第二区域上的图像,其中第一区域对应的分辨率高于第二区域对应的分辨率,这样在全景视频图像的播放过程中,视频画面从第一视角切换到第二视角的过程中,用户的视线将扫过第一区域中视角切换缓冲区域上分辨率较高的画面,从而在视角切换时,可避免画面的跳动,使得相邻视角的图像的衔接更为平滑,进而克服相关技术中视频流切换时视频画面会产生跳动的问题。Industrial Applicability: It can be seen from the above description that the first image frame matched by the first view in the panoramic video image frame of the present application includes a first region and a second region, wherein the first region includes the first region of interest and the view switching buffer region The view switching buffer area is a partial area of the picture corresponding to the second region of interest corresponding to the first region of interest in the corresponding region on the first image frame, and the first region and the second region are respectively played at different resolutions. An image on the area, wherein the resolution corresponding to the first area is higher than the resolution corresponding to the second area, so that during the playback of the panoramic video image, the video picture is switched from the first perspective to the second perspective, and the user The line of sight will sweep through the higher resolution picture on the view switching buffer area in the first area, so that when the view angle is switched, the bounce of the picture can be avoided, so that the image of the adjacent view is more smoothly connected, thereby overcoming the video in the related art. Video screens can cause jitter when streaming.
Claims (12)
- 一种全景视频图像的播放方法,包括:A method for playing a panoramic video image, comprising:播放设备从待播放的视频流数据中的全景视频图像帧中获取与第一视角相匹配的第一图像画面,并从所述全景视频图像帧中获取与第二视角相匹配的第二图像画面,其中,所述第一视角为当前播放视角,所述第二视角为与所述第一视角相邻的视角;The playback device acquires a first image frame that matches the first view from the panoramic video image frame in the video stream data to be played, and acquires a second image frame that matches the second view from the panoramic video image frame. The first view angle is a current play view angle, and the second view angle is a view angle adjacent to the first view angle;所述播放设备确定所述第一图像画面中所包含的第一感兴趣区域,并确定所述第二图像画面中所包含的第二感兴趣区域;The playing device determines a first region of interest included in the first image frame, and determines a second region of interest included in the second image frame;所述播放设备根据所述第一感兴趣区域和所述第二感兴趣区域确定所述第一图像画面中所包含的视角切换缓冲区域,其中,所述视角切换缓冲区域为所述第二感兴趣区域对应的画面在所述第一图像画面上对应的区域中与所述第一感兴趣区域相邻的部分区域,所述第一感兴趣区域和视角切换缓冲区域组成所述第一图像画面中的第一区域,所述第一图像画面中除所述第一区域之外的区域为第二区域;Determining, by the playback device, the view switching buffer region included in the first image frame according to the first region of interest and the second region of interest, wherein the view switching buffer region is the second sense a picture corresponding to the region of interest is a partial region adjacent to the first region of interest in a corresponding region on the first image frame, the first region of interest and a view switching buffer region constituting the first image frame a first area of the first image, wherein the area other than the first area is a second area;所述播放设备以所述第一区域对应的分辨率播放所述第一区域上的图像,以所述第二区域对应的分辨率播放所述第二区域上的图像,其中,所述第一区域对应的分辨率高于所述第二区域对应的分辨率。The playing device plays an image on the first area at a resolution corresponding to the first area, and plays an image on the second area in a resolution corresponding to the second area, where the first The resolution corresponding to the region is higher than the resolution corresponding to the second region.
- 根据权利要求1所述的方法,其中,所述播放设备根据所述第一感兴趣区域和所述第二感兴趣区域确定所述第一图像画面中所包含的视角切换缓冲区域包括:The method according to claim 1, wherein the determining, by the playback device, the view switching buffer area included in the first image frame according to the first region of interest and the second region of interest comprises:所述播放设备确定所述第二感兴趣区域对应的画面在所述第一图像画面上对应的区域;The playing device determines an area corresponding to the picture corresponding to the second region of interest on the first image frame;所述播放设备确定所述对应的区域中与所述第一感兴趣区域相邻的像素宽度为目标像素宽度的区域为所述视角切换缓冲区域。The playing device determines that the area of the corresponding area that has a pixel width adjacent to the first area of interest as the target pixel width is the view switching buffer area.
- 根据权利要求2所述的方法,其中,在所述播放设备确定所述对应的区域中与所述第一感兴趣区域相邻的像素宽度为目标像素宽度的区 域为所述视角切换缓冲区域之前,所述方法还包括:The method according to claim 2, wherein the playback device determines that a region of the corresponding region adjacent to the first region of interest has a pixel width of a target pixel width before the view switching buffer region The method further includes:所述播放设备获取跳转角度对应的像素宽度,其中,所述跳转角度为从播放所述第一视角对应的视频数据转换至播放所述第二视角对应的视频数据时转动的角度;The playback device acquires a pixel width corresponding to the jump angle, where the jump angle is an angle that is rotated when the video data corresponding to the first view is played to be converted to the video data corresponding to the second view;所述播放设备根据所述跳转角度对应的像素宽度确定所述目标像素宽度。The playback device determines the target pixel width according to a pixel width corresponding to the jump angle.
- 根据权利要求3所述的方法,其中,所述播放设备根据所述跳转角度对应的像素宽度确定所述目标像素宽度包括:The method according to claim 3, wherein the determining, by the playback device, the target pixel width according to a pixel width corresponding to the jump angle comprises:所述播放设备将所述跳转角度对应的像素宽度确定为所述目标像素宽度;或者,The playing device determines a pixel width corresponding to the jump angle as the target pixel width; or所述播放设备从大于所述跳转角度对应的像素宽度且小于所述第二感兴趣区域对应的像素宽度的像素宽度中选取一个像素宽度,并将所述一个像素宽度确定为所述目标像素宽度。The playback device selects one pixel width from a pixel width greater than a pixel width corresponding to the jump angle and smaller than a pixel width corresponding to the second region of interest, and determines the one pixel width as the target pixel width.
- 根据权利要求1至4中任一项所述的方法,其中,所述播放设备以所述第一区域对应的分辨率播放所述第一区域对应的图像包括:The method according to any one of claims 1 to 4, wherein the playing of the image corresponding to the first area by the playback device corresponding to the resolution of the first area comprises:所述播放设备以第一分辨率播放所述第一感兴趣区域上的图像,并以第二分辨率播放所述视角切换缓冲区域上的图像,其中,所述第一分辨率高于或者等于所述第二分辨率,所述第二分辨率高于所述第二区域对应的分辨率。Playing the image on the first region of interest at a first resolution, and playing an image on the view switching buffer region at a second resolution, wherein the first resolution is higher than or equal to The second resolution is higher than a resolution corresponding to the second area.
- 一种全景视频图像的播放装置,应用于播放设备,包括一个或多个处理器,以及一个或多个存储程序单元的存储器,其中,所述程序单元由所述处理器执行,所述程序单元包括:A playback device for a panoramic video image, applied to a playback device, comprising one or more processors, and one or more memories storing program units, wherein the program units are executed by the processor, the program units include:获取模块,被设置为在播放虚拟实境场景下的视频数据时,从待播放的视频流数据中的全景视频图像帧中获取与第一视角相匹配的第一图像画面,并从所述全景视频图像帧中获取与第二视角相匹配的第二图像画面,其中,所述第一视角为当前播放视角,所述第二视角为与所述第一视角相邻的视角;An acquiring module, configured to: when playing video data in a virtual reality scene, acquire a first image image that matches a first perspective from a panoramic video image frame in the video stream data to be played, and from the panorama Obtaining, in the video image frame, a second image frame that matches the second viewing angle, wherein the first viewing angle is a current playback perspective, and the second perspective is a perspective adjacent to the first perspective;第一确定模块,被设置为确定所述第一图像画面中所包含的第一感兴趣区域,并确定所述第二图像画面中所包含的第二感兴趣区域;a first determining module, configured to determine a first region of interest included in the first image frame, and determine a second region of interest included in the second image frame;第二确定模块,被设置为根据所述第一感兴趣区域和所述第二感兴趣区域确定所述第一图像画面中所包含的视角切换缓冲区域,其中,所述视角切换缓冲区域为所述第二感兴趣区域对应的画面在所述第一图像画面上对应的区域中与所述第一感兴趣区域相邻的部分区域,所述第一感兴趣区域和视角切换缓冲区域组成所述第一图像画面中的第一区域,所述第一图像画面中除所述第一区域之外的区域为第二区域;a second determining module, configured to determine, according to the first region of interest and the second region of interest, a view switching buffer region included in the first image frame, wherein the view switching buffer region is a picture corresponding to the second region of interest, a partial region adjacent to the first region of interest in a corresponding region on the first image frame, the first region of interest and a view switching buffer region a first area in the first image frame, wherein the area other than the first area in the first image picture is a second area;播放模块,被设置为以所述第一区域对应的分辨率播放所述第一区域上的图像,以所述第二区域对应的分辨率播放所述第二区域上的图像,其中,所述第一区域对应的分辨率高于所述第二区域对应的分辨率。a playing module, configured to play an image on the first area at a resolution corresponding to the first area, and play an image on the second area in a resolution corresponding to the second area, where The resolution corresponding to the first region is higher than the resolution corresponding to the second region.
- 根据权利要求6所述的装置,其中,所述第二确定模块包括:The apparatus of claim 6, wherein the second determining module comprises:第一确定单元,被设置为确定所述第二感兴趣区域对应的画面在所述第一图像画面上对应的区域;a first determining unit, configured to determine an area corresponding to the picture corresponding to the second region of interest on the first image frame;第二确定单元,被设置为确定所述对应的区域中与所述第一感兴趣区域相邻的像素宽度为目标像素宽度的区域为所述视角切换缓冲区域。The second determining unit is configured to determine that the region of the corresponding region that has a pixel width adjacent to the first region of interest as the target pixel width is the view switching buffer region.
- 根据权利要求7所述的装置,其中,所述装置还包括:The apparatus of claim 7 wherein said apparatus further comprises:获取模块,被设置为获取跳转角度对应的像素宽度,其中,所述跳转角度为从播放所述第一视角对应的视频数据转换至播放所述第二视角对应的视频数据时转动的角度;The obtaining module is configured to obtain a pixel width corresponding to the jump angle, wherein the jump angle is an angle of rotation when the video data corresponding to the first view is played to be converted to the video data corresponding to the second view ;第三确定模块,被设置为根据所述跳转角度对应的像素宽度确定所述目标像素宽度。The third determining module is configured to determine the target pixel width according to a pixel width corresponding to the jump angle.
- 根据权利要求8所述的装置,其中,所述第三确定模块包括:The apparatus of claim 8, wherein the third determining module comprises:第三确定单元,被设置为将所述跳转角度对应的像素宽度确定为 所述目标像素宽度;或者,a third determining unit, configured to determine a pixel width corresponding to the jump angle as the target pixel width; or第四确定单元,被设置为从大于所述跳转角度对应的像素宽度且小于所述第二感兴趣区域对应的像素宽度的像素宽度中选取一个像素宽度,并将所述一个像素宽度确定为所述目标像素宽度。a fourth determining unit, configured to select one pixel width from a pixel width greater than a pixel width corresponding to the jump angle and smaller than a pixel width corresponding to the second region of interest, and determine the one pixel width as The target pixel width.
- 根据权利要求6至9中任一项所述的装置,其中,所述播放模块包括:The apparatus according to any one of claims 6 to 9, wherein the play module comprises:播放单元,被设置为以第一分辨率播放所述第一感兴趣区域上的图像,并以第二分辨率播放所述视角切换缓冲区域上的图像,其中,所述第一分辨率高于或者等于所述第二分辨率,所述第二分辨率高于所述第二区域对应的分辨率。a playing unit configured to play an image on the first region of interest at a first resolution and play an image on the view switching buffer region at a second resolution, wherein the first resolution is higher Or equal to the second resolution, the second resolution being higher than a resolution corresponding to the second region.
- 一种存储介质,所述存储介质包括存储的程序,其中,所述程序运行时执行上述权利要求1至5任一项中所述的方法。A storage medium comprising a stored program, wherein the program is executed to perform the method of any of the preceding claims 1 to 5.
- 一种电子装置,包括存储器、处理器及存储在所述存储器上并可在所述处理器上运行的计算机程序,所述处理器通过所述计算机程序执行上述权利要求1至5任一项中所述的方法。An electronic device comprising a memory, a processor, and a computer program stored on the memory and operable on the processor, the processor executing the above claims 1 to 5 by the computer program Said method.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201710997898.8A CN109698952B (en) | 2017-10-23 | 2017-10-23 | Panoramic video image playing method and device, storage medium and electronic device |
CN201710997898.8 | 2017-10-23 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2019080792A1 true WO2019080792A1 (en) | 2019-05-02 |
Family
ID=66226190
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2018/111185 WO2019080792A1 (en) | 2017-10-23 | 2018-10-22 | Panoramic video image playing method and device, storage medium and electronic device |
Country Status (2)
Country | Link |
---|---|
CN (1) | CN109698952B (en) |
WO (1) | WO2019080792A1 (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110944239A (en) * | 2019-11-28 | 2020-03-31 | 重庆爱奇艺智能科技有限公司 | Video playing method and device |
Families Citing this family (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111182218A (en) * | 2020-01-07 | 2020-05-19 | 影石创新科技股份有限公司 | Panoramic video processing method, device, equipment and storage medium |
CN111246237A (en) * | 2020-01-22 | 2020-06-05 | 视联动力信息技术股份有限公司 | Panoramic video live broadcast method and device |
CN111314739B (en) * | 2020-02-17 | 2022-05-17 | 聚好看科技股份有限公司 | Image processing method, server and display device |
CN111447462B (en) * | 2020-05-20 | 2022-07-05 | 上海科技大学 | Video live broadcast method, system, storage medium and terminal based on visual angle switching |
CN113052166A (en) * | 2021-02-05 | 2021-06-29 | 杭州依图医疗技术有限公司 | Pathological image display method and device |
CN114143602B (en) * | 2021-12-06 | 2023-07-07 | 深圳创维数字技术有限公司 | Panoramic picture playing method and device, panoramic playing server and storage medium |
CN114630097A (en) * | 2022-03-15 | 2022-06-14 | 中国电信股份有限公司 | Image processing method, device, system and computer readable storage medium |
Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20070058034A1 (en) * | 2005-09-12 | 2007-03-15 | Shunichi Numazaki | Stereoscopic image display device, stereoscopic display program, and stereoscopic display method |
US20130141523A1 (en) * | 2011-12-02 | 2013-06-06 | Stealth HD Corp. | Apparatus and Method for Panoramic Video Hosting |
CN105898139A (en) * | 2015-12-23 | 2016-08-24 | 乐视致新电子科技(天津)有限公司 | Panoramic video production method and device and panoramic video play method and device |
CN106534827A (en) * | 2016-12-19 | 2017-03-22 | 暴风集团股份有限公司 | Method and system for playing panoramic video based on user perspective |
CN107087145A (en) * | 2017-06-02 | 2017-08-22 | 深圳市本道科技有限公司 | Multi-channel video carries out the method and device that 360 degree of panoramic videos are shown |
CN107123080A (en) * | 2017-03-29 | 2017-09-01 | 北京疯景科技有限公司 | Show the method and device of panorama content |
CN107277474A (en) * | 2017-06-26 | 2017-10-20 | 深圳看到科技有限公司 | Panorama generation method and generating means |
Family Cites Families (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN104159026A (en) * | 2014-08-07 | 2014-11-19 | 厦门亿联网络技术股份有限公司 | System for realizing 360-degree panoramic video |
KR101844032B1 (en) * | 2014-12-26 | 2018-05-14 | 주식회사 케이티 | Method for sending video in region of interest from panoramic-video, server and device |
CN104735464A (en) * | 2015-03-31 | 2015-06-24 | 华为技术有限公司 | Panorama video interactive transmission method, server and client end |
US9996895B2 (en) * | 2015-05-01 | 2018-06-12 | Ricoh Company, Ltd. | Image display system, information processing apparatus, and image display method |
CN106454321A (en) * | 2016-10-26 | 2017-02-22 | 深圳市道通智能航空技术有限公司 | Panoramic video processing method, device and system |
-
2017
- 2017-10-23 CN CN201710997898.8A patent/CN109698952B/en active Active
-
2018
- 2018-10-22 WO PCT/CN2018/111185 patent/WO2019080792A1/en active Application Filing
Patent Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20070058034A1 (en) * | 2005-09-12 | 2007-03-15 | Shunichi Numazaki | Stereoscopic image display device, stereoscopic display program, and stereoscopic display method |
US20130141523A1 (en) * | 2011-12-02 | 2013-06-06 | Stealth HD Corp. | Apparatus and Method for Panoramic Video Hosting |
CN105898139A (en) * | 2015-12-23 | 2016-08-24 | 乐视致新电子科技(天津)有限公司 | Panoramic video production method and device and panoramic video play method and device |
CN106534827A (en) * | 2016-12-19 | 2017-03-22 | 暴风集团股份有限公司 | Method and system for playing panoramic video based on user perspective |
CN107123080A (en) * | 2017-03-29 | 2017-09-01 | 北京疯景科技有限公司 | Show the method and device of panorama content |
CN107087145A (en) * | 2017-06-02 | 2017-08-22 | 深圳市本道科技有限公司 | Multi-channel video carries out the method and device that 360 degree of panoramic videos are shown |
CN107277474A (en) * | 2017-06-26 | 2017-10-20 | 深圳看到科技有限公司 | Panorama generation method and generating means |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110944239A (en) * | 2019-11-28 | 2020-03-31 | 重庆爱奇艺智能科技有限公司 | Video playing method and device |
Also Published As
Publication number | Publication date |
---|---|
CN109698952B (en) | 2020-09-29 |
CN109698952A (en) | 2019-04-30 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2019080792A1 (en) | Panoramic video image playing method and device, storage medium and electronic device | |
US11303881B2 (en) | Method and client for playing back panoramic video | |
US11037365B2 (en) | Method, apparatus, medium, terminal, and device for processing multi-angle free-perspective data | |
CN108648257B (en) | Panoramic picture acquisition method and device, storage medium and electronic device | |
JP6410918B2 (en) | System and method for use in playback of panoramic video content | |
KR102013403B1 (en) | Spherical video streaming | |
BR102017010904A2 (en) | METHOD, APPARATUS AND FLOW FOR IMMERSIVE VIDEO FORMAT | |
US20160277772A1 (en) | Reduced bit rate immersive video | |
KR20190031504A (en) | Method and system for interactive transmission of panoramic video | |
KR20170132098A (en) | Method, apparatus and stream of formatting an immersive video for legacy and immersive rendering devices | |
US11138460B2 (en) | Image processing method and apparatus | |
JP2019534500A (en) | Method, apparatus and system for an immersive video format | |
KR102308604B1 (en) | Method, apparatus and stream for formatting immersive video for legacy and immersive rendering devices | |
CN107197135B (en) | Video generation method and video generation device | |
US10638029B2 (en) | Shared experiences in panoramic video | |
US11622099B2 (en) | Information-processing apparatus, method of processing information, and program | |
JP2024050737A (en) | Information processing device, method for operating information processing device, and program | |
US11295453B2 (en) | Image processing apparatus, transmission method, and storage medium | |
KR101883981B1 (en) | Apparatus and method for providing virtual reality video | |
KR101773929B1 (en) | System for processing video with wide viewing angle, methods for transmitting and displaying vide with wide viewing angle and computer programs for the same | |
JP5919544B1 (en) | Image reproduction method, image reproduction apparatus, and image reproduction program |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 18869858 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 18869858 Country of ref document: EP Kind code of ref document: A1 |