WO2019080792A1

WO2019080792A1 - Panoramic video image playing method and device, storage medium and electronic device

Info

Publication number: WO2019080792A1
Application number: PCT/CN2018/111185
Authority: WO
Inventors: 曾新海; 涂远东; 管坤
Original assignee: 腾讯科技（深圳）有限公司
Priority date: 2017-10-23
Filing date: 2018-10-22
Publication date: 2019-05-02
Also published as: CN109698952B; CN109698952A

Abstract

Disclosed are a panoramic video image playing method and device, a storage medium and an electronic device. The method comprises: acquiring, from a panoramic video image frame in the to-be-played video streaming data, a first screen image matched with a first view angle, and acquiring, from the panoramic video image frame, a second screen image matched with a second view angle; determining a first area-of-interest included in the first screen image and a second area-of-interest included in the second screen image; determining, according to the first area-of-interest and the second area-of-interest, a view angle switching buffer region included in the first screen image; and playing images in the first area at a resolution ratio corresponding to the first area, and playing images in the second area at a resolution ratio corresponding to the second area. The present application solves the technical problem that video images will jitter when video streaming is switched in the related art.

Description

Method, device, storage medium and electronic device for playing panoramic video image

This application claims the priority of the Chinese patent application filed on October 23, 2017, the Chinese Patent Office, the application number is 2017109978988, the invention name "playing method, device, storage medium and electronic device for panoramic video images", the entire contents of which are The citations are incorporated herein by reference.

Technical field

The present application relates to the field of computers, and in particular, to a method, an apparatus, a storage medium, and an electronic device for playing a panoramic video image.

Background technique

Currently, the method of playing panoramic video is to convert the video data stream into a FOV video stream. The field of view (FOV) image is an image between the edge of the display device and the angle of the line connecting the observation point (eye). It is said that the user can clearly see the content of the picture and the residual light, representing the angle image that the user sees. At present, the video data stream is converted into a FOV video stream. The bottom of the model is converted to the maximum extent of the picture quality. The four sides of the conversion model are generated by projection, and there is a certain image distortion. The scheme moves in the head (ie, the observation point). When the FOV video is switched, there will be a certain time delay due to the current video stream i switching to the next video stream j. Before the video j picture is presented, the picture is still presented. Is the image on the side of the video i, due to the difference in the angle of view of the video i side image and the video of the bottom positive angle image on the projection, which causes the user to feel the jitter of the video when viewing the video, and also because of the pixel density and pyramid at the bottom of the conversion model. The difference in pixel spatial density on the side is too large, and the sharpness also varies greatly.

In response to the above problems, no effective solution has been proposed yet.

Summary of the invention

The embodiment of the present invention provides a method, a device, a storage medium, and an electronic device for playing a panoramic video image, so as to at least solve the technical problem that the video image may be beat when the video stream is switched in the related art.

According to an aspect of the embodiments of the present application, a method for playing a panoramic video image is provided, including: acquiring a first image image that matches a first viewing angle from a panoramic video image frame in video stream data to be played, and Obtaining, from the panoramic video image frame, a second image frame that matches a second viewing angle, where the first viewing angle is a current playback perspective, and the second perspective is a perspective adjacent to the first perspective; Determining a first region of interest included in the first image frame and determining a second region of interest included in the second image frame; according to the first region of interest and the second region of interest And determining, by the region, the view switching buffer region included in the first image frame, wherein the view switching buffer region is a region corresponding to the second region of interest in a corresponding region on the first image frame a partial region adjacent to the first region of interest, the first region of interest and a view switching buffer region constituting a first region in the first image frame, the first image An area other than the first area in the picture is a second area; playing an image on the first area at a resolution corresponding to the first area, and playing the resolution in a resolution corresponding to the second area An image on the second area, wherein the first area corresponds to a higher resolution than the second area.

According to another aspect of the embodiments of the present application, a playback apparatus for a panoramic video image is further provided, including: an acquisition module, configured to: when playing video data in a virtual reality scene, from video stream data to be played Obtaining a first image frame matching the first viewing angle in the panoramic video image frame, and acquiring a second image frame matching the second viewing angle from the panoramic video image frame, wherein the first viewing angle is current Playing a viewing angle, the second viewing angle is a viewing angle adjacent to the first viewing angle; the first determining module is configured to determine a first region of interest included in the first image frame, and determine the first a second region of interest included in the two image frames; a second determining module configured to determine a viewing angle switch included in the first image frame according to the first region of interest and the second region of interest a buffer area, wherein the view switching buffer area is a picture corresponding to the second area of interest in a corresponding area on the first image picture and the first area of interest An adjacent partial region, the first region of interest and a view switching buffer region constitute a first region in the first image frame, and an area of the first image frame other than the first region is a second area; a playing module, configured to play an image on the first area at a resolution corresponding to the first area, and play an image on the second area in a resolution corresponding to the second area, where The resolution corresponding to the first area is higher than the resolution corresponding to the second area.

According to another aspect of an embodiment of the present application, there is also provided a storage medium comprising a stored program, wherein the program is executed to perform the method described in any of the above.

According to another aspect of an embodiment of the present application, there is also provided an electronic device comprising a memory, a processor, and a computer program stored on the memory and operable on the processor, the processor The computer program performs the method described in any of the above.

In the embodiment of the present application, a first image image matching the first viewing angle is acquired from the panoramic video image frame in the video stream data to be played, and the second viewing angle is matched from the panoramic video image frame. a second image frame, wherein the first view angle is a current play view angle, the second view angle is a view angle adjacent to the first view angle; determining a first region of interest included in the first image frame, and determining a second image frame a second region of interest included; determining a view switching buffer region included in the first image frame according to the first region of interest and the second region of interest, wherein the view switching buffer region is a picture corresponding to the second region of interest a partial region adjacent to the first region of interest in the corresponding region on the first image frame, the first region of interest and the view switching buffer region constitute a first region in the first image frame, the first image image is divided by the first image The area outside the area is the second area; the image on the first area is played at the resolution corresponding to the first area, and the second is played in the resolution corresponding to the second area. The image on the domain, wherein the first region is higher than the second resolution corresponding to a resolution corresponding to the region. That is, the first image frame matched by the first angle of view in the panoramic video image frame includes a first area and a second area, wherein the first area includes a first area of interest and a view switching buffer area, and the view switching buffer area is The picture corresponding to the second region of interest is a partial region adjacent to the first region of interest in the corresponding region on the first image frame, and the images on the first region and the second region are respectively played at different resolutions, wherein The resolution corresponding to an area is higher than the resolution corresponding to the second area, so that during the playback of the panoramic video image, during the process of switching the video picture from the first view to the second view, the user's line of sight will sweep through the first area. The middle view switchover screen has a higher resolution on the buffer area, so that when the view angle is switched, the bounce of the picture can be avoided, so that the image of the adjacent view is more smoothly connected, thereby overcoming the video picture generated when the video stream is switched in the related art. The problem of beating.

DRAWINGS

The drawings described herein are intended to provide a further understanding of the present application, and are intended to be a part of this application. In the drawing:

1 is a schematic diagram of an application environment of an optional method for playing a panoramic video image according to an embodiment of the present application;

2 is a flowchart of an optional method for playing a panoramic video image according to an embodiment of the present application;

FIG. 3 is a schematic diagram of an optional method for playing a panoramic video image according to an embodiment of the present application; FIG.

4 is a schematic diagram of another optional method for playing a panoramic video image according to an embodiment of the present application;

FIG. 5 is a schematic diagram of still another optional method for playing a panoramic video image according to an embodiment of the present application; FIG.

6 is a schematic diagram of an optional playback device for panoramic video images according to an embodiment of the present application;

FIG. 7 is a schematic diagram of an application scenario of an optional method for playing a panoramic video image according to an embodiment of the present application;

FIG. 8 is a schematic diagram of an application scenario of another optional method for playing a panoramic video image according to an embodiment of the present application;

9 is a schematic diagram of an alternative electronic device in accordance with an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present application. It is an embodiment of the present application, and not all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present application without departing from the inventive scope shall fall within the scope of the application.

It should be noted that the terms "first", "second" and the like in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or order. It is to be understood that the data so used may be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in a sequence other than those illustrated or described herein. In addition, the terms "comprises" and "comprises" and "the" and "the" are intended to cover a non-exclusive inclusion, for example, a process, method, system, product, or device that comprises a series of steps or units is not necessarily limited to Those steps or units may include other steps or units not explicitly listed or inherent to such processes, methods, products or devices.

In the embodiment of the present application, an embodiment of a method for playing a panoramic video image is provided. As an optional implementation manner, the method for playing the panoramic video image may be, but is not limited to, being applied to an application environment as shown in FIG. 1. The terminal 102 records the panoramic video content in the video stream data to be broadcasted in the field. The network 104 transmits the panoramic video image frame in the video stream data to the playback device 106 (such as Virtual Reality (VR) glasses), and the playback device 106 is in the panoramic video image frame in the video stream data to be played. Obtaining a first image frame that matches the first viewing angle, and acquiring a second image image that matches the second viewing angle from the panoramic video image frame, where the first viewing angle is the current playback viewing angle, and the second viewing angle is the first a view angle adjacent to the view; determining a first region of interest included in the first image frame, and determining a second region of interest included in the second image frame; determining according to the first region of interest and the second region of interest a view switching buffer region included in the first image frame, wherein the view switching buffer region is a picture corresponding to the second region of interest in the first image a partial region adjacent to the first region of interest in the corresponding region on the surface, the first region of interest and the view switching buffer region constitute a first region in the first image frame, except for the first region in the first image frame The area is the second area; the image corresponding to the first area is played at a resolution corresponding to the first area, and the image corresponding to the second area is played at a resolution corresponding to the second area, wherein the resolution corresponding to the first area is higher than The resolution corresponding to the second region.

In this embodiment, the first image frame matched by the first angle of view in the panoramic video image frame includes a first area and a second area, wherein the first area includes a first area of interest and a view switching buffer area, and the view switching buffer The area is a partial area of the picture corresponding to the second area of interest corresponding to the first area of interest in the corresponding area on the first image picture, and the images on the first area and the second area are respectively played at different resolutions, The resolution corresponding to the first area is higher than the resolution corresponding to the second area, so that during the playback of the panoramic video image, the user's line of sight will be swept during the process of switching the video picture from the first perspective to the second perspective. In a region, the view angle switches the picture with higher resolution on the buffer area, so that when the view angle is switched, the bounce of the picture can be avoided, so that the image of the adjacent view is more smoothly connected, thereby overcoming the video picture when the video stream is switched in the related art. There will be a problem with beating.

Optionally, in this embodiment, the foregoing playback device may include, but is not limited to, at least one of the following: a mobile phone, a tablet computer, a notebook computer, a desktop PC, a digital television, a VR glasses, and the like for displaying live video stream data. Panoramic video image of hardware device. The above network may include, but is not limited to, at least one of the following: a wide area network, a metropolitan area network, and a local area network. The above is only an example, and the embodiment does not limit this.

According to an embodiment of the present application, a method for playing a panoramic video image is provided. As shown in FIG. 2, the method includes:

S202. The playback device acquires a first image frame that matches the first view from the panoramic video image frame in the video stream data to be played, and acquires a second image frame that matches the second view from the panoramic video image frame. Wherein the first perspective is a current playback perspective, and the second perspective is a perspective adjacent to the first perspective;

S204. The playback device determines a first region of interest included in the first image frame, and determines a second region of interest included in the second image frame.

S206. The playback device determines, according to the first region of interest and the second region of interest, a view switching buffer region included in the first image frame, where the view switch buffer region is a picture corresponding to the second region of interest in the first image frame. a partial region adjacent to the first region of interest in the corresponding region, the first region of interest and the view switching buffer region constitute a first region in the first image frame, except for the first region in the first image frame The area is the second area;

S208: The playback device plays the image corresponding to the first region at a resolution corresponding to the first region, and plays the image corresponding to the second region at a resolution corresponding to the second region, where the resolution corresponding to the first region is higher than the second region. Corresponding resolution.

Optionally, in this embodiment, the foregoing image encoding method may be, but is not limited to, being applied to a scenario in which a panoramic video image is played, such as a virtual reality (VR) video playback process, and a panoramic video playback process. The above play may include but is not limited to: video on demand, live video. The above video may be, but is not limited to, applied to a terminal application such as a video playing application, a game application, a shopping application, and the like. The above is only an example, and is not limited in this embodiment.

Optionally, in this embodiment, the panoramic video image frame may be, but is not limited to, a 360-degree spherical panoramic image, wherein the 360-degree spherical panoramic image may be, but is not limited to, divided into multiple views (FOV). The image image corresponding to each view will generate a video stream data for playback by the playback device.

Optionally, in this embodiment, the projection model of the video image provided by the method for playing the panoramic video image is similar to a diamond in shape, and may be referred to as a diamond model.

For example, as shown in FIG. 3, the playback device is illustrated by taking VR glasses as an example. The left side of FIG. 3 shows that a 360-degree spherical panoramic image in a video can be viewed using VR glasses, and the right side of FIG. 3 shows a 360-degree spherical panoramic image. In the middle, the built-in diamond model (each vertex is located on the spherical surface) is mapped and projected. Further, each view will get a corresponding image (including the projection of the diamond model on each face), as shown in FIG. 4, and the left side of FIG. 4 shows the perspective image of the diamond model after being projected on each face. Image processing such as stretching will result in an image picture matching the angle of view shown on the right side of Fig. 4, wherein the shaded portion indicates the corresponding picture at the bottom of the model. It can be seen that the panoramic video image is projected using the above diamond projection model, and the picture at the bottom of the model is larger than the pyramid model, so the model has a larger range of high definition images than the pyramid model. The bottom area of the diamond projection model also uses more pixels than the pyramid model, and the image quality is better.

Optionally, in the embodiment, the first region of interest is a Region of Interest (ROI) in the first image frame. For example, as shown in the right side of FIG. 3, the first region may be a bottom region of the diamond model corresponding to the current playback angle, the bottom region including the first region of interest and the view switching buffer region around the first region of interest, and the other regions of the diamond model (eg, the side region) correspond to the second region region.

Optionally, in this embodiment, the region of interest in each of the above views may be, but is not limited to, a rectangular region, a diamond region, a regular hexagon region, or other polygonal regions of any shape. In this embodiment, the case where the region of interest is a rectangular region is taken as an example, and other situations are similar thereto, and details are not described herein again.

Optionally, in this embodiment, the second viewing angle may be, but is not limited to, a plurality of viewing angles adjacent to the first viewing angle, and the second viewing angle may be, but not limited to, the first viewing angle. Four perspectives of up, down, left, and right adjacent to each other. For example, as shown in FIG. 5, there are four second viewing angles adjacent to the first viewing angle, which are a second viewing angle A, a second viewing angle B, a second viewing angle C, and a second viewing angle D, respectively. The corresponding region of the second region of interest on the first image frame (the region of the shaded portion in the figure) is region A, region B, region C, and region D, respectively, in each of the above regions. The partial area is used as the view switching buffer area, then the first area in the first image picture includes the view switching buffer area A, the view switching buffer area B, the view switching buffer area C, the view switching buffer area D, and the first region of interest. . The other area of the first image picture other than the above area is the second area.

It can be seen that, by using the above steps, the first image frame matched by the first view in the panoramic video image frame includes a first region and a second region, wherein the first region includes the first region of interest and the view switching buffer region, and the view switching buffer The area is a partial area of the picture corresponding to the second area of interest corresponding to the first area of interest in the corresponding area on the first image picture, and the images on the first area and the second area are respectively played at different resolutions, The resolution corresponding to the first area is higher than the resolution corresponding to the second area, so that during the playback of the panoramic video image, the user's line of sight will be swept during the process of switching the video picture from the first perspective to the second perspective. In a region, the view angle switches the picture with higher resolution on the buffer area, so that when the view angle is switched, the bounce of the picture can be avoided, so that the image of the adjacent view is more smoothly connected, thereby overcoming the video picture when the video stream is switched in the related art. There will be a problem with beating.

As an optional solution, determining, according to the first region of interest and the second region of interest, the view switching buffer region included in the first image frame includes:

S1. The playback device determines an area corresponding to the picture corresponding to the second region of interest on the first image frame.

S2. The playback device determines that a region of the corresponding region that is adjacent to the first region of interest has a pixel width of the target pixel width as a view switching buffer region.

Optionally, in this embodiment, the regions of interest of two adjacent viewing angles may be adjacent, and thus may, but are not limited to, determine a region in which the pixel width adjacent to the first region of interest is the target pixel width. Switch the buffer area for the angle of view.

Optionally, in this embodiment, the target pixel width may be, but is not limited to, being set according to an actual application scenario of the playback method of the panoramic video image.

As an optional solution, before determining that the area of the target pixel width adjacent to the first region of interest is the view switching buffer area, the method further includes:

S1. The playback device obtains a pixel width corresponding to a jump angle, where the jump angle is an angle that is rotated when the video data corresponding to the first view angle is converted to the video data corresponding to the second view angle.

S2. The playback device determines the target pixel width according to the pixel width corresponding to the jump angle.

Optionally, in this embodiment, the video sequence currently viewed by the user is a video sequence corresponding to the first perspective. When the FOV video is switched, the second perspective can be started when the head posture crosses the bottom angle position. The video sequence, so that the target pixel width can be configured according to the angle of rotation when the angle of view is rotated, and the angle of rotation when the video data corresponding to the first view angle is switched to the video data corresponding to the second view angle can be obtained, and corresponding to the angle according to the angle The pixel width determines the target pixel width.

Optionally, in this embodiment, the target pixel width may be determined by one of the following methods:

In the first method, the pixel width corresponding to the jump angle is determined as the target pixel width. In this manner, the pixel width corresponding to the jump angle can be used as the target pixel width to ensure that when the viewing point of the viewing angle is rotated from the first viewing angle to the second viewing angle, the second viewing angle is loaded while the line of sight sweeps through the viewing angle switching buffer area. Corresponding video data so that the user does not feel the beating of the picture.

In the second manner, one pixel width is selected from a pixel width larger than a pixel width corresponding to the jump angle and smaller than a pixel width corresponding to the second region of interest, and one pixel width is determined as the target pixel width. In this manner, one pixel width may be selected as the target pixel width in a pixel width corresponding to the jump angle, and the target pixel width is selected such that the view switching buffer area does not exceed the range of the second region of interest.

As an optional solution, playing the image corresponding to the first area at a resolution corresponding to the first area includes:

S1. The playback device plays the image on the first region of interest at a first resolution, and plays the image on the buffer region at a second resolution. The first resolution is higher than or equal to the second resolution. The second resolution is higher than the resolution corresponding to the second region.

Optionally, in this embodiment, when the user views the panoramic video, the region of interest is equivalent to the region directly opposite the user, and the viewing angle switching buffer region and the second region are equivalent to the region covered by the user's residual light and the user does not In the area seen, the user is often not very sensitive to the image scanned in the residual light. Therefore, the image of the viewing angle switching buffer area can be played at a resolution slightly lower than the resolution corresponding to the first region of interest, thereby It is possible, but not limited to, to play images on different areas with different resolutions. For example, the resolution corresponding to the first region of interest may be, but is not limited to, a resolution corresponding to the view switching buffer region, and the resolution corresponding to the view switching buffer region may be, but not limited to, higher than the resolution corresponding to the second region. In order to ensure that the screen is not obviously perceived when the user's perspective is rotated, the occupancy of the system is reduced, thereby improving the playback efficiency of the video.

It should be noted that, for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations, but those skilled in the art should understand that the present application is not limited by the described action sequence. Because certain steps may be performed in other sequences or concurrently in accordance with the present application. In the following, those skilled in the art should also understand that the embodiments described in the specification are optional embodiments, and the actions and modules involved are not necessarily required by the present application.

Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, by hardware, but in many cases, the former is A better implementation. Based on such understanding, the technical solution of the present application, which is essential or contributes to the related art, may be embodied in the form of a software product stored in a storage medium (such as ROM/RAM, disk, CD-ROM). The instructions include a number of instructions for causing a terminal device (which may be a cell phone, computer, server, or network device, etc.) to perform the methods of various embodiments of the present application.

According to an embodiment of the present application, there is also provided a playback device for implementing a panoramic video image of the above-described method for playing a panoramic video image, which is applied to a playback device. As shown in FIG. 6, the device includes:

1) The obtaining module 62 is configured to acquire a first image frame that matches the first viewing angle from the panoramic video image frame in the video stream data to be played, and obtain a matching with the second viewing angle from the panoramic video image frame. a second image frame, wherein the first view angle is a current view angle of view, and the second view angle is a view angle adjacent to the first view angle;

2) The first determining module 64 is configured to determine a first region of interest included in the first image frame, and determine a second region of interest included in the second image frame;

3) The second determining module 66 is configured to determine, according to the first region of interest and the second region of interest, a view switching buffer region included in the first image frame, where the view switching buffer region is corresponding to the second region of interest The first image region and the view switching buffer region form a first region in the first image frame, in the first image frame, in the corresponding region on the first image frame adjacent to the first region of interest The area other than the first area is the second area;

The playback module 68 is configured to play the image corresponding to the first area at a resolution corresponding to the first area, and play the image corresponding to the second area at a resolution corresponding to the second area, where the resolution corresponding to the first area Higher than the resolution corresponding to the second region.

Optionally, in this embodiment, the foregoing image encoding apparatus may be, but is not limited to, being applied to a scene for playing a panoramic video image, such as a virtual reality (VR) video playing process, and a panoramic video playing process. The above play may include but is not limited to: video on demand, live video. The above video may be, but is not limited to, applied to a terminal application such as a video playing application, a game application, a shopping application, and the like. The above is only an example, and is not limited in this embodiment.

Optionally, in this embodiment, the projection model of the video image provided by the playback device of the panoramic video image is similar to a diamond in shape, and may be referred to as a diamond model.

Optionally, in this embodiment, the second viewing angle may be, but is not limited to, a plurality of viewing angles adjacent to the first viewing angle, and the second viewing angle may be, but not limited to, the first viewing angle. Four perspectives of up, down, left, and right adjacent to each other. For example, as shown in FIG. 5, there are four second viewing angles adjacent to the first viewing angle, which are a second viewing angle A, a second viewing angle B, a second viewing angle C, and a second viewing angle D, respectively. The corresponding region of the second region of interest on the first image frame (the region of the shaded portion in the figure) is region A, region B, region C, region D, respectively, and a partial region in each of the above regions As the view switching buffer area, the first area in the first image picture includes the view switching buffer area A, the view switching buffer area B, the view switching buffer area C, the view switching buffer area D, and the first region of interest. The other area of the first image picture other than the above area is the second area.

It can be seen that, by using the foregoing apparatus, the first image frame matched by the first view in the panoramic video image frame includes a first region and a second region, wherein the first region includes the first region of interest and the view switching buffer region, and the view switching buffer The area is a partial area of the picture corresponding to the second area of interest corresponding to the first area of interest in the corresponding area on the first image picture, and the images on the first area and the second area are respectively played at different resolutions, The resolution corresponding to the first area is higher than the resolution corresponding to the second area, so that during the playback of the panoramic video image, the user's line of sight will be swept during the process of switching the video picture from the first perspective to the second perspective. In a region, the view angle switches the picture with higher resolution on the buffer area, so that when the view angle is switched, the bounce of the picture can be avoided, so that the image of the adjacent view is more smoothly connected, thereby overcoming the video picture when the video stream is switched in the related art. There will be a problem with beating.

As an optional solution, the second determining module includes:

1) a first determining unit, configured to determine an area corresponding to a picture corresponding to the second region of interest on the first image frame;

2) The second determining unit is configured to determine that a region of the corresponding region adjacent to the first region of interest has a pixel width of the target pixel width as a view switching buffer region.

As an alternative, the above device further includes:

1) an obtaining module, configured to obtain a pixel width corresponding to a jump angle, wherein the jump angle is an angle that is rotated when the video data corresponding to the first viewing angle is converted to the video data corresponding to the second viewing angle;

2) The third determining module is configured to determine the target pixel width according to the pixel width corresponding to the jump angle.

As an optional solution, the second determining module includes:

1) a third determining unit configured to determine a pixel width corresponding to the jump angle as the target pixel width; or

2) The fourth determining unit is configured to select one pixel width from a pixel width larger than a pixel width corresponding to the jump angle and smaller than a pixel width corresponding to the second region of interest, and determine one pixel width as the target pixel width.

In the first manner, the pixel width corresponding to the jump angle may be used as the target pixel width to ensure that when the viewing point of the viewing angle is rotated from the first viewing angle to the second viewing angle, the second viewing angle is loaded while the line of sight sweeps through the viewing angle switching buffer area. The video data so that the user does not feel the beating of the picture.

In the second manner, one pixel width may be selected as the target pixel width in a pixel width corresponding to the jump angle, and the target pixel width is selected such that the view switching buffer area does not exceed the range of the second region of interest.

As an alternative, the play module includes:

1) a playback unit configured to play an image on the first region of interest at a first resolution and to play an image on the buffer region at a second resolution, wherein the first resolution is higher than or equal to the second The resolution, the second resolution is higher than the resolution corresponding to the second region.

The application environment of the embodiment of the present application may be, but is not limited to, the reference to the application environment in the foregoing embodiments. An embodiment of the present application provides an optional application example for implementing the above-described method for playing a panoramic video image.

As an optional embodiment, the above method for playing a panoramic video image may be, but is not limited to, applied to a process of playing a video image as shown in FIG. 7. The process can include the following steps:

Step S702, monitoring the gyro sensor of the mobile phone device, calculating the current posture position according to the current position information, and notifying the FovPlayer if the posture position is changed.

Step S704, the FOV player (FovPlayer module) transmits information for indicating the FOV view switching to the perspective receiving (FovPv) module of the Fov Push Server (FovPush). The FOV player (FovPlayer module) has one and only one network connection with the Fov Push Server (FovPush), that is, the FovPlayer module is connected to the Fov Recv module, and is set to transmit FOV video stream information and control information for switching the position of the FOV attitude view.

Step S706, the FovPlayer module receives the FOV video stream information transmitted by the Fov Recv module, and decodes the video frame to determine whether the current video frame FOV index changes.

Step S708, the posture position information of the new video frame changes, and the logic of the VR Rendering Module (VR Render) is updated, so that the video rendering position is consistent with the posture position of the frame.

In an alternative embodiment, the FOV video is switched from view A (FOV A) video to view B (FOV B) video, as shown in FIG. 8, and the left side of FIG. 8 shows the FOV video switching process using the pyramid model. When the FOV video is switched in the pyramid model, the FOV B video sequence (that is, the process of passing through the FOV buffer area) is generally started when the head posture crosses the bottom angle position. At this time, the image of the A sequence is still the side of the pyramid. The image of the area is more deformed than the bottom of the image, and the quality of the video quality is also general. In addition, due to image distortion on the side of the diamond, it may also cause the user to watch the VR video to produce a sense of jumping.

The right side of Figure 8 shows the FOV video switching process using the diamond model. In the diamond projection model, the bottom of the model consists of two areas, the positive viewing angle area, and the viewing angle switching buffer area, when the VR helmet attitude moves to the angle of the FOV buffer area. At this time, the quality of the picture viewed by the user is still relatively clear relative to the bottom area. At this time, the FOV B video sequence is loaded, and when viewing the B video sequence, the viewing angle viewing area is the bottom area of the FOV B; therefore, when the video loading speed is sufficient, the FOV video sequence using the diamond model can always provide a stable high for the user. Clear quality picture.

According to still another aspect of the embodiments of the present application, an electronic device for implementing the above-described method for playing a panoramic video image is further provided. As shown in FIG. 9, the electronic device may include one or more (only shown in the figure) The processor 902, the memory 904, the sensor 906, the display 908, the encoder 910, and the transmission device 912 are provided.

The memory 904 can be configured to store a software program and a module, such as a method and a device for playing a panoramic video image in the embodiment of the present application.

Corresponding program instructions/modules, processor 902 executes various functional applications and data processing, ie, image encoding methods, by running software programs and modules stored in memory 904. Memory 904 can include high speed random access memory and can also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid state memory. In some examples, memory 904 can further include memory remotely located relative to processor 902, which can be connected to the terminal over a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The transmission device 912 described above is arranged to receive or transmit data via a network. Examples of network options described above may include wired networks and wireless networks. In one example, the transmission device 912 includes a Network Interface Controller (NIC) that can be connected to other network devices and routers via a network cable to communicate with the Internet or a local area network. In one example, transmission device 912 is a radio frequency (RF) module that is configured to communicate with the Internet wirelessly.

Optionally, the memory 904 is configured to store preset action conditions and information of the preset rights user, and an application.

For an alternative example in this embodiment, reference may be made to the example described in the foregoing embodiment, and details are not described herein again.

A person skilled in the art can understand that the structure shown in FIG. 9 is only schematic, and the electronic device can also be a smart phone (such as an Android mobile phone, an iOS mobile phone, etc.), a tablet computer, an applause computer, and a mobile Internet device (Mobile Internet Devices, MID). ), PAD and other terminal devices. FIG. 9 does not limit the structure of the above electronic device. For example, the electronic device may also include more or fewer components (such as a network interface, display device, etc.) than shown in FIG. 9, or have a different configuration than that shown in FIG.

A person of ordinary skill in the art may understand that all or part of the steps of the foregoing embodiments may be completed by a program to instruct terminal device related hardware, and the program may be stored in a computer readable storage medium, and the storage medium may be Including: flash disk, read-only memory (ROM), random access memory (RAM), disk or optical disk.

Embodiments of the present application also provide a storage medium. Optionally, in this embodiment, the foregoing storage medium may be located in at least one of the plurality of network devices in the network.

Optionally, in the present embodiment, the storage medium is arranged to store program code for performing the following steps:

S1. Acquire a first image frame that matches the first view from the panoramic video image frame in the video stream data to be played, and obtain a second image frame that matches the second view from the panoramic video image frame, where The first viewing angle is a current viewing angle, and the second viewing angle is a viewing angle adjacent to the first viewing angle;

S2, determining a first region of interest included in the first image frame, and determining a second region of interest included in the second image frame;

S3. Determine, according to the first region of interest and the second region of interest, a view switching buffer region included in the first image frame, where the view switching buffer region corresponds to the image corresponding to the second region of interest on the first image frame. a partial region adjacent to the first region of interest in the region, the first region of interest and the view switching buffer region constitute a first region in the first image frame, and the region other than the first region in the first image frame is Second area;

S4: The image corresponding to the first area is played at a resolution corresponding to the first area, and the image corresponding to the second area is played at a resolution corresponding to the second area, where the resolution corresponding to the first area is higher than that of the second area. Resolution.

Optionally, in this embodiment, the foregoing storage medium may include, but not limited to, a USB flash drive, a Read-Only Memory (ROM), a Random Access Memory (RAM), a mobile hard disk, and a magnetic memory. A variety of media that can store program code, such as a disc or a disc.

The serial numbers of the embodiments of the present application are merely for the description, and do not represent the advantages and disadvantages of the embodiments.

The integrated unit in the above embodiment, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in the above-described computer readable storage medium. Based on such understanding, the technical solution of the present application may be embodied in the form of a software product, or the whole or part of the technical solution, which is stored in the storage medium, including The instructions are used to cause one or more computer devices (which may be a personal computer, server or network device, etc.) to perform all or part of the steps of the methods described in the various embodiments of the present application.

In the above-mentioned embodiments of the present application, the descriptions of the various embodiments are different, and the parts that are not detailed in a certain embodiment can be referred to the related descriptions of other embodiments.

In the several embodiments provided by the present application, it should be understood that the disclosed client may be implemented in other manners. The device embodiments described above are merely illustrative. For example, the division of the unit is only a logical function division. In actual implementation, there may be another division manner. For example, multiple units or components may be combined or may be Integrate into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, unit or module, and may be electrical or otherwise.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

The above description is only an alternative embodiment of the present application, and it should be noted that those skilled in the art can make several improvements and retouchings without departing from the principles of the present application. It should also be considered as the scope of protection of this application.

Industrial Applicability: It can be seen from the above description that the first image frame matched by the first view in the panoramic video image frame of the present application includes a first region and a second region, wherein the first region includes the first region of interest and the view switching buffer region The view switching buffer area is a partial area of the picture corresponding to the second region of interest corresponding to the first region of interest in the corresponding region on the first image frame, and the first region and the second region are respectively played at different resolutions. An image on the area, wherein the resolution corresponding to the first area is higher than the resolution corresponding to the second area, so that during the playback of the panoramic video image, the video picture is switched from the first perspective to the second perspective, and the user The line of sight will sweep through the higher resolution picture on the view switching buffer area in the first area, so that when the view angle is switched, the bounce of the picture can be avoided, so that the image of the adjacent view is more smoothly connected, thereby overcoming the video in the related art. Video screens can cause jitter when streaming.

Claims

A method for playing a panoramic video image, comprising:

The playback device acquires a first image frame that matches the first view from the panoramic video image frame in the video stream data to be played, and acquires a second image frame that matches the second view from the panoramic video image frame. The first view angle is a current play view angle, and the second view angle is a view angle adjacent to the first view angle;

The playing device determines a first region of interest included in the first image frame, and determines a second region of interest included in the second image frame;

Determining, by the playback device, the view switching buffer region included in the first image frame according to the first region of interest and the second region of interest, wherein the view switching buffer region is the second sense a picture corresponding to the region of interest is a partial region adjacent to the first region of interest in a corresponding region on the first image frame, the first region of interest and a view switching buffer region constituting the first image frame a first area of the first image, wherein the area other than the first area is a second area;

The playing device plays an image on the first area at a resolution corresponding to the first area, and plays an image on the second area in a resolution corresponding to the second area, where the first The resolution corresponding to the region is higher than the resolution corresponding to the second region.
The method according to claim 1, wherein the determining, by the playback device, the view switching buffer area included in the first image frame according to the first region of interest and the second region of interest comprises:

The playing device determines an area corresponding to the picture corresponding to the second region of interest on the first image frame;

The playing device determines that the area of the corresponding area that has a pixel width adjacent to the first area of interest as the target pixel width is the view switching buffer area.
The method according to claim 2, wherein the playback device determines that a region of the corresponding region adjacent to the first region of interest has a pixel width of a target pixel width before the view switching buffer region The method further includes:

The playback device acquires a pixel width corresponding to the jump angle, where the jump angle is an angle that is rotated when the video data corresponding to the first view is played to be converted to the video data corresponding to the second view;

The playback device determines the target pixel width according to a pixel width corresponding to the jump angle.
The method according to claim 3, wherein the determining, by the playback device, the target pixel width according to a pixel width corresponding to the jump angle comprises:

The playing device determines a pixel width corresponding to the jump angle as the target pixel width; or

The playback device selects one pixel width from a pixel width greater than a pixel width corresponding to the jump angle and smaller than a pixel width corresponding to the second region of interest, and determines the one pixel width as the target pixel width.
The method according to any one of claims 1 to 4, wherein the playing of the image corresponding to the first area by the playback device corresponding to the resolution of the first area comprises:

Playing the image on the first region of interest at a first resolution, and playing an image on the view switching buffer region at a second resolution, wherein the first resolution is higher than or equal to The second resolution is higher than a resolution corresponding to the second area.
A playback device for a panoramic video image, applied to a playback device, comprising one or more processors, and one or more memories storing program units, wherein the program units are executed by the processor, the program units include:

An acquiring module, configured to: when playing video data in a virtual reality scene, acquire a first image image that matches a first perspective from a panoramic video image frame in the video stream data to be played, and from the panorama Obtaining, in the video image frame, a second image frame that matches the second viewing angle, wherein the first viewing angle is a current playback perspective, and the second perspective is a perspective adjacent to the first perspective;

a first determining module, configured to determine a first region of interest included in the first image frame, and determine a second region of interest included in the second image frame;

a second determining module, configured to determine, according to the first region of interest and the second region of interest, a view switching buffer region included in the first image frame, wherein the view switching buffer region is a picture corresponding to the second region of interest, a partial region adjacent to the first region of interest in a corresponding region on the first image frame, the first region of interest and a view switching buffer region a first area in the first image frame, wherein the area other than the first area in the first image picture is a second area;

a playing module, configured to play an image on the first area at a resolution corresponding to the first area, and play an image on the second area in a resolution corresponding to the second area, where The resolution corresponding to the first region is higher than the resolution corresponding to the second region.
The apparatus of claim 6, wherein the second determining module comprises:

a first determining unit, configured to determine an area corresponding to the picture corresponding to the second region of interest on the first image frame;

The second determining unit is configured to determine that the region of the corresponding region that has a pixel width adjacent to the first region of interest as the target pixel width is the view switching buffer region.
The apparatus of claim 7 wherein said apparatus further comprises:

The obtaining module is configured to obtain a pixel width corresponding to the jump angle, wherein the jump angle is an angle of rotation when the video data corresponding to the first view is played to be converted to the video data corresponding to the second view ;

The third determining module is configured to determine the target pixel width according to a pixel width corresponding to the jump angle.
The apparatus of claim 8, wherein the third determining module comprises:

a third determining unit, configured to determine a pixel width corresponding to the jump angle as the target pixel width; or

a fourth determining unit, configured to select one pixel width from a pixel width greater than a pixel width corresponding to the jump angle and smaller than a pixel width corresponding to the second region of interest, and determine the one pixel width as The target pixel width.
The apparatus according to any one of claims 6 to 9, wherein the play module comprises:

a playing unit configured to play an image on the first region of interest at a first resolution and play an image on the view switching buffer region at a second resolution, wherein the first resolution is higher Or equal to the second resolution, the second resolution being higher than a resolution corresponding to the second region.
A storage medium comprising a stored program, wherein the program is executed to perform the method of any of the preceding claims 1 to 5.
An electronic device comprising a memory, a processor, and a computer program stored on the memory and operable on the processor, the processor executing the above claims 1 to 5 by the computer program Said method.