CN111866485A - Stereoscopic picture projection and transmission method, device and computer readable storage medium - Google Patents
Stereoscopic picture projection and transmission method, device and computer readable storage medium Download PDFInfo
- Publication number
- CN111866485A CN111866485A CN201910340536.0A CN201910340536A CN111866485A CN 111866485 A CN111866485 A CN 111866485A CN 201910340536 A CN201910340536 A CN 201910340536A CN 111866485 A CN111866485 A CN 111866485A
- Authority
- CN
- China
- Prior art keywords
- chip
- area
- area ratio
- preset
- determining
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
- H04N13/194—Transmission of image signals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
- H04N13/106—Processing image signals
- H04N13/139—Format conversion, e.g. of frame-rate or size
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/238—Interfacing the downstream path of the transmission network, e.g. adapting the transmission rate of a video stream to network bandwidth; Processing of multiplex streams
- H04N21/2385—Channel allocation; Bandwidth allocation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
- H04N21/845—Structuring of content, e.g. decomposing content into time segments
- H04N21/8456—Structuring of content, e.g. decomposing content into time segments by decomposing the content in the time domain, e.g. in time segments
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The embodiment of the invention provides a method and a device for projecting and transmitting a stereoscopic picture and a computer readable storage medium, wherein the method for transmitting the stereoscopic picture comprises the following steps: determining the area ratio among preset areas in the projection; determining the data volume of the current transmission code sheet based on the area ratio among the preset areas; bandwidth is allocated to each chip stream based on the amount of data per chip.
Description
Technical Field
The present invention relates to the field of mobile communications technologies, and in particular, to a Virtual Reality (VR) video multi-stream adaptive transmission method, apparatus, and computer-readable storage medium.
Background
Currently, the transmission scheme of VR video includes: most VR video uses conventional techniques, view-adaptive transmission schemes (i.e., single-stream adaptive transmission schemes), and block-based rate-adaptive schemes (i.e., multi-stream adaptive transmission schemes as described herein).
Current VR video multi-stream adaptive transmission schemes do not account for the waste of resources caused by the natural distortion of the top and bottom during isometric projection. When the network fluctuates, the overall picture quality is degraded synchronously, but this is not an optimal choice for improving the overall QoE. Meanwhile, the frame is a result that the data amount of the blocks at different positions is greatly different when the dynamic adaptive streaming media is used for transmission, and the difference may cause that the cache contents are not synchronous and are blocked in the playing process.
Disclosure of Invention
In view of the foregoing, embodiments of the present invention are directed to a VR video multi-stream adaptive transmission method, apparatus, and computer-readable storage medium.
In order to achieve the above purpose, the technical solution of the embodiment of the present invention is realized as follows:
the embodiment of the invention provides a method for transmitting a three-dimensional picture, which comprises the following steps:
determining the area ratio among preset areas in the projection;
determining the data volume of the current transmission code sheet based on the area ratio among the preset areas;
bandwidth is allocated to each chip stream based on the amount of data per chip.
Wherein the allocating bandwidth for each chip stream based on the amount of data per chip comprises:
determining a proportion of the data amount based on each chip to the total data amount;
and allocating the bandwidth for each chip stream based on the proportion of the data amount of each chip to the total data amount.
Wherein the allocating bandwidth for each chip stream based on the amount of data per chip comprises:
setting the weight of each chip stream based on the data amount of each chip and/or the proportion of the data amount of each chip to the total data amount;
and allocating bandwidth for each stream based on the weight of each chip stream.
Wherein the determining the data amount of the current transmission chip based on the area ratio between the preset regions includes:
and determining the data quantity of the current transmission chip based on the area ratio among the preset areas and/or the compression rate of different chips.
The embodiment of the invention also provides a method for projecting and transmitting the stereoscopic picture, which comprises the following steps:
determining the area ratio among preset areas in the projection;
sampling the projected full picture based on the area ratio.
And the pixel density of each preset area in the projection obtained by sampling is the same.
Wherein the area ratio is: the area size is taken as a weight of the total sampling rate distributed among the different preset regions.
Wherein, when each preset area is divided in the direction of spherical latitude, the method comprises the following steps:
a top region near the pole of the sphere, a bottom region near the pole of the sphere, and a middle region on either side of the equator.
Wherein, each preset region is realized based on spherical equiangular division, and the area ratio between the preset regions is as follows: smiddle:Stop:Sbottom＝4.826:1:1；
Wherein, the SmiddleDenotes the area of the middle region on both sides of the equator, StopRepresenting the area of the top region near the pole of the sphere, S bottomRepresenting the area of the bottom region near the pole of the sphere.
In the equiangular projection, the ratio of the number of pixels of the middle area on both sides of the equator to the top area close to the spherical pole and the ratio of the number of pixels of the middle area on both sides of the equator to the bottom area close to the spherical pole are both:
wherein said sampling the projected full picture based on the area ratio comprises:
determining the resolution of each preset area based on the area ratio and the preset total resolution of the complete picture;
and sampling the projected complete picture based on the resolution of each preset area.
The embodiment of the invention also provides a three-dimensional picture transmission device, which comprises:
the second determining module is used for determining the area ratio among all preset areas in the projection;
a third determining module, configured to determine a data amount of a current transmission chip based on an area ratio between the preset regions;
and the allocation module is used for allocating bandwidth for each chip stream based on the data quantity of each chip.
The embodiment of the invention also provides a device for projecting and transmitting the stereoscopic picture, which comprises:
the first determining module is used for determining the area ratio among all preset areas in the projection;
And the sampling module is used for sampling the projected complete picture based on the area ratio.
The embodiment of the invention also provides a virtual reality VR video multi-stream self-adaptive transmission device, which comprises: a processor and a memory for storing a computer program capable of running on the processor,
wherein the processor is configured to perform the steps of the above method when running the computer program.
Embodiments of the present invention also provide a computer-readable storage medium, on which a computer program is stored, which, when executed by a processor, implements the steps of the above-mentioned method.
According to the VR video multi-stream self-adaptive transmission method, the VR video multi-stream self-adaptive transmission device and the computer readable storage medium, the area ratio among preset areas in projection is determined; sampling the projected full picture based on the area ratio. The embodiment of the invention adopts a multi-stream self-adaptive transmission mode based on the block (the preset area), avoids the problems that the visual angle of a user is changed and the cache content needs to be completely discarded during single-stream transmission, and improves the resolution of the block within the visual angle of the user under the same bandwidth and the whole QoE (quality of experience) because only the resolution of part of the block is changed.
Meanwhile, aiming at the problems existing in multi-stream transmission, the embodiment of the invention determines the data volume of the current transmission chip based on the area ratio among all the preset areas; based on the data volume of each chip, bandwidth is allocated to each chip stream, and the problem caused by asynchronous video cache can be solved.
Drawings
Fig. 1 is a schematic flow chart of a stereoscopic image projection and transmission method according to an embodiment of the present invention;
fig. 2 is a schematic flow chart of a stereoscopic image transmission method according to an embodiment of the invention;
FIG. 3 is a schematic structural diagram of a stereoscopic projection and transmission apparatus according to an embodiment of the invention;
fig. 4 is a schematic structural diagram of a stereoscopic image transmission device according to an embodiment of the invention;
FIG. 5 is a schematic illustration of a regional downsampling of an isometric projection according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of calculating the surface area of the spherical cap according to the embodiment of the present invention.
Detailed Description
The invention is described below with reference to the figures and examples.
An embodiment of the present invention provides a stereoscopic image projection and transmission method, as shown in fig. 1, the method includes:
step 101: determining the area ratio among preset areas in the projection;
step 102: sampling the projected full picture based on the area ratio.
The embodiment of the invention adopts a multi-stream self-adaptive transmission mode based on the block (the preset area), avoids the problems that the visual angle of a user is changed and the cache content needs to be completely discarded during single-stream transmission, and improves the resolution of the block within the visual angle of the user under the same bandwidth and the whole QoE (quality of experience) because only the resolution of part of the block is changed.
In the embodiment of the invention, the pixel density of each preset area in the projection obtained by sampling is the same.
In the embodiment of the present invention, the area ratio is: the area size is taken as a weight of the total sampling rate distributed among the different preset regions.
In the embodiment of the present invention, when dividing each preset area in the spherical latitude direction, the method includes:
a top region near the pole of the sphere, a bottom region near the pole of the sphere, and a middle region on either side of the equator.
In the embodiment of the invention, each preset area is realized based on spherical equiangular division, and the area ratio of the preset areas is as follows: smiddle:Stop:Sbottom＝4.826:1:1；
Wherein, the SmiddleDenotes the area of the middle region on both sides of the equator, StopRepresenting the area of the top region near the pole of the sphere, SbottomRepresenting the area of the bottom region near the pole of the sphere.
In the embodiment of the present invention, in the isometric projection, the ratio of the number of pixels in the middle area on both sides of the equator to the top area near the spherical pole and the ratio of the number of pixels in the middle area on both sides of the equator to the bottom area near the spherical pole are both:
In this embodiment of the present invention, the sampling the projected complete picture based on the area ratio includes:
determining the resolution of each preset area based on the area ratio and the preset total resolution of the complete picture;
and sampling the projected complete picture based on the resolution of each preset area.
Here, the resolution directly corresponds to the number of pixels in one screen space; the sampling rate expresses the interval of picture acquisition spatially to achieve the corresponding pixel density.
An embodiment of the present invention further provides a method for transmitting a stereoscopic image, as shown in fig. 2, the method includes:
step 201: determining the area ratio among preset areas in the projection;
step 202: determining the data volume of the current transmission code sheet based on the area ratio among the preset areas;
step 203: bandwidth is allocated to each chip stream based on the amount of data per chip.
Here, each preset region is a corresponding spatial region of the projection, and may correspond to one chip, and the chip streams based on different preset regions may be distinguished.
In this embodiment of the present invention, the allocating a bandwidth to each chip stream based on the data amount of each chip includes:
Determining a proportion of the data amount based on each chip to the total data amount;
and allocating the bandwidth for each chip stream based on the proportion of the data amount of each chip to the total data amount.
For example: assuming that the total bandwidth is 10M, the ratio of the data amount of different chips is 4.8:1:1, and the bandwidth corresponding to each chip stream is 10/(4.8+2) × 4.8, 10/(4.8+2), and 10/(4.8+2), respectively.
In this embodiment of the present invention, the allocating a bandwidth to each chip stream based on the data amount of each chip includes:
setting the weight of each chip stream based on the data amount of each chip and/or the proportion of the data amount of each chip to the total data amount;
and allocating bandwidth for each stream based on the weight of each chip stream.
In this embodiment of the present invention, the determining the data amount of the current transmission chip based on the area ratio between the preset regions includes:
and determining the data quantity of the current transmission chip based on the area ratio between the preset areas and/or the compression rate (or compression coefficient) of different chips.
In order to implement the foregoing method, an embodiment of the present invention further provides a stereoscopic image projection and transmission apparatus, as shown in fig. 3, the apparatus includes:
A first determining module 301, configured to determine an area ratio between preset regions in the projection;
a sampling module 302, configured to sample the projected complete picture based on the area ratio.
In this embodiment of the present invention, the pixel densities of the preset regions in the projection obtained by sampling by the sampling module 302 are the same.
In the embodiment of the present invention, the area ratio is: the area size is taken as a weight of the total sampling rate distributed among the different preset regions.
In the embodiment of the present invention, when dividing each preset area in the spherical latitude direction, the method includes:
a top region near the pole of the sphere, a bottom region near the pole of the sphere, and a middle region on either side of the equator.
In the embodiment of the invention, each preset area is realized based on spherical equiangular division, and the area ratio of the preset areas is as follows: smiddle:Stop:Sbottom＝4.826:1:1；
Wherein, the SmiddleDenotes the area of the middle region on both sides of the equator, StopRepresenting the area of the top region near the pole of the sphere, SbottomRepresenting the area of the bottom region near the pole of the sphere.
In the embodiment of the present invention, in the isometric projection, the ratio of the number of pixels in the middle area on both sides of the equator to the top area near the spherical pole and the ratio of the number of pixels in the middle area on both sides of the equator to the bottom area near the spherical pole are both:
In this embodiment of the present invention, the sampling module 302 samples the projected complete picture based on the area ratio, including:
determining the resolution of each preset area based on the area ratio and the preset total resolution of the complete picture;
and sampling the projected complete picture based on the resolution of each preset area.
Here, the resolution directly corresponds to the number of pixels in one screen space; the sampling rate expresses the interval of picture acquisition spatially to achieve the corresponding pixel density.
An embodiment of the present invention further provides a stereoscopic image transmission device, as shown in fig. 4, the device includes:
a second determining module 401, configured to determine an area ratio between preset regions in the projection;
a third determining module 402, configured to determine a data amount of a current transmission chip based on an area ratio between the preset regions;
an allocating module 403, configured to allocate a bandwidth for each chip stream based on the data amount of each chip.
Here, each preset region is a corresponding spatial region of the projection, and may correspond to one chip, and the chip streams based on different preset regions may be distinguished.
In this embodiment of the present invention, the allocating module 403 allocates a bandwidth to each chip stream based on the data amount of each chip, including:
Determining a proportion of the data amount based on each chip to the total data amount;
and allocating the bandwidth for each chip stream based on the proportion of the data amount of each chip to the total data amount.
For example: assuming that the total bandwidth is 10M, the ratio of the data amount of different chips is 4.8:1:1, and the bandwidth corresponding to each chip stream is 10/(4.8+2) × 4.8, 10/(4.8+2), and 10/(4.8+2), respectively.
In this embodiment of the present invention, the allocating module 403 allocates a bandwidth to each chip stream based on the data amount of each chip, including:
setting the weight of each chip stream based on the data amount of each chip and/or the proportion of the data amount of each chip to the total data amount;
and allocating bandwidth for each stream based on the weight of each chip stream.
In this embodiment of the present invention, the determining module 402 determines the data amount of the current transmission chip based on the area ratio between the preset regions, including:
and determining the data quantity of the current transmission chip based on the area ratio between the preset areas and/or the compression rate (or compression coefficient) of different chips.
The embodiment of the invention also provides a virtual reality VR video multi-stream self-adaptive transmission device, which comprises: a processor and a memory for storing a computer program capable of running on the processor,
Wherein the processor is configured to execute, when running the computer program:
determining the area ratio among preset areas in the projection;
sampling the projected full picture based on the area ratio.
And the pixel density of each preset area in the projection obtained by sampling is the same.
Wherein the area ratio is: the area size is taken as a weight of the total sampling rate distributed among the different preset regions.
Wherein, when each preset area is divided in the direction of spherical latitude, the method comprises the following steps:
a top region near the pole of the sphere, a bottom region near the pole of the sphere, and a middle region on either side of the equator.
Wherein, each preset region is realized based on spherical equiangular division, and the area ratio between the preset regions is as follows: smiddle:Stop:Sbottom＝4.826:1:1；
Wherein, the SmiddleDenotes the area of the middle region on both sides of the equator, StopRepresenting the area of the top region near the pole of the sphere, SbottomRepresenting the area of the bottom region near the pole of the sphere.
In the equiangular projection, the ratio of the number of pixels of the middle area on both sides of the equator to the top area close to the spherical pole and the ratio of the number of pixels of the middle area on both sides of the equator to the bottom area close to the spherical pole are both:
The processor is further configured to, when the projected complete picture is sampled based on the area ratio, execute:
determining the resolution of each preset area based on the area ratio and the preset total resolution of the complete picture;
and sampling the projected complete picture based on the resolution of each preset area.
The embodiment of the invention also provides a virtual reality VR video multi-stream self-adaptive transmission device, which comprises: a processor and a memory for storing a computer program capable of running on the processor,
wherein the processor is configured to execute, when running the computer program:
determining the area ratio among preset areas in the projection;
determining the data volume of the current transmission code sheet based on the area ratio among the preset areas;
bandwidth is allocated to each chip stream based on the amount of data per chip.
When the bandwidth is allocated to each chip stream based on the data amount of each chip, the processor is further configured to execute, when the computer program is executed:
determining a proportion of the data amount based on each chip to the total data amount;
and allocating the bandwidth for each chip stream based on the proportion of the data amount of each chip to the total data amount.
When the bandwidth is allocated to each chip stream based on the data amount of each chip, the processor is further configured to execute, when the computer program is executed:
setting the weight of each chip stream based on the data amount of each chip and/or the proportion of the data amount of each chip to the total data amount;
and allocating bandwidth for each stream based on the weight of each chip stream.
When the data amount of the current transmission chip is determined based on the area ratio between the preset regions, the processor is further configured to execute, when the computer program is run, the following steps:
and determining the data quantity of the current transmission chip based on the area ratio among the preset areas and/or the compression rate of different chips.
It should be noted that: in the above embodiment, when performing stereoscopic image projection and transmission, the device is only illustrated by dividing the program modules, and in practical applications, the above processing may be distributed and completed by different program modules according to needs, that is, the internal structure of the apparatus is divided into different program modules to complete all or part of the above-described processing. In addition, the apparatus provided in the above embodiments and the corresponding method embodiments belong to the same concept, and specific implementation processes thereof are described in the method embodiments and are not described herein again.
In an exemplary embodiment, the embodiment of the present invention also provides a computer-readable storage medium, which may be a Memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface Memory, optical disc, or CD-ROM; or may be a variety of devices including one or any combination of the above memories, such as a mobile phone, computer, tablet device, personal digital assistant, etc.
An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, performs:
determining the area ratio among preset areas in the projection;
sampling the projected full picture based on the area ratio.
And the pixel density of each preset area in the projection obtained by sampling is the same.
Wherein the area ratio is: the area size is taken as a weight of the total sampling rate distributed among the different preset regions.
Wherein, when each preset area is divided in the direction of spherical latitude, the method comprises the following steps:
a top region near the pole of the sphere, a bottom region near the pole of the sphere, and a middle region on either side of the equator.
Wherein, each preset region is realized based on spherical equiangular division, and the area ratio between the preset regions is as follows: smiddle:Stop:Sbottom＝4.826:1:1；
Wherein, the SmiddleDenotes the area of the middle region on both sides of the equator, StopRepresenting the area of the top region near the pole of the sphere, SbottomRepresenting the area of the bottom region near the pole of the sphere.
In the equiangular projection, the ratio of the number of pixels of the middle area on both sides of the equator to the top area close to the spherical pole and the ratio of the number of pixels of the middle area on both sides of the equator to the bottom area close to the spherical pole are both:
when the computer program is executed by a processor when sampling the projected complete picture based on the area ratio, further performing:
determining the resolution of each preset area based on the area ratio and the preset total resolution of the complete picture;
and sampling the projected complete picture based on the resolution of each preset area.
An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, performs:
determining the area ratio among preset areas in the projection;
Determining the data volume of the current transmission code sheet based on the area ratio among the preset areas;
bandwidth is allocated to each chip stream based on the amount of data per chip.
When the bandwidth is allocated to each chip stream based on the data amount per chip, the computer program when executed by the processor further performs:
determining a proportion of the data amount based on each chip to the total data amount;
and allocating the bandwidth for each chip stream based on the proportion of the data amount of each chip to the total data amount.
When the bandwidth is allocated to each chip stream based on the data amount per chip, the computer program when executed by the processor further performs:
setting the weight of each chip stream based on the data amount of each chip and/or the proportion of the data amount of each chip to the total data amount;
and allocating bandwidth for each stream based on the weight of each chip stream.
When the data amount of the current transmission chip is determined based on the area ratio between the preset regions, the computer program is executed by a processor, and further executes:
and determining the data quantity of the current transmission chip based on the area ratio among the preset areas and/or the compression rate of different chips.
The invention is described below in conjunction with the scenario embodiments.
The present embodiment proposes the following two optimization schemes on the basis of the multiflow adaptive transmission scheme:
first, video quality non-synchronous change (spherical coding differentiation sampling rate distribution) when network fluctuates
The problem with the equiangular projection format is that there are a large number of redundant samples of pixels in the polar regions of a spherical scene, and the processing and encoding of these extra samples results in high bit rates on the video data and an increase in codec computational complexity. Schemes related to regional down-sampling (RDS) have been used to improve the coding performance of panoramic video. Based on this, the present embodiment simplifies the model of the equiangular projection to that shown in fig. 5, resampling the equiangular projection panorama in the latitudinal direction, such that the region near the pole undergoes higher down-sampling than the equatorial region. Of course, the regions may be divided in the longitudinal direction, and equal angles are always required.
As can be seen from fig. 5, the panoramic video is divided into three parts, i.e., top, middle and bottom, by 45 ° -90 ° -45 ° in the horizontal direction, and after the area down-sampling of the embodiment, the middle part remains unchanged, and the sampling rates of the top and bottom parts are both reduced by a certain proportion. Specific sampling values of top and bottom are calculated as follows:
The distribution of pixels on the sphere varies based on the variation of the perimeter of each slice (or rather based on the variation of the area) and it produces a uniform sampling on the sphere, the pixel density near the equator of the sphere being close to that of the midle part in the equiangular projection. Based on this, the pixel density of the entire sphere is set to be constant, that is, the pixel density of the midle portion. As shown in fig. 6, the formula for calculating the surface area of the spherical cap is:
S＝2πR2(1-sinθ)；
the angle chosen when dividing the zone is 45-90-45, so here θ is:
θ＝45°；
i.e. the area of the top part is:
also, the area of the bottom part is:
the area of midle moiety is:
in this case, the area ratio of midle, top and bottom is:
Smiddle:Stop:Sbottom＝4.826:1:1；
it can be seen that, in the equiangular projection, the ratio of the number of pixels of the middle area midle on both sides of the equator to the top area top near the sphere pole, and the ratio of the number of pixels of the middle area midle on both sides of the equator to the bottom area bottom near the sphere pole are:
in this embodiment, the pixel densities of the preset regions in the projection obtained by sampling are the same. Therefore, in order to make the pixel density of the top and bottom parts equal to that of the midle part, in the isometric projection, assuming that the current total resolution is v, the resolution of each part is: the resolution of the midle part is 0.5v, and that of both top and bottom parts is 0.25 v. When the pixel density of the top and bottom parts is ensured to be midle, the ratio of the pixel number of the top to the pixel number of the bottom to the original is as follows:
Namely: under the condition of ensuring that the integral resolution is not changed, the resolution of the top part and the bottom part can be reduced to be 0.586 time of the original resolution. And sampling the panoramic image based on the resolution, wherein for the top area and the bottom area with low resolution, the sampling rate is relatively reduced to 0.586 time, and the middle part is unchanged to ensure the picture quality.
Block-based network bandwidth adaptive allocation strategy
It can be seen that the higher the sampling rate, the larger the filter bandwidth.
The multi-stream adaptive transmission scheme uses DASH to transmit multiple streams and each stream is transmitted independently. As a multi-stream, each stream competes for bandwidth collectively and is allocated independently of each other. To address this issue, resource scheduling may be performed over bandwidth. The average allocation of bandwidth is not suitable because there is a difference in the amount of data of each part. In order to ensure that the resolution of the whole picture is the same, but the data size of each part of the slice with the same time length is different, according to the calculation:
Smiddle:Stop:Sbotto＝4.826:1:1；
the area of each block of the middle four blocks is about 1.206 (i.e. 4.826/4), which is larger than 1 of the top and the bottom, and meanwhile, considering that the content of the middle part is usually the visual center of the user, there are many moving objects, the top and the bottom are usually sky and earth, and basically remain still, the compression ratio of the two parts of video is also very different, so that the data amount of the middle four parts is obviously higher than that of the other two parts.
The present embodiment proposes a simple and effective solution, i.e. a bandwidth allocation module can be set up. Before downloading (transmitting) each chip, according to information in an MPD file, counting data quantity of a current transmission chip and calculating proportion of each chip in total data quantity, giving different weight to each stream according to the proportion, and a bandwidth allocation module allocates bandwidth in real time according to the weight of each stream. Specifically, the method can comprise the following steps:
determining the area ratio among preset areas in the projection;
determining the data volume of the current transmission code sheet based on the area ratio among the preset areas;
bandwidth is allocated to each chip stream based on the amount of data per chip.
In one embodiment, the allocating bandwidth to each chip stream based on the data amount per chip includes:
determining a proportion of the data amount based on each chip to the total data amount;
and allocating the bandwidth for each chip stream based on the proportion of the data amount of each chip to the total data amount.
In another embodiment, the allocating bandwidth to each chip stream based on the data amount per chip includes:
setting the weight of each chip stream based on the data amount of each chip and/or the proportion of the data amount of each chip to the total data amount;
And allocating bandwidth for each stream based on the weight of each chip stream.
The embodiment of the invention adopts a multi-stream self-adaptive transmission mode based on the block (the preset area), avoids the problems that the visual angle of a user is changed and the cache content needs to be completely discarded during single-stream transmission, and improves the resolution of the block within the visual angle of the user under the same bandwidth and the whole QoE (quality of experience) because only the resolution of part of the block is changed.
Meanwhile, aiming at the problems existing in multi-stream transmission, the embodiment of the invention determines the data volume of the current transmission chip based on the area ratio among all the preset areas; based on the data volume of each chip, bandwidth is allocated to each chip stream, and the problem caused by asynchronous video cache can be solved.
The above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention.
Claims (15)
1. A method for transmitting a stereoscopic picture, the method comprising:
determining the area ratio among preset areas in the projection;
determining the data volume of the current transmission code sheet based on the area ratio among the preset areas;
bandwidth is allocated to each chip stream based on the amount of data per chip.
2. The method of claim 1, wherein the allocating bandwidth for each chip stream based on the amount of data per chip comprises:
Determining a proportion of the data amount based on each chip to the total data amount;
and allocating the bandwidth for each chip stream based on the proportion of the data amount of each chip to the total data amount.
3. The method of claim 1 or 2, wherein the allocating bandwidth for each chip stream based on the amount of data per chip comprises:
setting the weight of each chip stream based on the data amount of each chip and/or the proportion of the data amount of each chip to the total data amount;
and allocating bandwidth for each stream based on the weight of each chip stream.
4. The method of claim 1, wherein the determining the data amount of the current transmission chip based on the area ratio between the preset regions comprises:
and determining the data quantity of the current transmission chip based on the area ratio among the preset areas and/or the compression rate of different chips.
5. A stereoscopic picture projection and transmission method, comprising:
determining the area ratio among preset areas in the projection;
sampling the projected full picture based on the area ratio.
6. The method of claim 5, wherein the pixel density of each predetermined area in the projection obtained by sampling is the same.
7. The method of claim 5, wherein the area ratio is: the area size is taken as a weight of the total sampling rate distributed among the different preset regions.
8. The method of claim 5, wherein the dividing of each predetermined area in the latitudinal direction of the sphere comprises:
a top region near the pole of the sphere, a bottom region near the pole of the sphere, and a middle region on either side of the equator.
9. The method according to claim 8, wherein the preset regions are implemented based on spherical equiangular division, and the area ratio between the preset regions is as follows: smiddle:Stop:Sbottom＝4.826:1:1；
Wherein, the SmiddleDenotes the area of the middle region on both sides of the equator, StopRepresenting the area of the top region near the pole of the sphere, SbottomRepresenting the area of the bottom region near the pole of the sphere.
10. The method of claim 9, wherein in equiangular projection, the ratio of the number of pixels in the middle region on either side of the equator to the top region near the pole of the sphere and the ratio of the number of pixels in the middle region on either side of the equator to the bottom region near the pole of the sphere are:
11. the method of claim 5, wherein sampling the projected full picture based on the area ratio comprises:
Determining the resolution of each preset area based on the area ratio and the preset total resolution of the complete picture;
and sampling the projected complete picture based on the resolution of each preset area.
12. A stereoscopic picture transmission apparatus, comprising:
the second determining module is used for determining the area ratio among all preset areas in the projection;
a third determining module, configured to determine a data amount of a current transmission chip based on an area ratio between the preset regions;
and the allocation module is used for allocating bandwidth for each chip stream based on the data quantity of each chip.
13. A stereoscopic picture projection and transmission apparatus, comprising:
the first determining module is used for determining the area ratio among all preset areas in the projection;
and the sampling module is used for sampling the projected complete picture based on the area ratio.
14. A virtual reality VR video multi-stream adaptive transmission apparatus, comprising: a processor and a memory for storing a computer program capable of running on the processor,
wherein the processor is adapted to perform the steps of the method of any one of claims 1 to 4 or to perform the steps of the method of any one of claims 5 to 11 when running the computer program.
15. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 4 or carries out the steps of the method of any one of claims 5 to 11.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910340536.0A CN111866485A (en) | 2019-04-25 | 2019-04-25 | Stereoscopic picture projection and transmission method, device and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910340536.0A CN111866485A (en) | 2019-04-25 | 2019-04-25 | Stereoscopic picture projection and transmission method, device and computer readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111866485A true CN111866485A (en) | 2020-10-30 |
Family
ID=72951533
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910340536.0A Pending CN111866485A (en) | 2019-04-25 | 2019-04-25 | Stereoscopic picture projection and transmission method, device and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111866485A (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101494785A (en) * | 2008-12-19 | 2009-07-29 | 无锡亿普得科技有限公司 | Method for encoding H.264 interesting region |
| CN107409233A (en) * | 2015-03-05 | 2017-11-28 | 索尼公司 | Image processing apparatus and image processing method |
| US20180091577A1 (en) * | 2015-04-22 | 2018-03-29 | Samsung Electronics Co., Ltd. | Method and apparatus for transmitting and receiving image data for virtual-reality streaming service |
| US20180109810A1 (en) * | 2016-10-17 | 2018-04-19 | Mediatek Inc. | Method and Apparatus for Reference Picture Generation and Management in 3D Video Compression |
| CN108391103A (en) * | 2017-01-03 | 2018-08-10 | 黑帆科技有限公司 | VR video observings shadow zone domain classification method and device |
| CN108520492A (en) * | 2018-03-16 | 2018-09-11 | 中国传媒大学 | panoramic video mapping method and system |
| CN108566554A (en) * | 2018-05-11 | 2018-09-21 | 北京奇艺世纪科技有限公司 | A kind of VR panoramic videos processing method, system and electronic equipment |
| CN108833880A (en) * | 2018-04-26 | 2018-11-16 | 北京大学 | Using across user behavior pattern carry out view prediction and realize that virtual reality video optimizes the method and apparatus transmitted |
-
2019
- 2019-04-25 CN CN201910340536.0A patent/CN111866485A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101494785A (en) * | 2008-12-19 | 2009-07-29 | 无锡亿普得科技有限公司 | Method for encoding H.264 interesting region |
| CN107409233A (en) * | 2015-03-05 | 2017-11-28 | 索尼公司 | Image processing apparatus and image processing method |
| US20180091577A1 (en) * | 2015-04-22 | 2018-03-29 | Samsung Electronics Co., Ltd. | Method and apparatus for transmitting and receiving image data for virtual-reality streaming service |
| US20180109810A1 (en) * | 2016-10-17 | 2018-04-19 | Mediatek Inc. | Method and Apparatus for Reference Picture Generation and Management in 3D Video Compression |
| CN108391103A (en) * | 2017-01-03 | 2018-08-10 | 黑帆科技有限公司 | VR video observings shadow zone domain classification method and device |
| CN108520492A (en) * | 2018-03-16 | 2018-09-11 | 中国传媒大学 | panoramic video mapping method and system |
| CN108833880A (en) * | 2018-04-26 | 2018-11-16 | 北京大学 | Using across user behavior pattern carry out view prediction and realize that virtual reality video optimizes the method and apparatus transmitted |
| CN108566554A (en) * | 2018-05-11 | 2018-09-21 | 北京奇艺世纪科技有限公司 | A kind of VR panoramic videos processing method, system and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111615715B (en) | Method, apparatus and stream for encoding/decoding volumetric video | |
| KR102013403B1 (en) | Spherical video streaming | |
| Chiariotti | A survey on 360-degree video: Coding, quality of experience and streaming | |
| CN109327699B (en) | Image processing method, terminal and server | |
| CN108513119A (en) | Mapping, processing method, device and the machine readable media of image | |
| US10242462B2 (en) | Rate control bit allocation for video streaming based on an attention area of a gamer | |
| WO2018010695A1 (en) | Method and apparatus for video coding | |
| CN110149542B (en) | Transmission control method | |
| CN117201817A (en) | Method and system for signaling 360 degree video information | |
| CN104735464A (en) | Panorama video interactive transmission method, server and client end | |
| CN110036640B (en) | System and method for supporting video bitstream switching | |
| CN109362242A (en) | A kind of processing method and processing device of video data | |
| US11979544B2 (en) | 360 video transmission method, 360 video reception method, 360 video transmission apparatus, and 360 video reception apparatus | |
| CN104469398B (en) | A kind of Internet video picture processing method and device | |
| CN116912385B (en) | Video frame adaptive rendering processing method, computer device and storage medium | |
| CN106254779A (en) | A kind of panoramic video processing method and server and client side | |
| CN116456090A (en) | Video compression method based on target detection algorithm | |
| US20240119660A1 (en) | Methods for transmitting and rendering a 3d scene, method for generating patches, and corresponding devices and computer programs | |
| CN114930812B (en) | Method and apparatus for decoding 3D video | |
| CN111866485A (en) | Stereoscopic picture projection and transmission method, device and computer readable storage medium | |
| Hu et al. | Mobile edge assisted live streaming system for omnidirectional video | |
| CN117115326A (en) | Rendering method and device of three-dimensional scene model and distributed rendering server | |
| CN111212267A (en) | Partitioning method of panoramic image and server | |
| KR102183895B1 (en) | Indexing of tiles for region of interest in virtual reality video streaming | |
| CN108737820B (en) | Panoramic video compression method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20201030 |