CN115529474A - Panoramic video transmission method and device, electronic equipment and storage medium - Google Patents
Panoramic video transmission method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN115529474A CN115529474A CN202110704178.4A CN202110704178A CN115529474A CN 115529474 A CN115529474 A CN 115529474A CN 202110704178 A CN202110704178 A CN 202110704178A CN 115529474 A CN115529474 A CN 115529474A
- Authority
- CN
- China
- Prior art keywords
- video
- code stream
- key frame
- panoramic video
- frame interval
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 76
- 230000005540 biological transmission Effects 0.000 title claims abstract description 23
- 230000000007 visual effect Effects 0.000 claims abstract description 20
- 238000004891 communication Methods 0.000 claims description 19
- 230000008569 process Effects 0.000 claims description 18
- 238000004590 computer program Methods 0.000 claims description 9
- 239000002131 composite material Substances 0.000 claims description 5
- 230000008859 change Effects 0.000 abstract description 14
- 238000010586 diagram Methods 0.000 description 8
- 239000012634 fragment Substances 0.000 description 7
- 230000009471 action Effects 0.000 description 3
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 239000002151 riboflavin Substances 0.000 description 2
- 239000004149 tartrazine Substances 0.000 description 2
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000003111 delayed effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- H04N21/23418—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/136—Incoming video signal characteristics or properties
- H04N19/137—Motion inside a coding unit, e.g. average field, frame or block difference
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The application discloses transmission, device, electronic equipment and storage medium of panorama video. Wherein, the method comprises the following steps: coding a panoramic video to be played in a client according to a first key frame interval to obtain a first video code stream; sending a first video code stream to a client; under the condition that a first switching occurs in a main visual angle area displayed by the panoramic video, a second video code stream obtained by coding the panoramic video according to a second key frame interval is obtained; and sending the second video code stream to the client. The method and the device can be used for coding the panoramic video at two key frame intervals at the server side. When the main visual angle area of the client is not changed, the server transmits a video code stream of a first key frame interval; when the main view area of the client is obtained to change, the server sends a video code stream which is encoded according to the second key frame interval in advance to the client, so that switching delay is avoided when the main view is switched.
Description
Technical Field
The present application relates to the field of video coding, and in particular, to a method and an apparatus for transmitting a panoramic video, an electronic device, and a storage medium.
Background
In most video transmission schemes in the related art, before transmission, a server usually adopts an Omaf-dash protocol to encode a video to be transmitted, each frame of image of the video is divided into a plurality of fragments in a space domain, then the server selects the fragments in a corresponding region according to a main visual angle range fed back by a player end to perform high-definition streaming transmission, and the fragments outside the main visual angle are transmitted by adopting a low-definition streaming scheme.
However, the main problem existing in the prior art is that a low-definition stream transmission mode is uniformly adopted for blocks outside the main view angle area, once the main view angle of the client changes, the high-definition stream of the corresponding fragments needs to be pulled from the server again, decoded, rendered and played again, and a certain high-definition and low-definition switching delay is generated.
Disclosure of Invention
In order to solve the technical problems or at least partially solve the technical problems, the application provides a panoramic video transmission method, a panoramic video transmission device, an electronic device and a storage medium.
According to an aspect of an embodiment of the present application, there is provided a method for transmitting a panoramic video, including:
coding a panoramic video to be played in a client according to a first key frame interval to obtain a first video code stream;
sending the first video code stream to the client;
under the condition that first switching occurs in a main view angle area displayed by the panoramic video, acquiring a second video code stream obtained by coding the panoramic video according to a second key frame interval, wherein the second key frame interval is smaller than the first key frame interval;
and sending the second video code stream to the client.
Further, before the first switching occurs in the main view angle area of the panoramic video display, the method further comprises:
acquiring the frame number of key frames in the panoramic video;
generating video packets with the same number as the number of the key frames;
and sequentially taking the key frames as the first frames of the video packets, and encoding forward predicted frames associated with the key frames into corresponding video packets to obtain a second video code stream of the second key frame interval.
Further, after sending the second video bitstream to the client, the method further includes:
determining whether a second switching occurs in a main visual angle area of the panoramic video display within a time window;
and under the condition that the second switching is not performed in the main view angle area displayed by the panoramic video in the time window, sending the second video code stream to the client in the time window, and sending a third video code stream with a third key frame interval to the client after the time window, wherein the third key frame interval is larger than the second key frame interval or equal to the first key frame interval.
Further, after detecting whether the second switching occurs in the main view angle area of the panoramic video display within the time window, the method further includes:
under the condition that the second switching is detected to occur in the main view angle area of the panoramic video display in the time window, determining the time of the second switching to be the first starting time of the time window;
and starting from the first starting time, repeatedly executing the step of detecting whether the main visual angle area of the panoramic video display is switched or not in a time window.
Further, in a case where the first switching occurs in a main view angle area of the panoramic video display, the method further includes:
coding a video code stream starting from a first end time of a time window in the panoramic video according to the first key frame interval to obtain a fourth video code stream, wherein the second video code stream is sent to the client in the time window;
and sending the fourth video code stream to the client from the first end time.
Further, encoding a video code stream starting from a first end time of a time window in the panoramic video according to the first key frame interval to obtain a fourth video code stream includes:
acquiring a first key frame in a video code stream starting from a first end time of a time window in the panoramic video;
and taking the first key frame as the first frame in the video code stream starting from the first end time of the time window in the panoramic video, and coding the key frame behind the first key frame according to the first key frame interval to obtain the fourth video code stream.
Further, in a case that a third switch is detected to occur in a main view angle area displayed by the panoramic video in a process of encoding a video code stream starting from a first end time of a time window in the panoramic video according to the first key frame interval, the method further includes:
determining a second end time of the time window by taking the time of the third switching as a second starting time of the time window;
coding the video code stream starting from the second ending time according to the first key frame interval to obtain a fifth video code stream;
and sending the fifth video code stream to the client from the second end time.
According to another aspect of the embodiments of the present application, there is also provided a method for transmitting a panoramic video, including:
receiving a first video code stream corresponding to a panoramic video sent by a composite server, wherein the first video code stream is obtained by encoding the panoramic video according to a first key frame interval by the server;
playing the panoramic video according to the first video code stream;
in the playing process, under the condition that a first switch is determined to occur in a main view angle area displayed by the panoramic video, receiving a second video code stream of the panoramic video sent by the server, wherein the second video code stream is obtained by the server coding the panoramic video according to a second key frame interval, and the second key frame interval is smaller than the first key frame interval;
and playing the panoramic video according to the second video code stream.
According to another aspect of the embodiments of the present application, there is also provided a transmission apparatus for panoramic video, including:
the first sending module is used for sending a first video code stream of a first key frame interval of the panoramic video to the client;
the acquisition module is used for acquiring a second video code stream of a second key frame interval of the panoramic video under the condition that first switching occurs in a main view angle area displayed by the panoramic video, wherein the second key frame interval is smaller than the first key frame interval;
and the second sending module is used for sending the second video code stream to the client.
According to another aspect of the embodiments of the present application, there is also provided a transmission apparatus for panoramic video, including:
the system comprises a first receiving module, a second receiving module and a third receiving module, wherein the first receiving module is used for receiving a first video code stream corresponding to a panoramic video sent by a composite server, and the first video code stream is obtained by the server by encoding the panoramic video according to a first key frame interval;
the first display module is used for playing the panoramic video according to the first video code stream;
a second receiving module, configured to receive a second video code stream of the panoramic video sent by the server when it is determined that a first switch occurs in a main view area of the panoramic video during a playing process, where the second video code stream is obtained by the server encoding the panoramic video according to a second key frame interval, and the second key frame interval is smaller than the first key frame interval;
and the second display module is used for playing the panoramic video according to the second video code stream.
According to another aspect of the embodiments of the present application, there is also provided a storage medium including a stored program that executes the above steps when the program is executed.
According to another aspect of the embodiments of the present application, there is also provided an electronic apparatus, including a processor, a communication interface, a memory, and a communication bus, where the processor, the communication interface, and the memory complete communication with each other through the communication bus; wherein: a memory for storing a computer program; a processor for executing the steps of the method by running the program stored in the memory.
Embodiments of the present application also provide a computer program product containing instructions, which when run on a computer, cause the computer to perform the steps of the above method.
Compared with the prior art, the technical scheme provided by the embodiment of the application has the following advantages: the method and the device can be used for coding the panoramic video at two key frame intervals at the server side. Under the condition that the main view angle area of the client is not changed, the server transmits a video code stream of a first key frame interval; under the condition that the main view angle area of the client is obtained to be changed, the server sends a video code stream which is coded in advance according to the second key frame interval to the client in the time window, so that the switching delay of the main view angle is avoided after the main view angle is switched.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and, together with the description, serve to explain the principles of the application.
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious for those skilled in the art to obtain other drawings without inventive labor.
Fig. 1 is a flowchart of a method for transmitting a panoramic video according to an embodiment of the present disclosure;
fig. 2 is a schematic diagram of a video encoding technique according to an embodiment of the present application;
fig. 3 is a schematic view of a perspective transmission according to an embodiment of the present disclosure;
fig. 4 is a schematic diagram of a video encoding technique according to an embodiment of the present application;
fig. 5 is a block diagram of a method for transmitting panoramic video according to another embodiment of the present application;
fig. 6 is a block diagram of a device for transmitting panoramic video according to an embodiment of the present disclosure;
fig. 7 is a block diagram of a device for transmitting panoramic video according to another embodiment of the present application;
fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely with reference to the drawings in the embodiments of the present application, it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments, and the illustrative embodiments and descriptions thereof of the present application are used for explaining the present application and do not constitute a limitation to the present application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
It is noted that, in this document, relational terms such as "first" and "second," and the like, may be used solely to distinguish one entity or action from another similar entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrases "comprising a," "8230," "8230," or "comprising" does not exclude the presence of additional like elements in a process, method, article, or apparatus that comprises the element.
The embodiment of the application provides a panoramic video transmission method and device, electronic equipment and a storage medium. The method provided by the embodiment of the present invention may be applied to any required electronic device, for example, the electronic device may be an electronic device such as a server and a terminal, and the method is not specifically limited herein, and for convenience of description, the method is hereinafter simply referred to as an electronic device.
According to an aspect of embodiments of the present application, a method embodiment of a method for transmitting a panoramic video is provided. Fig. 1 is a flowchart of a method for transmitting a panoramic video according to an embodiment of the present application, where as shown in fig. 1, the method includes:
and S11, coding the panoramic video to be played in the client according to the first key frame interval to obtain a first video code stream.
And step S12, sending the first video code stream to the client.
In the embodiment of the present application, a process of sending a first video coding stream of a first key frame interval of a panoramic video to a client is as follows:
first, a key frame sequence of the panoramic video is obtained, and a key frame and a predictive coding frame in the key frame sequence are extracted, where the key frame in the embodiment of the present application is an I frame and the predictive coding frame is a P frame, where the I frame can be understood as an intra picture, and the I frame is usually the first frame of each GOP (a video compression technique used by MPEG), and is moderately compressed to serve as a reference point for random access, which can be used as an image. In the course of MPEG coding, a part of video frame sequence is compressed into I frame; the parts are compressed into P frames.
A first number of first video packets and a second number of second video packets are then generated based on the number of key frames, where the first number is the same as the number of key frames and the second number is greater than or equal to the first number. For example: if the number of the key frames in the panoramic video is 10, 10 first video packets and 10 second video packets are generated.
In the encoding process, key frames are sequentially inserted into a first video packet according to a playing sequence and serve as a first frame of the first video packet, predictive coding frames associated with the key frames are respectively inserted into the first video packet and a second video packet, the first video packet and the second video packet are arranged alternately, and finally a first video coding stream of a first key frame interval is obtained.
As an example, as shown in fig. 2, the first video encoded stream of the first key frame interval includes a plurality of first video packets and a plurality of second video packets, where a first frame in the first video packet is an I frame, the remaining video frames are P frames, and all the video frames in the second video packets are P frames.
Step S13, under the condition that the first switching occurs in the main visual angle area displayed by the panoramic video, acquiring a second video code stream obtained by coding the panoramic video according to a second key frame interval, wherein the second key frame interval is smaller than the first key frame interval;
in the embodiment of the application, the process of detecting that the first switching occurs in the main view angle area displayed by the panoramic video comprises the following steps:
in the embodiment of the application, a video to be transmitted is encoded according to an Omaf-dash protocol, each frame of image of the video is divided into a plurality of fragments in a space domain (refer to fig. 3), then the fragments corresponding to a main view angle region fed back by a client are determined, and if the fragments corresponding to the main view angle region change, the main view angle region displayed by the panoramic video is determined to change.
In this embodiment of the present application, the process of detecting that the first switching occurs in the main view angle area of the panoramic video display may further include the following steps A1 to A4:
a1, acquiring at least two continuous video frames of a panoramic video currently played at a client;
in the embodiment of the application, in order to periodically determine whether the panoramic video has a view angle change in the playing process, the video frames of the panoramic video are monitored in the server according to the time period, and at least two video frames of the current panoramic video in the playing process are obtained.
Step A2, determining a main visual angle area in each video frame;
step A3, calculating the similarity between the view angle areas in every two adjacent video frames;
in the embodiment of the application, the characteristic information of each video frame in the current time period is obtained; and calculating the similarity between every two adjacent video frames according to the characteristic information.
And step A4, determining whether the panoramic video is switched or not based on the similarity between the main view angle areas in every two adjacent video frames, wherein when the similarity between the main view angle areas in every two adjacent video frames is greater than the preset similarity, the main view angle areas are not switched, or when the similarity between the main view angle areas in every two adjacent video frames is less than the preset similarity, the main view angle areas are switched.
In the embodiment of the present application, acquiring a second video code stream of a second key frame interval of a panoramic video includes the following steps B1 to B3:
step B1, acquiring the frame number of key frames in the panoramic video;
in the embodiment of the application, the key frame sequence in the unplayed video clip is obtained, and the key frame and the predictive coding frame in the key frame sequence are extracted, so that the frame number of the key frame is obtained.
B2, generating video packets with the same number as the number of the key frames;
and step B3, sequentially taking the key frames as the first frames of the video packets, and encoding the forward predicted frames associated with the key frames into the corresponding video packets to obtain a second video code stream at a second key frame interval.
In the embodiment of the application, video packets with corresponding number are generated according to the frame number of the key video frame; and correspondingly encoding the key video frames and the preset encoding frames into each video packet, thereby obtaining a second video code stream with a second key frame interval.
As an example, as shown in fig. 4, key frames are coded into each video packet in sequence and serve as a first frame of each video packet, and then predictive coding frames corresponding to each key frame are coded into the video packet, so as to finally obtain a second video code stream at a second key frame interval.
In the prior art, the inventor also finds that by encoding the panoramic video by adopting a fixed I-frame interval encoding scheme, the video packets of the time period can be downloaded and decoded immediately when the view angle is switched. If the decoded video packet is a video packet with a full P frame, I frames referred to in the previous video packet need to be decoded backwards, which may result in a long decoding time and a large high-low switching delay. Each mp4 file contains an I-frame, which increases the amount of data actually transmitted, resulting in a waste of bandwidth.
Therefore, the video code stream with larger key frame interval is transmitted under the condition that the main visual angle does not send the switching, so that the data transmission quantity is reduced, the video code stream with smaller key frame interval is transmitted under the condition that the main visual angle is switched, the switching delay condition is avoided, and meanwhile, the bandwidth is saved on the premise of ensuring the original performance.
And step S14, sending the second video code stream to the client.
In this embodiment of the present application, in step S13, after sending the second video stream to the client, the method further includes the following steps C1-C2:
step C1, determining whether a second switching occurs in a main visual angle area displayed by the panoramic video in a time window;
and step C2, under the condition that the second switching does not occur in the main visual angle area displayed by the panoramic video in the time window, sending a second video code stream to the client in the time window, and sending a third video code stream with a third key frame interval to the client after the time window, wherein the third key frame interval is greater than the second key frame interval or equal to the first key frame interval.
In the embodiment of the application, the time window is a preset time interval for judging the switching of the view angle, and if it is not detected in the time window that whether the second switching occurs in the main view angle area displayed by the panoramic video, it is determined that the main view angle of the panoramic video cannot be switched within a certain time in the future, so that a second video code stream, namely a video code stream with a smaller key frame interval, is sent in the time window. And then sending a third video code stream of the first key frame interval to the client after the time window, wherein the third video code stream is obtained by encoding the video code stream starting from the end time of the time window in the panoramic video according to the first key frame interval.
In this embodiment of the present application, after detecting whether the second switching occurs in the main view angle region of the panoramic video display within the time window, the method further includes the following steps C3-C4:
step C3, under the condition that the second switching of the main view angle area displayed by the panoramic video in the time window is detected, determining the time of the second switching as the first starting time of the time window;
and C4, from the first starting time, repeatedly executing the step of detecting whether the main visual angle area displayed by the panoramic video is switched or not in the time window.
In the embodiment of the present application, the time window occurs only when the switching of the main view angle area of the panoramic video is detected. On one hand, whether the main view angle area displayed by the panoramic video is continuously switched in a time window can be detected in the embodiment of the application; on the other hand, when the switching of the main view angle area of the panoramic video display in the time window is detected, the time of the second switching is determined as the first starting time of the time window, so that the time window is shifted backwards, and meanwhile, the subsequent panoramic video can be coded according to the second frame interval in the process of shifting backwards the time window.
As an example, the total video duration is 5min, the length of the time window is 5s, when the main view angle area of the panoramic video display is switched at the 30 th time, the time window is 30 s-35 s, if the main view angle area of the panoramic video is switched at the 33 th time within the 30 s-35 s, the starting time of the time window is 33s, and the ending time is 38s.
In the embodiment of the application, in the case that the first switching occurs in the main view angle area of the panoramic video display, the method further comprises the following steps E1-E2:
e1, coding a video code stream starting from a first end time of a time window in the panoramic video according to a first key frame interval to obtain a fourth video code stream, wherein a second video code stream is sent to a client in the time window;
in this embodiment of the present application, in step E1, a video code stream starting from a first end time of a time window in a panoramic video is encoded according to a first key frame interval, so as to obtain a fourth video code stream, including the following steps E101 to E102:
step E101, a first key frame in a video code stream starting from a first end time of a time window in the panoramic video is obtained;
and step E102, taking the first key frame as the first frame in the video code stream starting from the first end time of the time window in the panoramic video, and coding the key frame behind the first key frame according to the first key frame interval to obtain a fourth video code stream.
And E2, sending a fourth video code stream to the client from the first end time.
In this embodiment of the present application, in a process of encoding a video stream starting from a first end time of a time window in a panoramic video according to a first key frame interval, when a third switch is detected to occur in a main view area displayed by the panoramic video, the method further includes the following steps F1 to F3:
step F1, taking the time of the third switching as a second starting time of the time window, and determining a second ending time of the time window;
step F2, coding the video code stream starting from the second ending time according to the first key frame interval to obtain a fifth video code stream;
and F3, starting from the second end time, sending a fifth video code stream to the client.
It should be noted that, in the existing panoramic video transmission scheme, a part of a corresponding area is selected from the server side according to a main view area fed back by the player side to perform high definition stream transmission, and a part outside the main view is transmitted by using a low definition stream scheme. The inventor finds that, in the prior art, a low-definition stream transmission mode is uniformly adopted outside a main view angle area, once the main view angle area changes, a client needs to pull a corresponding high-definition stream after the corresponding view angle changes from a server side again, and then the client needs to decode, render and play again, so that a certain high-definition and low-definition switching delay is generated.
According to the method and the device, after the server finds that the main view angle area of the client is switched, the server sends the video code stream of the second key frame interval to the client in the time window, and meanwhile, the server encodes the video code stream starting from the second ending time according to the first key frame interval in the time window. If the main view angle area of the client is not changed in the time window, the server sends the video code stream of the first key frame interval to the client when the time window is ended.
If the main view area of the client changes in the time window, the server takes the time when the main view sends the change as the starting time of the time window, i.e. the time window moves backwards smoothly.
Compared with the prior art, the method and the device can be used for coding the panoramic video at two key frame intervals at the server side. Under the condition that the main view angle area of the client is not changed, the server transmits a video code stream of a first key frame interval; under the condition that the main view angle area of the client is obtained to be changed, the server sends a video code stream which is coded according to the second key frame interval in advance to the client in the time window, so that switching delay of the main view angle after switching is avoided, and the problem that switching delay of the main view angle area is required to be delayed in the prior art is solved.
In addition, if the main view angle area is not changed in the time window, the server transmits and transmits the video code stream with the first key frame interval to the client after the time window is finished, and the first key frame interval is larger than the second key frame interval, so that the size of the video code stream transmitted by the server under the condition that the main view angle area is not changed is reduced, and the bandwidth is saved.
As an example, a time window t for judging the switching of the main view angle area is set change When the server does not receive a signal for switching the main view of the client in a time window, the server determines that the main view area is not switched in the time window, and sends a video code stream of a first frame interval to the client. When the server receives a signal of switching the main visual angle of the client in a time window, the server is immediately switched to send a video code stream of a second frame interval to the client, and the video code stream is sent at t change Continuously transmitting the video code stream of the second frame interval in time, and simultaneously transmitting the video code stream of the second frame interval at t change Time is according to the first key frame interval pair t change The video code stream starting from the ending time of the video coding is coded.
At t change Within the time, if a signal that the main view angle area of the client is switched is not received any more, at t change After finishing, sending the data to the client at t change And coding the video code stream according to the first key frame interval. If t is change Still receiving the signal of switching the main visual angle area of the client, and updating t change And continuously transmitting the video code stream of the second frame interval.
As shown in fig. 5, an embodiment of the present application further provides a transmission method of a panoramic video, which is applied to a client, and the method includes:
step S21, receiving a first video code stream corresponding to the panoramic video sent by the composite server, wherein the first video code stream is obtained by the server coding the panoramic video according to a first key frame interval;
s22, playing the panoramic video according to the first video code stream;
step S23, in the playing process, under the condition that the first switching of the main visual angle area displayed by the panoramic video is determined, receiving a second video code stream of the panoramic video sent by the server, wherein the second video code stream is obtained by the server coding the panoramic video according to a second key frame interval, and the second key frame interval is smaller than the first key frame interval;
and S24, playing the panoramic video according to the second video code stream.
Fig. 6 is a block diagram of a panoramic video transmission apparatus provided in an embodiment of the present application, where the apparatus may be implemented as part of or all of an electronic device through software, hardware, or a combination of the two. As shown in fig. 6, the apparatus includes:
the encoding module 61 is configured to encode a panoramic video to be played in a client according to a first key frame interval to obtain a first video code stream;
a first sending module 62, configured to send a first video code stream to the client;
an obtaining module 63, configured to obtain a second video code stream obtained by encoding the panoramic video according to a second key frame interval when a first switching occurs in a main view area of the panoramic video display, where the second key frame interval is smaller than the first key frame interval;
and a second sending module 64, configured to send the second video code stream to the client.
In this embodiment of the present application, the obtaining module 63 is configured to obtain a frame number of a key frame in a panoramic video; generating video packets with the same number as the number of the key frames; and sequentially taking the key frames as the first frames of the video packets, and coding the forward predicted frames associated with the key frames into the corresponding video packets to obtain a second video code stream at intervals of a second key frame.
In an embodiment of the present application, the transmitting of the panoramic video further includes: the detection module is used for detecting whether a second switching occurs in a main visual angle area displayed by the panoramic video in a time window; and under the condition that the second switching is not detected in the main view angle area displayed by the panoramic video in the time window, sending a second video code stream to the client in the time window, and sending a third video code stream with the first key frame interval to the client after the time window.
In this embodiment of the present application, the apparatus for transmitting a panoramic video further includes: the first coding module is used for determining the time of second switching as the first starting time of the time window under the condition that the second switching of the main view angle area of the panoramic video display in the time window is detected; and starting from the first starting time, repeatedly executing the step of detecting whether the main view angle area displayed by the panoramic video is switched or not in the time window.
In this embodiment, the apparatus for transmitting panoramic video further includes: a second encoding module comprising:
the coding submodule is used for coding a video code stream starting from a first end time of a time window in the panoramic video according to a first key frame interval to obtain a fourth video code stream, wherein a second video code stream is sent to the client side in the time window;
and the sending submodule is used for sending the fourth video code stream to the client from the first end time.
In the embodiment of the application, the encoding submodule is used for acquiring a first key frame in a video code stream starting from a first end time of a time window in a panoramic video; and taking the first key frame as the first frame in the video code stream starting from the first end time of the time window in the panoramic video, and coding the key frame behind the first key frame according to the first key frame interval to obtain a fourth video code stream.
In this embodiment of the present application, the apparatus for transmitting a panoramic video further includes: the third coding module is used for determining a second ending time of the time window by taking the time of the third switching as a second starting time of the time window; coding the video code stream starting from the second ending time according to the first key frame interval to obtain a fifth video code stream; and sending a fifth video code stream to the client from the second end time.
Fig. 7 is a block diagram of a transmission apparatus for panoramic video according to an embodiment of the present disclosure, which may be implemented as part of or all of an electronic device through software, hardware, or a combination of the two. As shown in fig. 7, the apparatus includes:
the first receiving module 71 is configured to receive a first video code stream corresponding to the panoramic video sent by the composite server, where the first video code stream is obtained by the server encoding the panoramic video according to a first key frame interval;
the first display module 72 is configured to play the panoramic video according to a first video code stream;
a second receiving module 73, configured to receive a second video code stream of the panoramic video sent by the server when it is determined that the first switching occurs in the main view angle area displayed by the panoramic video in the playing process, where the second video code stream is obtained by the server coding the panoramic video according to a second key frame interval, and the second key frame interval is smaller than the first key frame interval;
and a second display module 74, configured to play the panoramic video according to the second video code stream.
An embodiment of the present application further provides an electronic device, as shown in fig. 8, the electronic device may include: the system comprises a processor 1501, a communication interface 1502, a memory 1503 and a communication bus 1504, wherein the processor 1501, the communication interface 1502 and the memory 1503 complete communication with each other through the communication bus 1504.
A memory 1503 for storing a computer program;
the processor 1501 is configured to implement the steps of the above embodiments when executing the computer program stored in the memory 1503.
The communication bus mentioned in the above terminal may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this is not intended to represent only one bus or type of bus.
The communication interface is used for communication between the terminal and other equipment.
The Memory may include a Random Access Memory (RAM) or a non-volatile Memory (non-volatile Memory), such as at least one disk Memory. Alternatively, the memory may be at least one memory device located remotely from the processor.
The Processor may be a general-purpose Processor, and includes a Central Processing Unit (CPU), a Network Processor (NP), and the like; the Integrated Circuit may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, or discrete hardware components.
In yet another embodiment provided by the present application, there is further provided a computer-readable storage medium having stored therein instructions, which when run on a computer, cause the computer to execute the transmission method of panoramic video described in any of the above embodiments.
In yet another embodiment provided by the present application, there is also provided a computer program product containing instructions which, when run on a computer, cause the computer to perform the method for transmitting panoramic video according to any of the above embodiments.
In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. The procedures or functions described in accordance with the embodiments of the application are all or partially generated when the computer program instructions are loaded and executed on a computer. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (e.g., coaxial cable, fiber optic, digital subscriber line) or wirelessly (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that includes one or more available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid State Disk), among others.
The above description is only for the preferred embodiment of the present application, and is not intended to limit the scope of the present application. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application are included in the protection scope of the present application.
The previous description is only an example of the present application, and is provided to enable any person skilled in the art to understand or implement the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (12)
1. A method for transmitting panoramic video, comprising:
coding a panoramic video to be played in a client according to a first key frame interval to obtain a first video code stream;
sending the first video code stream to the client;
under the condition that first switching occurs in a main view angle area displayed by the panoramic video, acquiring a second video code stream obtained by coding the panoramic video according to a second key frame interval, wherein the second key frame interval is smaller than the first key frame interval;
and sending the second video code stream to the client.
2. The method of claim 1, wherein prior to the first switch occurring in a primary view area of the panoramic video display, the method further comprises:
acquiring the frame number of key frames in the panoramic video;
generating video packets with the same number as the number of the key frames;
and sequentially using the key frames as the first frames of the video packets, and encoding forward predicted frames associated with the key frames into corresponding video packets to obtain a second video code stream at intervals of the second key frames.
3. The method of claim 1, wherein after sending the second video bitstream to the client, the method further comprises:
determining whether a second switching occurs in a main visual angle area of the panoramic video display within a time window;
and under the condition that the second switching is not performed in the main view angle area displayed by the panoramic video in the time window, sending the second video code stream to the client in the time window, and sending a third video code stream with a third key frame interval to the client after the time window, wherein the third key frame interval is larger than the second key frame interval or equal to the first key frame interval.
4. The method of claim 3, wherein after detecting whether the second switch occurs in a primary view region of the panoramic video display within a time window, the method further comprises:
under the condition that the second switching of the main visual angle area of the panoramic video display in the time window is detected, determining the time of the second switching as the first starting time of the time window;
and starting from the first starting time, repeatedly executing the step of detecting whether the main visual angle area of the panoramic video display is switched or not in a time window.
5. The method of claim 1, wherein if the first switch occurs in a primary view area of the panoramic video display, the method further comprises:
coding a video code stream starting from a first end time of a time window in the panoramic video according to the first key frame interval to obtain a fourth video code stream, wherein the second video code stream is sent to the client in the time window;
and sending the fourth video code stream to the client from the first end time.
6. The method of claim 5, wherein encoding the video stream starting from the first end time of the time window in the panoramic video according to the first keyframe interval to obtain a fourth video stream comprises:
acquiring a first key frame in a video code stream starting from a first end time of a time window in the panoramic video;
and taking the first key frame as a first frame in a video code stream starting from a first end time of a time window in the panoramic video, and coding key frames behind the first key frame according to the first key frame interval to obtain the fourth video code stream.
7. The method of claim 5, wherein in the event that a third switch of a main view region of the panoramic video display is detected during encoding of a video bitstream of the panoramic video starting from a first end time of a time window according to the first key frame interval, the method further comprises:
determining a second end time of the time window by taking the time of the third switching as a second starting time of the time window;
coding the video code stream starting from the second ending time according to the first key frame interval to obtain a fifth video code stream;
and sending the fifth video code stream to the client from the second end time.
8. A method for transmitting panoramic video, comprising:
receiving a first video code stream corresponding to a panoramic video sent by a multi-server, wherein the first video code stream is obtained by the server coding the panoramic video according to a first key frame interval;
playing the panoramic video according to the first video code stream;
in the playing process, receiving a second video code stream of the panoramic video sent by the server under the condition that a first switch is determined to occur in a main view angle area displayed by the panoramic video, wherein the second video code stream is obtained by the server by encoding the panoramic video according to a second key frame interval, and the second key frame interval is smaller than the first key frame interval;
and playing the panoramic video according to the second video code stream.
9. A transmission apparatus of a panoramic video, comprising:
the coding module is used for coding the panoramic video to be played in the client according to the first key frame interval to obtain a first video code stream;
the first sending module is used for sending the first video code stream to the client;
the acquisition module is used for acquiring a second video code stream obtained by coding the panoramic video according to a second key frame interval under the condition that the first switching occurs in a main view angle area displayed by the panoramic video, wherein the second key frame interval is smaller than the first key frame interval;
and the second sending module is used for sending the second video code stream to the client.
10. A transmission apparatus of a panoramic video, comprising:
the system comprises a first receiving module, a second receiving module and a third receiving module, wherein the first receiving module is used for receiving a first video code stream corresponding to a panoramic video sent by a composite server, and the first video code stream is obtained by the server by encoding the panoramic video according to a first key frame interval;
the first display module is used for playing the panoramic video according to the first video code stream;
a second receiving module, configured to receive a second video code stream of the panoramic video sent by the server when it is determined that a first switch occurs in a main view area of the panoramic video during a playing process, where the second video code stream is obtained by the server encoding the panoramic video according to a second key frame interval, and the second key frame interval is smaller than the first key frame interval;
and the second display module is used for playing the panoramic video according to the second video code stream.
11. A storage medium, characterized in that the storage medium comprises a stored program, wherein the program when executed performs the method steps of any of the preceding claims 1 to 8.
12. An electronic device is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor and the communication interface are used for realizing mutual communication by the memory through the communication bus; wherein:
a memory for storing a computer program;
a processor for performing the method steps of any one of claims 1-8 by executing a program stored on a memory.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110704178.4A CN115529474A (en) | 2021-06-24 | 2021-06-24 | Panoramic video transmission method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110704178.4A CN115529474A (en) | 2021-06-24 | 2021-06-24 | Panoramic video transmission method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115529474A true CN115529474A (en) | 2022-12-27 |
Family
ID=84694297
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110704178.4A Pending CN115529474A (en) | 2021-06-24 | 2021-06-24 | Panoramic video transmission method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115529474A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101729898A (en) * | 2009-11-16 | 2010-06-09 | 中国人民解放军国防科学技术大学 | Video coding and decoding methods and devices |
| CN106937141A (en) * | 2017-03-24 | 2017-07-07 | 北京奇艺世纪科技有限公司 | A kind of bitstreams switching method and device |
| CN112040233A (en) * | 2020-11-04 | 2020-12-04 | 北京金山云网络技术有限公司 | Video encoding method, video decoding method, video encoding device, video decoding device, electronic device, and storage medium |
| CN112165653A (en) * | 2020-09-28 | 2021-01-01 | 中国建设银行股份有限公司 | Video playing method, device and equipment |
| WO2021082299A1 (en) * | 2019-11-01 | 2021-05-06 | 杭州海康威视系统技术有限公司 | Video playback method and device |
-
2021
- 2021-06-24 CN CN202110704178.4A patent/CN115529474A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101729898A (en) * | 2009-11-16 | 2010-06-09 | 中国人民解放军国防科学技术大学 | Video coding and decoding methods and devices |
| CN106937141A (en) * | 2017-03-24 | 2017-07-07 | 北京奇艺世纪科技有限公司 | A kind of bitstreams switching method and device |
| WO2021082299A1 (en) * | 2019-11-01 | 2021-05-06 | 杭州海康威视系统技术有限公司 | Video playback method and device |
| CN112165653A (en) * | 2020-09-28 | 2021-01-01 | 中国建设银行股份有限公司 | Video playing method, device and equipment |
| CN112040233A (en) * | 2020-11-04 | 2020-12-04 | 北京金山云网络技术有限公司 | Video encoding method, video decoding method, video encoding device, video decoding device, electronic device, and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2016131223A1 (en) | Frame loss method for video frame and video sending apparatus | |
| CN110784740A (en) | Video processing method, device, server and readable storage medium | |
| WO2021147448A1 (en) | Video data processing method and apparatus, and storage medium | |
| CN110012251B (en) | Video recording method, device and readable storage medium | |
| CN109391585B (en) | Video data processing method, device, terminal and computer readable storage medium | |
| EP3493547B1 (en) | Video streaming delivery | |
| CN111669619B (en) | Video stream data switching method, device, terminal and readable storage medium | |
| CN106998485B (en) | Video live broadcasting method and device | |
| US11350104B2 (en) | Method for processing a set of images of a video sequence | |
| CN104394426A (en) | Rate playing method and device of stream media | |
| CN110519640B (en) | Video processing method, encoder, CDN server, decoder, device, and medium | |
| US11563962B2 (en) | Seamless content encoding and transmission | |
| CN104918123A (en) | Method and system for playback of motion video | |
| CN111836076A (en) | Video resolution switching method and device and electronic equipment | |
| WO2021057697A1 (en) | Video encoding and decoding methods and apparatuses, storage medium, and electronic device | |
| CN111447451A (en) | Image coding and decoding method and device | |
| CN115529474A (en) | Panoramic video transmission method and device, electronic equipment and storage medium | |
| CN110798713B (en) | Time-shifted television on-demand method, terminal, server and system | |
| CN110062003B (en) | Video data transmitting method, video data transmitting device, electronic equipment and storage medium | |
| KR20060043050A (en) | Method for encoding and decoding video signal | |
| CN108933762B (en) | Media stream playing processing method and device | |
| CN115695918B (en) | Multi-camera broadcast guide control method and device, readable storage medium and terminal equipment | |
| KR20130141368A (en) | Reception device and program for reception device | |
| CN114615549B (en) | Streaming media seek method, client, storage medium and mobile device | |
| US12034952B2 (en) | Seamless content encoding and transmission |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |