CN113691797A - Video playing processing method, device, equipment and storage medium - Google Patents
Video playing processing method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN113691797A CN113691797A CN202110994334.5A CN202110994334A CN113691797A CN 113691797 A CN113691797 A CN 113691797A CN 202110994334 A CN202110994334 A CN 202110994334A CN 113691797 A CN113691797 A CN 113691797A
- Authority
- CN
- China
- Prior art keywords
- video stream
- frame
- definition video
- current
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 51
- 239000012634 fragment Substances 0.000 claims abstract description 67
- 230000000007 visual effect Effects 0.000 claims abstract description 15
- 238000000034 method Methods 0.000 claims description 26
- 238000004891 communication Methods 0.000 claims description 17
- 238000013467 fragmentation Methods 0.000 description 6
- 238000006062 fragmentation reaction Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 4
- 230000005484 gravity Effects 0.000 description 3
- 238000004590 computer program Methods 0.000 description 2
- 206010047513 Vision blurred Diseases 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 208000002173 dizziness Diseases 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000011022 operating instruction Methods 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/30—Image reproducers
- H04N13/332—Displays for viewing with the aid of special glasses or head-mounted displays [HMD]
- H04N13/344—Displays for viewing with the aid of special glasses or head-mounted displays [HMD] with head-mounted left-right displays
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/30—Image reproducers
- H04N13/366—Image reproducers using viewer tracking
- H04N13/378—Image reproducers using viewer tracking for tracking rotational head movements around an axis perpendicular to the screen
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/442—Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
- H04N21/44213—Monitoring of end-user related data
- H04N21/44218—Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/81—Monomedia components thereof
- H04N21/816—Monomedia components thereof involving special video data, e.g 3D video
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
- H04N21/845—Structuring of content, e.g. decomposing content into time segments
- H04N21/8456—Structuring of content, e.g. decomposing content into time segments by decomposing the content in the time domain, e.g. in time segments
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Social Psychology (AREA)
- Computer Networks & Wireless Communication (AREA)
- Databases & Information Systems (AREA)
- Television Signal Processing For Recording (AREA)
Abstract
The invention discloses a video playing processing method, a device, equipment and a storage medium, wherein the video playing processing method comprises the following steps: determining the current rotating speed of the head-mounted display equipment, and the current playing frame and the current playing video stream in the current visual angle according to the current operating data of the head-mounted display equipment; pulling a target fragment of a target high-definition video stream from the multi-path high-definition video stream for display according to the current rotating speed, the current playing frame and the current playing video stream; the I frame frequencies of the multiple paths of high-definition video streams are different, the video contents are the same, the I frame frequency of each path of high-definition video stream is fixed, each path of high-definition video stream comprises a plurality of fragments, and the first frame of each fragment is an I frame. By pulling the target fragment of the target high-definition video stream from the multiple paths of high-definition video streams with different I frame frequencies for display, the target high-definition video stream can be automatically selected, the bandwidth can be dynamically adjusted, and the requirements of low delay and low bandwidth are met.
Description
Technical Field
The present invention relates to the field of communications technologies, and in particular, to a method, an apparatus, a device, and a storage medium for video playback processing.
Background
At present, a field of view (FOV) -based Virtual Reality (VR) player adopts a panoramic low-definition (4K or 2K) video stream + a local high-definition (8K) video stream scheme within a view angle as shown in fig. 1.
The method for processing the panoramic low-definition video stream comprises the following steps: the panoramic 360-degree video space domain is not divided, the time domain is continuously downloaded, and the user can see the picture no matter how to rotate the head display, namely, rotate any angle. The processing method for the high-definition video stream in the view angle comprises the following steps: dividing a panoramic 360-degree video space domain into a plurality of fragments, downloading only the fragments in a visual angle in a time domain, calculating to obtain corresponding fragments according to the visual angle of a user after the user rotates a head display, and downloading the corresponding fragments; the fragment file starts with an I frame, each frame contains the same number, the interval of the I frame is fixed during video coding, and the processing mode has the characteristics of high video quality, more I frames, high bandwidth and slow decoding.
In the prior art, one path of high definition video is usually adopted, on one hand, in order to ensure that a user can see a high definition image as soon as possible after rotating a head display, high definition video fragmentation (time domain) cannot be too large, and in order to decode in time, the frequency of an I frame of the fragmentation cannot be too low, but the too high frequency of the I frame causes a large bandwidth and high power consumption of equipment. On the other hand, in order to reduce the bandwidth, the I frame frequency may be appropriately reduced, but if the I frame frequency is too low, the decoding latency is long, the user feels blurred vision, the low definition and high definition switching delay is high, and the user is easily dizzy. Moreover, if the user does not rotate the head display for a long time, i.e. there is no view switching, the I frame fragment is still downloaded at this time, resulting in waste of bandwidth and resources.
Therefore, the prior art solutions do not fulfill the requirements of low delay and low bandwidth.
Disclosure of Invention
In view of the above problems, embodiments of the present invention are proposed to provide a video playback processing method, apparatus, device, and storage medium that overcome or at least partially solve the above problems.
According to an aspect of the embodiments of the present invention, there is provided a video playback processing method, including:
determining the current rotating speed of the head-mounted display equipment, and the current playing frame and the current playing video stream in the current visual angle according to the current operating data of the head-mounted display equipment; the currently played video stream is one of the multiple paths of high-definition video streams;
pulling a target fragment of a target high-definition video stream from the multi-path high-definition video stream for display according to the current rotating speed, the current playing frame and the current playing video stream; the I frame frequencies of the multiple paths of high-definition video streams are different, the video contents are the same, the I frame frequency of each path of high-definition video stream is fixed, each path of high-definition video stream comprises a plurality of fragments, and the first frame of each fragment is an I frame.
According to another aspect of the embodiments of the present invention, there is provided a video playback processing apparatus including:
the determining module is used for determining the current rotating speed of the head-mounted display equipment, and the current playing frame and the current playing video stream in the current visual angle according to the current operating data of the head-mounted display equipment; the currently played video stream is one of the multiple paths of high-definition video streams;
the display module is used for pulling the target fragments of the target high-definition video stream from the multi-path high-definition video stream to display according to the current rotating speed, the current playing frame and the current playing video stream; the I frame frequencies of the multiple paths of high-definition video streams are different, the video contents are the same, the I frame frequency of each path of high-definition video stream is fixed, each path of high-definition video stream comprises a plurality of fragments, and the first frame of each fragment is an I frame.
According to another aspect of embodiments of the present invention, there is provided a computing device including: the system comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface complete mutual communication through the communication bus;
the memory is used for storing at least one executable instruction, and the executable instruction enables the processor to execute the operation corresponding to the video playing processing method.
According to another aspect of the embodiments of the present invention, there is provided a computer storage medium, in which at least one executable instruction is stored, and the executable instruction causes a processor to perform operations corresponding to the video playing processing method.
According to the scheme provided by the embodiment of the invention, the target fragment of the target high-definition video stream is pulled from the multiple paths of high-definition video streams with different I frame frequencies for display, so that the target high-definition video stream can be automatically selected, the bandwidth can be dynamically adjusted, and the requirements of low delay and low bandwidth are met.
The foregoing description is only an overview of the technical solutions of the embodiments of the present invention, and the embodiments of the present invention can be implemented according to the content of the description in order to make the technical means of the embodiments of the present invention more clearly understood, and the detailed description of the embodiments of the present invention is provided below in order to make the foregoing and other objects, features, and advantages of the embodiments of the present invention more clearly understandable.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the embodiments of the invention. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:
fig. 1 shows a flow chart of a video playing processing method provided by the prior art;
fig. 2 is a flowchart illustrating a video playing processing method according to an embodiment of the present invention;
fig. 3 is a flowchart illustrating a video playing processing method according to a second embodiment of the present invention;
fig. 4 shows an example of low-definition video streams in the video playing processing method provided by the second embodiment of the present invention;
fig. 5 shows an example of a first high definition video stream in a video playing processing method according to a second embodiment of the present invention;
fig. 6 shows an example of a second path of high definition video stream in the video playing processing method according to the second embodiment of the present invention;
fig. 7 shows an example of a third high-definition video stream in the video playing processing method provided by the second embodiment of the present invention;
fig. 8 shows a first example of pull slices in the video playing processing method according to the second embodiment of the present invention;
fig. 9 shows a second example of pull slices in the video playing processing method according to the second embodiment of the present invention;
fig. 10 shows a third example of pull slices in the video playing processing method according to the second embodiment of the present invention;
fig. 11 shows a fourth example of pull slices in the video playing processing method according to the second embodiment of the present invention;
fig. 12 shows a fifth example of pull slices in the video playing processing method according to the second embodiment of the present invention;
fig. 13 shows a sixth example of pull slices in the video playing processing method according to the second embodiment of the present invention;
fig. 14 shows a seventh example of pull fragmentation in the video playing processing method according to the second embodiment of the present invention;
fig. 15 shows an eighth example of pull slices in the video playing processing method according to the second embodiment of the present invention;
fig. 16 shows a ninth pull fragment example in the video playing processing method according to the second embodiment of the present invention;
fig. 17 shows a tenth example of pull slices in the video playback processing method according to the second embodiment of the present invention;
fig. 18 shows an eleventh example of pull slices in the video playback processing method according to the second embodiment of the present invention;
fig. 19 shows a twelfth example of pull slices in the video playing processing method according to the second embodiment of the present invention;
fig. 20 shows a thirteenth pull slice example in the video playing processing method according to the second embodiment of the present invention;
fig. 21 shows a fourteenth example of pull fragmentation in the video playback processing method according to the second embodiment of the present invention;
fig. 22 shows a fifteenth example of pull slices in the video playing processing method according to the second embodiment of the present invention;
fig. 23 shows a sixteenth pull fragment example in the video playing processing method according to the second embodiment of the present invention;
fig. 24 shows a seventeenth pull slice example in the video playing processing method according to the second embodiment of the present invention;
fig. 25 shows an eighteenth example of pull slices in the video playing processing method according to the second embodiment of the present invention;
fig. 26 shows a nineteenth example of pull fragmentation in the video playback processing method according to the second embodiment of the present invention;
fig. 27 shows a twentieth example of pull fragmentation in the video playback processing method according to the second embodiment of the present invention;
fig. 28 shows a twenty-first pull fragment example in the video playback processing method according to the second embodiment of the present invention;
fig. 29 shows a twenty-second example of pull slices in the video playback processing method according to the second embodiment of the present invention;
fig. 30 shows a twenty-third example of pull slices in the video playing processing method according to the second embodiment of the present invention;
fig. 31 is a schematic structural diagram illustrating a video playback processing apparatus according to an embodiment of the present invention;
fig. 32 is a schematic structural diagram of a computing device according to an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention can be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.
Example one
Fig. 2 is a flowchart illustrating a video playing processing method according to an embodiment of the present invention. As shown in fig. 2, the method comprises the steps of:
step S110, determining the current rotating speed of the head-mounted display equipment, and the current playing frame and the current playing video stream in the current visual angle according to the current operating data of the head-mounted display equipment; the currently played video stream is one of the multiple paths of high-definition video streams.
The executing subject of this embodiment may be a VR player based on the FOV, and the VR player integrates the video playing processing apparatus disclosed in this embodiment, and the apparatus may be implemented in a software or hardware manner.
The current operation data may be at least one of a current rotation speed (unit may be °/s) of the head-mounted display device, a current rotation direction, and a current viewing angle. Specifically, the current operation data may be acquired and transmitted back through a gravity sensor of the head-mounted display device. Further, a slice list in the current view angle can be obtained through calculation according to the current operation data, and the current playing frame and the current playing video stream in the current view angle can be determined based on the slice list. When the current rotating speed of the head-mounted display equipment exceeds a threshold value or the current playing fragment is required to be pulled up when the time is up, the video playing processing method provided by the embodiment of the invention is triggered to be executed.
The multiple paths of high-definition video streams at least comprise one path of low-I frame frequency high-definition video stream and one path of high-I frame frequency high-definition video stream, the number of the multiple paths of high-definition video streams and the I frame frequency can be set in a self-defined mode according to actual conditions, and the number of the multiple paths of high-definition video streams and the I frame frequency can be three or four, and the setting is not specifically limited here.
Step S120, pulling a target fragment of the target high-definition video stream from the multi-path high-definition video stream for display according to the current rotating speed, the current playing frame and the current playing video stream; the I frame frequency of the multiple paths of high-definition video streams is different, the video content is the same, the I frame frequency of each path of high-definition video stream is fixed, each path of high-definition video stream comprises a plurality of fragments, and the first frame of each fragment is an I frame.
In order to adapt to video requirements in various visual angles and at rotating speeds, multiple paths of high-definition video streams are respectively encoded according to the same video content, the I frame frequencies of the multiple paths of high-definition video streams are different, the I frame frequency of each path of high-definition video stream is fixed and divided into slices, the first frame of each slice is an I frame, the rest frames are P frames, and the decoding of the P frames depends on the I frames in the slices.
In an alternative embodiment, in executing the video playing processing method according to the embodiment of the present invention, a low-definition video stream needs to be encoded according to an original video (e.g., a panoramic 360-degree video), the low-definition video stream is divided into slices in a fixed time (e.g., each slice is 30 frames long by 1 second), a first frame of each slice is an I frame, and the rest of each slice is a P frame, and decoding of the P frame depends on the I frame in the corresponding slice. And encoding multiple paths of high-definition video streams according to the original video, wherein the I frame frequencies of the multiple paths of high-definition video streams are different, the I frame frequency of each path of high-definition video stream is fixed and is divided into slices, and the first frame of each slice is an I frame. When displaying, pulling one path of low-definition video stream fragments and one path of target high-definition video stream fragments, and fusing and displaying the one path of low-definition video stream fragments and the target high-definition video stream fragments.
In the embodiment, the target fragment of the target high-definition video stream is pulled from the multiple paths of high-definition video streams with different I frame frequencies for display, so that the target high-definition video stream can be automatically selected, the bandwidth can be dynamically adjusted, and the requirements of low delay and low bandwidth are met.
In an optional embodiment, step S120 specifically includes: when the current rotating speed reaches a preset condition, pulling the target fragment which is closest to the current playing frame time in the target high-definition video stream from the multi-path high-definition video stream according to the current playing video stream and the current playing frame for displaying.
The preset conditions can be set in a self-defined mode, and the unit can be degrees/s. For example, the predetermined condition may be any value or range of values from 0 to 180 °/s or over 180 °/s.
When the current rotating speed reaches the preset condition, the target fragment closest to the current playing frame time can be pulled for display, so that the low-definition high-definition switching with zero delay is achieved under the ideal condition.
In an optional embodiment, when the preset condition is a first preset condition, the target high-definition video stream is a high-definition video stream with the highest I-frame frequency in the multiple high-definition video streams.
The first preset condition represents that the rotation speed of the head-mounted display device is relatively high, for example, more than 90 °/s, at this time, in order to ensure that the viewing angle is quickly switched to the high-definition video image, the high-definition video stream with the highest I-frame frequency is selected as the target high-definition video stream, and because the high-definition video stream with the highest I-frame frequency has a relatively small I-frame interval, no matter which video stream is currently played, the target fragment closest to the currently played frame time in the high-definition video stream with the highest I-frame frequency is pulled for display, and therefore the target fragment can be well connected, and the middle of the target fragment does not wait for too long time.
In an optional embodiment, when the preset condition is a second preset condition, the step S120 specifically includes: determining the initial frame number of the current playing fragment according to the current playing video stream and the current playing frame; and pulling the target fragment closest to the current playing frame time in the target high-definition video stream from the multi-path high-definition video stream for display according to the initial frame number of the current playing fragment and the current playing frame.
Wherein the second preset condition is indicative of a situation where the rotational speed of the head mounted display device is small, e.g. less than 90 °/s. Under the condition, when the currently played video stream is a high-definition video stream with a low I-frame frequency, the start frame number of the currently played segment needs to be determined first, the target video stream is determined according to the start frame number, and then the target segment closest to the currently played frame time is pulled for display. See the description below for details relating to example two.
In an optional embodiment, when the preset condition is a third preset condition, the method of this embodiment further includes: and pausing the pulling of any one of the multiple paths of high-definition video streams.
Wherein the third preset condition represents that the rotation speed of the head-mounted display device is too fast, for example, greater than 180 °/s, and the high definition video stream is temporarily suspended since the rotation speed is too fast and it has no meaning to pull the high definition video stream.
In an optional embodiment, step S120 specifically includes: when the visual angle of the user is determined to be fixed and unchanged according to the current rotating speed, pulling a high-definition video stream with the lowest I frame frequency from the multiple paths of high-definition video streams as a target high-definition video stream; and pulling the nearest fragment from the target high-definition video stream according to the current playing frame for displaying.
When the visual angle of a user is fixed, the high-definition video stream with the lowest I frame frequency is pulled to serve as the target high-definition video stream, and then the nearest fragment is pulled from the target high-definition video stream to be displayed, so that the bandwidth can be effectively reduced.
Example two
Fig. 3 shows a flowchart of a video playing processing method according to a second embodiment of the present invention. The embodiment is applicable to the following specific scenes: the frame rate of the original video is 30 FPS; slice decoding must start with an I-frame and P-frames cannot be decoded independently, requiring reliance on the previous I-frame. As shown in fig. 3, the method comprises the steps of:
step S210, encoding a path of low definition (2K or 8K) video stream, and slicing the video stream according to fixed time, wherein each slice is 1 second long and is 30 frames in total, and the first frame of each slice is an I frame.
As shown in fig. 4, is an example of a one-way low-definition (2K or 8K) video stream.
And S220, coding the first path of high-definition video stream, and recording as h 1.
As shown in fig. 5, I frame frequency is fixed, one I frame is provided for every 5 frames, and the rest are P frames without B frames; every 5 frames are one fragment; each slice covers a viewing angle of 30. According to the formula, the duration of one slice is 5/30 ═ 1/6 seconds; a slice covers a viewing angle of 30 deg., and this slice can support up to 30 deg./(1/6 s) — 180 deg./s.
And step S230, encoding the second path of high-definition video stream, and recording as h 2.
As shown in fig. 6, I frame frequency is fixed, one I frame is provided for every 10 frames, and the rest are P frames without B frames; every 10 frames are one slice; each slice covers a viewing angle of 30. According to the formula, the duration of one slice is 10/30 ═ 1/3 seconds; a slice covers a viewing angle of 30 deg., and this slice can support up to 30 deg./(1/3 s) — 90 deg./s.
And S240, encoding the third path of high-definition video stream, and recording as h 3.
As shown in fig. 7, I frame frequency is fixed, one I frame is provided every 20 frames, and the rest are P frames without B frames; every 20 frames is a slice; each slice covers a viewing angle of 30. According to the formula, the duration of one slice is 20/30 ═ 2/3 seconds; a slice covers a viewing angle of 30 deg., and this slice can support up to 30 deg./(2/3 s) ═ 45 deg./s.
And step S250, when the rotating speed of the head-mounted display equipment exceeds a threshold value or a new fragment needs to be pulled when the time of the currently played fragment is up, obtaining the current rotating speed and direction according to data transmitted back by a gravity sensor of the head-mounted display equipment.
The current rotational speed is given in degrees/s and direction as v 1.
Step S260, obtaining a current visual angle according to data sent back by a gravity sensor of the head-mounted display device, calculating to obtain a slicing list in the current visual angle, and determining a current playing frame and a current playing video stream according to the slicing list.
And step S270, according to the current rotating speed, the current playing frame and the current playing video stream, pulling the target fragment of one high-definition video stream from the three high-definition video streams for displaying.
Specifically, the implementation of this step can be divided into the following cases according to the current rotation speed:
in the first case: the current rotating speed is larger than 180 degrees per second, and the high-definition video stream is not pulled because the rotating speed is too fast, and the high-definition video stream is temporarily stopped being pulled.
In the second case: the current rotation speed is 90 °/s < v1 ≤ 180 °/s, as shown in fig. 8, if the current playing video stream is h1, i.e. the current playing stream, the pulled high-definition video stream is h1, i.e. a new pulling stream, and the newly pulled segment is calculated according to the frame number of the current playing frame and is the segment closest to the current playing frame time;
as shown in fig. 9, if the current play stream is h2, the pulled high definition video stream is h1, and the pulled segment is the segment closest to the current play frame calculated according to the frame number of the current play frame.
As shown in fig. 10, if the current play stream is h3, the pulled high definition video stream is h1, and the pulled segment is the segment closest to the current play frame calculated according to the frame number of the current play frame.
Therefore, no matter which stream the currently played video stream is, the fragment of the high-definition video stream h1 closest to the currently played frame is pulled, so that the view angle can be ensured to be quickly switched to a high-definition image; meanwhile, h1 has the minimum I frame interval, so the newly pulled fragments of h1 can be just connected, and no multiple frames wait in the middle. In the third case, the current rotation speed is 45 °/s < v1 ≤ 90 °/s, as shown in fig. 11, the current playing video stream is h1 stream, and the starting frame number of the current playing segment is 2 × N × 5, where N is a positive integer, because the h1 stream segment at this time is located in the first half of the h2 stream segment with the same starting frame number, if the segment closest to the current playing frame in the h2 stream is directly pulled, the middle of the segment needs to wait for at least 5 frames, and the waiting time is longer, so the segment closest to the current playing frame in the h1 stream is pulled as the target segment.
As shown in fig. 12, the currently played video stream is h1 stream, and the start frame number of the currently played slice is (2 × N-1) × 5, because the h1 stream slice at this time is located in the second half of the h2 stream slice with the same start frame number, the slice closest to the currently played frame in the h2 stream can be directly pulled as the target slice, and there is no long waiting time in the middle.
As shown in fig. 13, the currently played video stream is an h2 stream, and the currently played segment is played to the first 5 frames, if the segment closest to the currently played frame in the h2 stream is directly pulled, at least 5 frames of time need to be waited in the middle, and the waiting time is long, so that the segment closest to the currently played frame in the h1 stream is pulled as the target segment.
As shown in fig. 14, when the currently played video stream is an h2 stream, and the currently played segment is played to the last 5 frames, the segment closest to the currently played frame in the h2 stream can be directly pulled as the target segment, and no long time is left in the middle.
As shown in fig. 15, the currently played video stream is an h3 stream, and the currently played segment is played to the first 5 frames, if the segment closest to the currently played frame in the h2 stream is directly pulled, at least 5 frames of time need to be waited in the middle, and the waiting time is long, so the segment closest to the currently played frame in the h1 stream is pulled as the target segment.
As shown in fig. 16, the currently played video stream is h3 stream, and the currently played segment is played to 5 to 10 frames, the segment closest to the currently played frame in the h2 stream can be directly pulled as the target segment, and no long waiting time exists in the middle.
As shown in fig. 17, the currently played video stream is an h3 stream, and the currently played segment is played to 10 to 15 frames, if the segment closest to the currently played frame in the h2 stream is directly pulled, at least 5 frames of time need to be waited in the middle, and the waiting time is long, so the segment closest to the currently played frame in the h1 stream is pulled as the target segment.
As shown in fig. 18, the currently played video stream is h3 stream, and the currently played segment is played to 15 to 20 frames, the segment closest to the currently played frame in the h2 stream can be directly pulled as the target segment, and no long waiting time exists in the middle.
In a fourth case: the current rotation speed v1 is not more than 45 °/s, as shown in fig. 19, the currently played video stream is h1 stream, the starting frame number of the currently played segment is 20 × N × 5, the currently played segment is located in the first half segment of the h3 stream segment with the same starting frame number, if the segment closest to the currently played frame in the h3 stream is directly pulled, at least 15 frames of time need to be waited in the middle, and the waiting time is longer, so the segment closest to the currently played frame in the h1 stream is pulled as the target segment.
As shown in fig. 20, the currently played video stream is h1 stream, the start frame number of the currently played segment is 20 × N-15, the currently played segment is located in the first half of the h3 stream segment with the same start frame number, if the segment closest to the currently played frame in the h3 stream is directly pulled, the middle of the segment needs to wait for at least 10 frames, and the waiting time is long, so the segment closest to the currently played frame in the h2 stream is pulled as the target segment.
As shown in fig. 21, the currently played video stream is h1 stream, the start frame number of the currently played segment is 20 × N-10, the currently played segment is located in the middle of the h3 stream segment with the same start frame number, if the segment closest to the currently played frame in the h3 stream is directly pulled, the waiting time needs to be at least 5 frames in the middle, and the waiting time is long, so the segment closest to the currently played frame in the h1 stream is pulled as the target segment.
As shown in fig. 22, the currently played video stream is h1 stream, the start frame number of the currently played segment is 20 × N-5, the currently played segment is located at the back of the h3 stream segment with the same start frame number, the segment closest to the currently played frame in the h3 stream is directly pulled as the target segment, and the waiting time is short.
As shown in fig. 23, the currently played video stream is h2 stream, the start frame number of the currently played segment is 2 × N × 10, the currently played segment is played to the first 5 frames, the currently played segment is located in the first half segment of the h3 stream segment with the same start frame number, if the segment closest to the currently played frame in the h3 stream is directly pulled, the waiting time needs to be at least 10 frames in the middle, and the waiting time is longer, so the segment closest to the currently played frame in the h1 stream is pulled as the target segment.
As shown in fig. 24, the currently played video stream is an h2 stream, the start frame number of the currently played segment is 2 × N × 10, the currently played segment is played to the last 5 frames, the currently played segment is located in the first half segment of the h3 stream segment with the same start frame number, if the segment closest to the currently played frame in the h3 stream is directly pulled, at least 10 frames of time need to be waited in the middle, and the waiting time is long, so that the segment closest to the currently played frame in the h2 stream needs to be pulled again as the target segment.
As shown in fig. 25, the currently played video stream is h2 stream, the starting frame number of the currently played segment is (2 × N-1) × 10, the currently played segment is played to the first 5 frames, the currently played segment is located in the middle of the h3 stream segment with the same starting frame number, if the segment closest to the currently played frame in the h3 stream is directly pulled, the middle of the segment needs to wait for at least 5 frames, and the waiting time is long, so the segment closest to the currently played frame in the h1 stream is pulled as the target segment.
As shown in fig. 26, the currently played video stream is h2 stream, the starting frame number of the currently played segment is (2 × N-1) × 10, and the current segment is played to the last 5 frames, and the segment closest to the currently played frame in the h3 stream is directly pulled as the target segment, so that the waiting time is short.
As shown in fig. 27, the currently played video stream is an h3 stream, and the currently played segment is played to the first 5 frames, if the segment closest to the currently played frame in the h3 stream is directly pulled, at least 15 frames of time need to be waited in the middle, and the waiting time is long, so that the segment closest to the currently played frame in the h1 stream is pulled as the target segment.
As shown in fig. 28, the currently played video stream is an h3 stream, and the currently played segment is played to 5 to 10 frames, if the segment closest to the currently played frame in the h3 stream is directly pulled, at least 10 frames of time need to be waited for in the middle, and the waiting time is long, so the segment closest to the currently played frame in the h2 stream is pulled as the target segment.
As shown in fig. 29, the currently played video stream is an h3 stream, and the currently played segment is played to 10 to 15 frames, if the segment closest to the currently played frame in the h3 stream is directly pulled, at least 5 frames of time need to be waited in the middle, and the waiting time is long, so the segment closest to the currently played frame in the h1 stream is pulled as the target segment.
As shown in fig. 30, the currently played video stream is h3 stream, and the currently played segment is played to the last 5 frames, and the segment closest to the currently played frame in the h3 stream is directly pulled as the target segment, so that the waiting time is short.
And for the third situation and the fourth situation, namely when the current rotating speed is below 90 degrees, determining the initial frame number of the current playing fragment according to the current playing video stream and the current playing frame, and further pulling the target fragment which is closest to the current playing frame time in the target high-definition video stream from the multi-channel high-definition video stream according to the initial frame number of the current playing fragment and the current playing frame for displaying.
In this embodiment, 3 channels of high definition video streams with different I frame frequencies in an interdependent relationship are adopted, and the high definition video streams and the fragments are switched in a self-adaptive manner, so that the defect that the high definition video streams fix the I frame frequencies is overcome, the problem that high bandwidth and low delay of the high definition video streams cannot be realized simultaneously is solved, the bandwidth is reduced, and high definition switching delay is reduced.
EXAMPLE III
Fig. 31 is a block diagram illustrating a video playing process according to a third embodiment of the present invention. As shown in fig. 31, the apparatus includes: a determination module 311 and a display module 312; wherein,
the determining module 311 is configured to determine, according to current operating data of the head-mounted display device, a current rotational speed of the head-mounted display device, and a current playing frame and a current playing video stream within a current viewing angle; the currently played video stream is one of the multiple paths of high-definition video streams;
the display module 312 is configured to pull a target segment of a target high-definition video stream from the multiple high-definition video streams for display according to the current rotation speed, the current playing frame, and the current playing video stream; the I frame frequencies of the multiple paths of high-definition video streams are different, the video contents are the same, the I frame frequency of each path of high-definition video stream is fixed, each path of high-definition video stream comprises a plurality of fragments, and the first frame of each fragment is an I frame.
Further, the display module 312 is specifically configured to: and when the current rotating speed reaches a preset condition, pulling the target fragment which is closest to the current playing frame time in the target high-definition video stream from the multi-path high-definition video stream according to the current playing video stream and the current playing frame for displaying.
Further, when the preset condition is a first preset condition, the target high-definition video stream is a high-definition video stream with the highest I-frame frequency in the multiple paths of high-definition video streams.
Further, when the preset condition is a second preset condition, the display module 312 is specifically configured to: determining the initial frame number of the current playing fragment according to the current playing video stream and the current playing frame; and pulling the target fragment which is closest to the current playing frame time in the target high-definition video stream from the multi-path high-definition video stream for display according to the initial frame number and the current playing frame of the current playing fragment.
Further, the display module 312 is further configured to: and pausing the pulling of any one of the multiple paths of high-definition video streams.
Further, the display module 312 is further configured to: when the visual angle of the user is determined to be fixed according to the current rotating speed, pulling a high-definition video stream with the lowest I frame frequency from multiple paths of high-definition video streams as the target high-definition video stream; and pulling the nearest fragment from the target high-definition video stream according to the current playing frame for displaying.
Further, the remaining frames of each slice are P frames, and the decoding of P frames depends on the I frames in the slice.
The video playing processing apparatus according to the embodiment of the present invention is configured to execute the video playing processing method according to the above embodiment, and the working principle and the technical effect are similar, which are not described herein again.
Example four
An embodiment of the present invention provides a non-volatile computer storage medium, where the computer storage medium stores at least one executable instruction, and the computer executable instruction may execute the video playing processing method in any of the above method embodiments.
EXAMPLE five
Fig. 32 is a schematic structural diagram of a computing device according to an embodiment of the present invention, and a specific embodiment of the present invention does not limit a specific implementation of the computing device.
As shown in fig. 32, the computing device may include: a processor (processor), a Communications Interface (Communications Interface), a memory (memory), and a Communications bus.
Wherein: the processor, the communication interface, and the memory communicate with each other via a communication bus. A communication interface for communicating with network elements of other devices, such as clients or other servers. The processor is configured to execute a program, and may specifically execute the video playing processing method for the computing device and the related steps in the embodiment of the cell azimuth prediction method.
In particular, the program may include program code comprising computer operating instructions.
The processor may be a central processing unit CPU or an application Specific Integrated circuit asic or one or more Integrated circuits configured to implement embodiments of the present invention. The computing device includes one or more processors, which may be the same type of processor, such as one or more CPUs; or may be different types of processors such as one or more CPUs and one or more ASICs.
And the memory is used for storing programs. The memory may comprise high-speed RAM memory, and may also include non-volatile memory (non-volatile memory), such as at least one disk memory.
The program may specifically be configured to cause the processor to execute the video playing processing method in any of the above method embodiments. For specific implementation of each step in the program, reference may be made to corresponding steps and corresponding descriptions in units in the foregoing embodiments of the video playing processing method, which are not described herein again. It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described devices and modules may refer to the corresponding process descriptions in the foregoing method embodiments, and are not described herein again.
The algorithms or displays presented herein are not inherently related to any particular computer, virtual system, or other apparatus. Various general purpose systems may also be used with the teachings herein. The required structure for constructing such a system will be apparent from the description above. In addition, embodiments of the present invention are not directed to any particular programming language. It is appreciated that a variety of programming languages may be used to implement the teachings of embodiments of the present invention as described herein, and any descriptions of specific languages are provided above to disclose the best modes of embodiments of the invention.
In the description provided herein, numerous specific details are set forth. It is understood, however, that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the invention, various features of the embodiments of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be interpreted as reflecting an intention that: that is, the claimed embodiments of the invention require more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this invention.
Those skilled in the art will appreciate that the modules in the device in an embodiment may be adaptively changed and disposed in one or more devices different from the embodiment. The modules or units or components of the embodiments may be combined into one module or unit or component, and furthermore they may be divided into a plurality of sub-modules or sub-units or sub-components. All of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or elements of any method or apparatus so disclosed, may be combined in any combination, except combinations where at least some of such features and/or processes or elements are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.
Furthermore, those skilled in the art will appreciate that while some embodiments herein include some features included in other embodiments, rather than other features, combinations of features of different embodiments are meant to be within the scope of the invention and form different embodiments. For example, in the following claims, any of the claimed embodiments may be used in any combination.
The various component embodiments of the invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art will appreciate that a microprocessor or Digital Signal Processor (DSP) may be used in practice to implement some or all of the functionality of some or all of the components according to embodiments of the present invention. Embodiments of the invention may also be implemented as apparatus or device programs (e.g., computer programs and computer program products) for performing a portion or all of the methods described herein. Such programs implementing embodiments of the present invention may be stored on computer-readable media or may be in the form of one or more signals. Such a signal may be downloaded from an internet website or provided on a carrier signal or in any other form.
It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. Embodiments of the invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The usage of the words first, second and third, etcetera do not indicate any ordering. These words may be interpreted as names. The steps in the above embodiments should not be construed as limiting the order of execution unless specified otherwise.
Claims (10)
1. A video playback processing method, comprising:
determining the current rotating speed of the head-mounted display equipment, and the current playing frame and the current playing video stream in the current visual angle according to the current operating data of the head-mounted display equipment; the currently played video stream is one of the multiple paths of high-definition video streams;
pulling a target fragment of a target high-definition video stream from the multi-path high-definition video stream for display according to the current rotating speed, the current playing frame and the current playing video stream; the I frame frequencies of the multiple paths of high-definition video streams are different, the video contents are the same, the I frame frequency of each path of high-definition video stream is fixed, each path of high-definition video stream comprises a plurality of fragments, and the first frame of each fragment is an I frame.
2. The method according to claim 1, wherein the pulling a target slice of a target high definition video stream from a multi-channel high definition video stream for display according to the current rotation speed, the current playing frame and the current playing video stream comprises:
and when the current rotating speed reaches a preset condition, pulling the target fragment which is closest to the current playing frame time in the target high-definition video stream from the multi-path high-definition video stream according to the current playing video stream and the current playing frame for displaying.
3. The method according to claim 2, wherein when the preset condition is a first preset condition, the target high-definition video stream is a high-definition video stream with a highest I-frame frequency in the multiple high-definition video streams.
4. The method according to claim 2, wherein when the preset condition is a second preset condition, the pulling, according to the currently played video stream and the currently played frame, a target segment closest to the currently played frame time from a target high definition video stream from the multiple high definition video streams for display comprises:
determining the initial frame number of the current playing fragment according to the current playing video stream and the current playing frame;
and pulling the target fragment which is closest to the current playing frame time in the target high-definition video stream from the multi-path high-definition video stream for display according to the initial frame number and the current playing frame of the current playing fragment.
5. The method according to claim 2, wherein when the preset condition is a third preset condition, the method further comprises:
and pausing the pulling of any one of the multiple paths of high-definition video streams.
6. The method according to claim 1, wherein the pulling a target slice of a target high definition video stream from a multi-channel high definition video stream for display according to the current rotation speed, the current playing frame and the current playing video stream comprises:
when the visual angle of the user is determined to be fixed according to the current rotating speed, pulling a high-definition video stream with the lowest I frame frequency from multiple paths of high-definition video streams as the target high-definition video stream;
and pulling the nearest fragment from the target high-definition video stream according to the current playing frame for displaying.
7. The method of claim 1, wherein the remaining frames of each slice are P-frames, and wherein the decoding of a P-frame depends on the I-frames in the slice.
8. A video playback processing apparatus, comprising:
the determining module is used for determining the current rotating speed of the head-mounted display equipment, and the current playing frame and the current playing video stream in the current visual angle according to the current operating data of the head-mounted display equipment; the currently played video stream is one of the multiple paths of high-definition video streams;
the display module is used for pulling the target fragments of the target high-definition video stream from the multi-path high-definition video stream to display according to the current rotating speed, the current playing frame and the current playing video stream; the I frame frequencies of the multiple paths of high-definition video streams are different, the video contents are the same, the I frame frequency of each path of high-definition video stream is fixed, each path of high-definition video stream comprises a plurality of fragments, and the first frame of each fragment is an I frame.
9. A computing device, comprising: the system comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface complete mutual communication through the communication bus;
the memory is used for storing at least one executable instruction, and the executable instruction causes the processor to execute the operation corresponding to the video playing processing method according to any one of claims 1-7.
10. A computer storage medium having at least one executable instruction stored therein, the executable instruction causing a processor to perform operations corresponding to the video playback processing method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110994334.5A CN113691797B (en) | 2021-08-27 | 2021-08-27 | Video playing processing method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110994334.5A CN113691797B (en) | 2021-08-27 | 2021-08-27 | Video playing processing method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113691797A true CN113691797A (en) | 2021-11-23 |
| CN113691797B CN113691797B (en) | 2023-08-01 |
Family
ID=78583362
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110994334.5A Active CN113691797B (en) | 2021-08-27 | 2021-08-27 | Video playing processing method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113691797B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106937141A (en) * | 2017-03-24 | 2017-07-07 | 北京奇艺世纪科技有限公司 | A kind of bitstreams switching method and device |
| CN109218739A (en) * | 2017-07-06 | 2019-01-15 | 阿里巴巴集团控股有限公司 | View angle switch method, apparatus, equipment and the computer storage medium of video flowing |
| CN111741314A (en) * | 2020-06-18 | 2020-10-02 | 聚好看科技股份有限公司 | Video playing method and display equipment |
| CN112738508A (en) * | 2020-12-31 | 2021-04-30 | 咪咕文化科技有限公司 | Video coding method, video determining method, video processing method, server and VR terminal |
-
2021
- 2021-08-27 CN CN202110994334.5A patent/CN113691797B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106937141A (en) * | 2017-03-24 | 2017-07-07 | 北京奇艺世纪科技有限公司 | A kind of bitstreams switching method and device |
| CN109218739A (en) * | 2017-07-06 | 2019-01-15 | 阿里巴巴集团控股有限公司 | View angle switch method, apparatus, equipment and the computer storage medium of video flowing |
| CN111741314A (en) * | 2020-06-18 | 2020-10-02 | 聚好看科技股份有限公司 | Video playing method and display equipment |
| CN112738508A (en) * | 2020-12-31 | 2021-04-30 | 咪咕文化科技有限公司 | Video coding method, video determining method, video processing method, server and VR terminal |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113691797B (en) | 2023-08-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110366033B (en) | Video playing method, device, equipment and storage medium | |
| WO2018171487A1 (en) | Panoramic video playback method and client terminal | |
| KR101477434B1 (en) | Out-of-order command execution in a multimedia processor | |
| US6959348B1 (en) | Method and system for accessing data | |
| EP3406310A1 (en) | Method and apparatuses for handling visual virtual reality content | |
| CN110784740A (en) | Video processing method, device, server and readable storage medium | |
| CN104639951B (en) | The pumping frame processing method and device of video code flow | |
| CN107911709A (en) | live interface display method, device and terminal | |
| CN107613302B (en) | Decoding method and device, storage medium and processor | |
| CN109218739B (en) | Method, device and equipment for switching visual angle of video stream and computer storage medium | |
| WO2019192481A1 (en) | Media information processing method, related device, and computer storage medium | |
| CN115883899B (en) | Image on-screen playing method and system, electronic equipment and readable storage medium | |
| CN107454434A (en) | Virtual reality net cast method and video playing terminal | |
| US9356981B2 (en) | Streaming content over a network | |
| KR101680545B1 (en) | Method and apparatus for providing panorama moving picture generation service | |
| CN114422799B (en) | Decoding method and device for video file, electronic equipment and program product | |
| US11678019B2 (en) | User interface (UI) engine for cloud UI rendering | |
| CN113691797A (en) | Video playing processing method, device, equipment and storage medium | |
| EP3205110B1 (en) | Video bitstream processing for enabling tune-in | |
| CN115460458B (en) | Video frame loss method and device | |
| CN109120939B (en) | Video App decoding method and device | |
| CN110446070A (en) | A kind of method and apparatus of video playing | |
| CN113316940A (en) | Method and device for circularly playing video content | |
| CN112243136A (en) | Content playing method, video storage method and equipment | |
| CN115914745A (en) | Video decoding method and device, electronic equipment and computer readable medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |