CN115174942A - Free visual angle switching method and interactive free visual angle playing system - Google Patents
Free visual angle switching method and interactive free visual angle playing system Download PDFInfo
- Publication number
- CN115174942A CN115174942A CN202210798544.1A CN202210798544A CN115174942A CN 115174942 A CN115174942 A CN 115174942A CN 202210798544 A CN202210798544 A CN 202210798544A CN 115174942 A CN115174942 A CN 115174942A
- Authority
- CN
- China
- Prior art keywords
- visual angle
- video stream
- view
- switching
- user
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/21—Server components or server architectures
- H04N21/218—Source of audio or video content, e.g. local disk arrays
- H04N21/21805—Source of audio or video content, e.g. local disk arrays enabling multiple viewpoints, e.g. using a plurality of cameras
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
- H04N21/4307—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
- H04N21/43072—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen of multiple content streams on the same device
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
- H04N21/643—Communication protocols
- H04N21/6437—Real-time Transport Protocol [RTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/85—Assembly of content; Generation of multimedia applications
- H04N21/854—Content authoring
- H04N21/8547—Content authoring involving timestamps for synchronizing content
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/85—Assembly of content; Generation of multimedia applications
- H04N21/858—Linking data to content, e.g. by linking an URL to a video object, by creating a hotspot
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
- H04N5/268—Signal distribution or switching
Abstract
The invention discloses a free visual angle switching method and an interactive free visual angle playing system. Wherein, the method comprises the following steps: acquiring a multi-view video stream; acquiring a visual angle switching signal according to an interactive instruction triggered by a user; and switching in a random access reference frame between the original visual angle video stream and the target visual angle video stream according to the visual angle switching signal, and generating and transmitting a single visual angle video stream. The method provided by the invention only needs to transmit one path of video, and by the method, the client always requests to decode and play one path of video stream, and transmits the interactive instruction of the user to the computing server through the signal service, and the video stream is recombined as required and distributed to the corresponding client, so that the waste of transmission bits is reduced to the maximum extent under the condition of not increasing the computing complexity, and the switching delay is shortened, thereby realizing the seamless switching between the view angles.
Description
Technical Field
The invention belongs to the field of digital image processing, and particularly relates to a free visual angle switching method and an interactive free visual angle playing system.
Background
With the rapid development of network communication technology and multimedia technology, people have higher and higher requirements on information processing capability. Carriers for information transmission have also evolved from text, images to video. Meanwhile, the requirements of people on the quality of video services and the diversity of contents are higher and higher, and the expression forms and presentation modes of videos are more and more diversified. The traditional single-view video cannot effectively meet the three-in-one enjoyment of vision, touch and hearing sought by a user, and the immersive video is born under the technical background that the terminal computing capacity and the camera shooting and collecting technology are greatly improved.
The traditional live video broadcasting and on-demand video broadcasting basically pushes the video picture of the guide to the user side for watching, the user cannot freely select the picture visual angle which the user wants to watch, and the freshness and entertainment are lacked. Riding 5G's spring breeze, the experience has been watched passively to traditional having broken through to the free viewpoint technique, lets the interactive live broadcast of incredible become reality, and the user is through the real-time interactive mode of sliding the cell-phone screen, around live broadcast screen free rotation, selects the most suitable angle of watching of oneself, enjoys the immersive experience of full scene. The generation of free viewpoint video requires large scale arrangement of cameras, forming an array, to capture a large amount of viewpoint information for a partial field of view of a scene. There are two categories that can be classified according to the number of viewpoints collected: super multi-view video and free-cruise video.
In the super multi-view video, cameras are arranged closely, and the number of collected viewpoints is large. At the playing end, because the captured viewpoints are densely distributed, viewpoint synthesis rendering is not required in a viewpoint range formed by the camera array, and smoother motion parallax can be provided only by switching viewpoints.
On the other hand, for the free-cruise video, the arrangement of the cameras is sparse, and the number of collected view angles is relatively small. At the playback end, in order to provide continuous motion parallax at the time of view switching, it is necessary to synthetically render a virtual view using a view synthesis technique. In order to improve the image quality of the virtual viewpoint, it is generally necessary to additionally transmit a depth map of the corresponding viewpoint. When a user interacts with a free viewpoint video, not only the continuity of viewpoint switching but also the timeliness of viewpoint switching response need to be ensured, it is not an easy matter to realize low delay under such a large data volume, and huge computation complexity exists at both ends of an encoder and a decoder. How to optimize the encoding and transmission of premium content based on the interactive content and actual bandwidth of the user becomes a key issue in immersive video interactive transmission.
Disclosure of Invention
In order to solve the technical problem that the transmission bandwidth and the terminal calculation pressure are too heavy in the prior art, the invention aims to provide a free visual angle switching method and an interactive free visual angle playing system. By using the free visual angle switching method and the interactive free visual angle playing system, a user freely rotates around a live broadcast picture in a real-time interactive mode through a sliding mobile phone screen, selects the most appropriate viewing angle, and enjoys full-scene and immersive experience at will. The system realizes smooth and free real-time picture switching among multi-view pictures through technologies such as accurate synchronous control, compression coding, multi-view synchronization, webRTC low-delay transmission and the like, and generates a dynamic bullet time effect, so that a user is changed from 'audience passively receiving contents' to 'director mastering control right', and the requirements of autonomy and diversified cultural expression of the user watching scenes are met.
The application provides a free visual angle switching method, which comprises the following steps:
acquiring a multi-view video stream;
acquiring a visual angle switching signal according to an interactive instruction triggered by a user;
and switching in a random access reference frame between the original visual angle video stream and the target visual angle video stream according to the visual angle switching signal, and generating and transmitting a single visual angle video stream.
According to an embodiment of the present application, after acquiring the multi-view video stream, the method further includes:
acquiring a time stamp of each frame image in each single-view video stream in the multi-view video stream;
and performing frame synchronization on the plurality of single-view video streams according to the time stamps.
According to an embodiment of the present application, the acquiring, according to an interactive instruction triggered by a user, a view switching signal includes:
determining the switched target visual angle information according to user sliding, input, gestures or clicking operations;
and determining the time information of the switching according to the sliding, input, gesture or clicking operation of the user.
According to one embodiment of the application, when a user slides a screen leftwards or upwards, the previous visual angle of the current visual angle along one direction is used as target visual angle information after the target visual angle is switched;
and when the user slides the screen to the right or downwards, taking the next visual angle of the current visual angle along one direction as the target visual angle information after the target visual angle is switched.
According to an embodiment of the present application, the switching in a random access reference frame between an original view video stream and a target view video stream according to a view switching signal to generate and output a single view video stream includes:
when switching occurs, accessing the random access reference frame after the original visual angle video stream;
and when the switching is finished, accessing the target visual angle video stream after the random access reference frame to obtain the single visual angle video stream.
According to an embodiment of the present application, accessing the random access reference frame after the original view video stream includes:
creating an instance;
according to the switching visual angle information, preselecting one or more random access code streams of single visual angle video streams from an original visual angle to a target visual angle in the multi-visual angle video streams;
extracting a corresponding random access reference frame from the pre-selected random access code stream;
and accessing the corresponding random access reference frame behind the original view video stream.
According to an embodiment of the present application, the random access code stream is formed by sequentially time-stamping the corresponding key frames of the single-view video stream.
According to one embodiment of the application, the single-view video stream is transmitted based on a WebRTC protocol.
The application provides an interactive free visual angle play system, includes:
the video stream acquisition device is in communication connection with the computing server;
one or more computing servers for acquiring a multi-view video stream;
a signal server in communication connection with the computing server;
a streaming media server communicatively coupled to the computing server;
more than one client, which is connected with the signaling server in a communication way;
the computing server acquires a multi-view video stream from the video stream acquisition device; the client transmits an interactive instruction to the computing server through the signal server; and the computing server outputs a single-channel video stream to the streaming media server according to the interactive instruction, and the streaming media server sends the single-channel video stream to the client.
According to one embodiment of the application, the video stream acquisition device comprises:
more than one video data acquisition module for acquiring multi-view video stream;
more than one image correction module for correcting image data;
more than one video coding plug flow module; and
RTMP streaming media server.
The invention has the beneficial effects that:
the method provided by the invention only needs to transmit one path of video, and by the method, the client always requests to decode and play one path of video stream, and transmits the interactive instruction of the user to the computing server through the signal service, and the video stream is recombined as required and distributed to the corresponding client, so that the waste of transmission bits is reduced to the maximum extent under the condition of not increasing the computing complexity, and the switching delay is shortened, thereby realizing the seamless switching between the view angles.
The interactive free visual angle playing system using the method of the invention combines the WebRTC technology with low-delay communication to realize the transmission of the free visual angle video, reduces the waste of transmission bit as much as possible under the condition of not increasing the calculation complexity, shortens the switching delay, thereby realizing the seamless switching between the visual angles.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the description below are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a flow chart illustrating a free view switching method in the present embodiment;
fig. 2 is a schematic diagram showing the structure of each view code stream in the present embodiment;
fig. 3 shows a schematic diagram of the structure of each view bit stream and its random access bit stream in this embodiment;
fig. 4 shows a schematic structural diagram of a single-view video stream in the method for switching free views in this embodiment.
Fig. 5 is a schematic structural diagram of the interactive freeview playback system in this embodiment.
The reference numbers illustrate:
1-a video stream acquisition device;
2-a computing server;
3-a signal server;
4-a streaming media server;
5-a client;
11-a video data acquisition module;
12-an image correction module;
13-video coding plug flow module;
14-RTMP streaming media server.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, are within the scope of the present invention.
In the description of the present invention, it should be noted that the terms "upper", "lower", "left", "right", "center", "vertical", "horizontal", "inner", "outer", and the like indicate orientations or positional relationships based on those shown in the drawings, and are only for convenience of description and simplification of description, but do not indicate or imply that the device or element referred to must have a specific orientation, be constructed in a specific orientation, and be operated, and thus, should not be construed as limiting the present invention. Furthermore, the terms "first," "second," "third," and the like are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.
In the description of the present invention, it is to be noted that, unless otherwise explicitly specified or limited, the terms "mounted," "disposed," "connected," and the like are to be construed broadly, and for example, "connected" may be a fixed connection, a detachable connection, or an integral connection; can be mechanically or electrically connected; either directly or indirectly through intervening media, or both elements may be in communication. The specific meanings of the above terms in the present invention can be understood in a specific case to those of ordinary skill in the art.
Fig. 1 shows a flow chart of the free view angle switching method in this embodiment. Wherein the method comprises the following steps:
s1: acquiring a multi-view video stream;
in the present method, other sources for acquiring the multi-view video stream include, but are not limited to: 1. the images are captured by a plurality of cameras and then directly sent to the computing server 2 to be acquired. 2. The data is captured by a plurality of cameras and then sent to the RTMP streaming media server for storage, and the RTMP streaming media server forwards the data to the computing server 2.
In one possible implementation, the first source is live video taken in real-time of the same scene by using different cameras. In yet another possible embodiment, the second source is an on-demand video obtained by shooting the same scene with the cameras from different viewing angles in advance.
In order to facilitate the subsequent time synchronization of the multi-view pictures, the display time stamp of each frame of picture is recorded during the encoding and packaging. The computing server 2 will pull all the multi-view video streams from the streaming media server, and in order to ensure the continuity of the video frames when the user switches the view, the computing server 2 must ensure the synchronization of the pulled multi-view frames, and the computing server 2 synchronizes through the timestamp information in each multi-view video stream. The pictures with the same timestamp are the pictures with different visual angles collected at the same time.
S2: acquiring a visual angle switching signal according to an interactive instruction triggered by a user;
the user-triggered interaction instructions include but are not limited to sliding, inputting, gesture or clicking operation, and the body function or the peripheral function based on the client 5 device further includes: limb movements captured by limb movements, eyeball movements captured by eyeball motion, and the like. For convenience of description, the interactive instruction refers to an interactive instruction generated by a user through a portable device such as a smart phone and a tablet computer, and includes: the method comprises the steps of sliding the screen in different directions by a user, inputting characters or numerical values in various text boxes or prompts, capturing gesture operations through a camera or the screen, clicking a motor operation of keys on the screen by fingers of the user, and the like.
In a possible implementation manner, when the user slides the screen to the left or upwards, the previous view angle of the current view angle along a direction is used as the target view angle information after the target view angle is switched. And when the user slides the screen to the right or downwards, taking the next visual angle of the current visual angle along one direction as the target visual angle information after the target visual angle is switched.
Specifically, the view switching signal obtained according to the interactive instruction triggered by the user includes the target view information after switching and the time information of switching.
S3: and switching in a random access reference frame between the original visual angle video stream and the target visual angle video stream according to the visual angle switching signal, and generating and transmitting a single visual angle video stream.
Fig. 2 shows a schematic diagram of the structure of each view stream in this embodiment. Each view in the figure is a video stream alone and is not spliced together, and the size of a Group of pictures (GoP) is 30, that is, each GoP includes 30 frames of data. The independently decodable frames in GoP are referred to as Random Access Points (RAPs) in H.265/HEVC, i.e., I-frames.
Fig. 3 shows a schematic structural diagram of each view code stream and its random access code stream in this embodiment. As can be seen from fig. 3, the code stream of each view is changed from the original one general code stream into two code streams, one is the same as the original general code stream, and the other is a version of the full I frame, which is also called a random access code stream. The I frames in the random access code stream are all in one-to-one correspondence with the corresponding I frames and P frames in the general code stream. When the user switches the view angle, these I frames replace their corresponding P frames as new Reference frames of the following P frames, which are called Random Access Reference (RARF).
When the user view angle changes, the calculation server 2 combines the corresponding data packets to send to the user based on the change of the user view angle, but this is only to switch between view angles at the RAP position. When the video is coded, the I frame has no inter-frame prediction dependency on the previous video frame, does not need a reference frame, and only needs to be coded by intra-frame prediction. So that the I-frame can be decoded independently. However, due to the limitation of the coding structure, the P frame needs to be decoded by relying on the previously decoded reference frame, so that the user cannot complete fast switching between views at the P frame position when switching views, and the new view can be normally switched to by the next I frame. The distance of the I frame affects the viewing angle switching response speed and the immersive experience of the user, and is one of the key factors of the user's viewing angle switching delay. However, if the distance of the I frame is reduced or the full I frame is encoded, the compression rate of the video encoding will be greatly reduced, and the code rate of the video will be increased, which causes network congestion. In fact, the user only needs to ensure that the next view is an I frame during the view switching process, and only needs to multiplex the data packets of the original multi-view video stream before and after the switching.
Therefore, based on the above principle, when a handover occurs, the random access reference frame is accessed after the original view video stream; and when the switching is finished, accessing the target visual angle video stream after the random access reference frame to obtain the single visual angle video stream. Specifically referring to fig. 4, fig. 4 is a schematic diagram illustrating a single-view video stream structure of the method for switching a free view in the embodiment. Assuming that the current View angle of the user is View i, before the View angle is switched by the user, the computing server 2 directly uses an original frame in the general stream of View i as an output frame after packet reassembly. When the computing server 2 receives the user switching View angle signal, switching from View I to View M, the computing server 2 switches to the synchronous I frame in the RARF stream of View I +1 with zero delay, and takes one frame of output frame after the I frame user packet is recombined from the RARF stream of View I +1 to View M according to time. When the View switching is finished, the computing server 2 then multiplexes the frames in the View M general stream as the reassembled output frames.
Fig. 5 shows an interactive free view playback system using the above-mentioned free view switching method in the present application. The system comprises: the video stream acquisition device 1 is in communication connection with the computing server 2; one or more computation servers 2 for acquiring multi-view video streams; a signal server communicatively connected to the computation server 2; a streaming media server communicatively connected to the computing server 2; and more than one client 5, which is connected with the signal server in communication; wherein, the computing server 2 acquires a multi-view video stream from the video stream acquisition device 1; the client 5 transmits an interactive instruction to the computing server 2 through the signal server; the computing server 2 outputs a single-channel video stream to the streaming media server according to the interactive instruction, and the streaming media server sends the single-channel video stream to the client 5.
The interactive free-viewing angle system in this embodiment transmits, according to the user interaction signal, the corresponding video frame to the user terminal for display and play based on the WebRTC on demand. WebRTC, the name of which is derived from the abbreviation of Web Real-Time Communication (Web Real-Time Communication), is a Real-Time Communication technology that allows Web applications or sites to establish point-to-point (Peer-to-Peer) connections between browsers without the aid of an intermediary, to enable transmission of video streams and/or audio streams or any other data, and to support Web browsers to perform Real-Time voice conversations or video conversations.
Specifically, the video stream capture device 1 includes: more than one video data acquisition module 11 for acquiring multi-view video stream; more than one image correction module 12 for image data correction; more than one video coding plug flow module 13; and an RTMP streaming media server.
The video data liniment module and the image correction module 12 adopt an annular camera array acquisition system, and the acquired multi-view video is transmitted to the RTMP streaming media server by each view picture code, so that time synchronization is performed on the multi-view pictures for the convenience of follow-up, and a display timestamp of each frame picture is recorded during code packaging. The RTMP streaming media server is a streaming media server based on Real-Time Messaging Protocol (Real Time Messaging Protocol), the computing server 2 will pull all the multi-view video streams from the RTMP streaming media server, and in order to ensure the continuity of the video frames when the users switch the view angles, the computing server 2 must ensure the synchronism of the pulled multi-view frames, so that the system synchronizes through the timestamp information in each multi-view video stream. The pictures with the same timestamp are pictures with different visual angles collected at the same moment.
The computing server 2 provides an application instance for each user, and after receiving an interactive instruction of the user, the application instance of the user takes out a view picture requested by the user from a time-synchronized multi-view data queue and transmits the view picture to the user for decoding and displaying through a WebRTC protocol.
Since the computing server 2 creates a separate application instance for each user to serve the user, when the number of users increases dramatically, the number of user instances increases accordingly, i.e., the required computing resources also increase. The computational complexity of the application instance cannot be very high. Because the users have difference only when the visual angles are switched, the original multi-visual-angle video code stream can be multiplexed under the condition that the visual angles are not changed, and encoding and decoding calculation for each user is not needed. Therefore, the system adopts the free visual angle switching method of reference frame replacement, and the application example only carries out the recombination operation of data packets on the multi-visual angle video stream in the method and does not carry out the coding and decoding operation, thereby greatly reducing the computational complexity and being beneficial to the transverse expansion of the application.
The computing server 2 provides an application instance for each user, and after receiving an interactive instruction of the user, the application instance of the user takes out a view picture requested by the user from a queue of multi-view data after time synchronization, and accesses the random access reference frame after the original view video stream; and when the switching is finished, accessing the target visual angle video stream after the random access reference frame to obtain the single visual angle video stream. Referring to fig. 4 in particular, fig. 4 shows a schematic diagram of fast view switching based on reference frame replacement in this embodiment. Assuming that the current View angle of the user is View i, before the View angle is switched, the computing server 2 directly uses the original frame in the general stream of View i as the output frame after packet reassembly. When the computing server 2 receives the user switching View angle signal, switching from View I to View M, the computing server 2 switches to the synchronous I frame in the RARF stream of View I +1 with zero delay, and takes one frame of output frame after the I frame user packet is recombined from the RARF stream of View I +1 to View M according to time. When the View switching is finished, the computing server 2 then multiplexes the frames in the View M general stream as the reassembled output frames.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, improvements, equivalents and the like that fall within the spirit and principle of the present invention are intended to be included within the scope of the present invention.
Claims (10)
1. A method of free view switching, the method comprising:
acquiring a multi-view video stream;
acquiring a visual angle switching signal according to an interactive instruction triggered by a user;
and switching in a random access reference frame between the original visual angle video stream and the target visual angle video stream according to the visual angle switching signal, and generating and transmitting a single visual angle video stream.
2. The method of claim 1, wherein after the obtaining the multi-view video stream, further comprising:
acquiring a time stamp of each frame image in each single-view video stream in the multi-view video stream;
and performing frame synchronization on the plurality of single-view video streams according to the time stamps.
3. The method according to claim 1, wherein the obtaining of the view switching signal according to the interactive instruction triggered by the user comprises:
determining the switched target visual angle information according to user sliding, input, gestures or clicking operations;
and determining the time information of the switching according to the sliding, input, gesture or clicking operation of the user.
4. The free viewing angle switching method according to claim 3,
when a user slides a screen leftwards or upwards, taking the previous visual angle of the current visual angle along one direction as target visual angle information after the target visual angle is switched;
and when the user slides the screen to the right or downwards, taking the next visual angle of the current visual angle along one direction as the target visual angle information after the target visual angle is switched.
5. The method of claim 1, wherein the switching a random access reference frame between an original view video stream and a target view video stream according to a view switching signal, and generating and outputting a single view video stream comprises:
when switching occurs, accessing the random access reference frame after the original visual angle video stream;
and when the switching is finished, accessing the target visual angle video stream after the random access reference frame to obtain the single visual angle video stream.
6. The method of claim 5, wherein accessing the random access reference frame after the original view video stream comprises:
creating an instance;
according to the switching visual angle information, preselecting one or more random access code streams of single visual angle video streams from the original visual angle to the target visual angle in the multi-visual angle video streams;
extracting a corresponding random access reference frame from the pre-selected random access code stream;
and accessing the corresponding random access reference frame behind the original view video stream.
7. The method according to claim 6, wherein the random access code stream is composed of the key frames of the single-view video stream corresponding thereto in chronological order according to time stamps.
8. The method according to any of claims 1 to 7, wherein the single-view video stream is transmitted based on a WebRTC protocol.
9. An interactive freeview playback system, comprising:
a video stream acquisition device in communication with the computing server;
one or more computing servers for acquiring a multi-view video stream;
a signal server in communication connection with the computing server;
a streaming media server communicatively coupled to the computing server;
more than one client communicatively connected with the signaling server;
the computing server acquires a multi-view video stream from the video stream acquisition device; the client transmits an interactive instruction to the computing server through the signal server; and the computing server outputs a single-channel video stream to the streaming media server according to the interactive instruction, and the streaming media server sends the single-channel video stream to the client.
10. The interactive freeview playback system according to claim 9, wherein said video stream capturing means includes:
more than one video data acquisition module for acquiring multi-view video stream;
more than one image correction module for correcting image data;
more than one video coding plug flow module; and
RTMP streaming media server.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210798544.1A CN115174942A (en) | 2022-07-08 | 2022-07-08 | Free visual angle switching method and interactive free visual angle playing system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210798544.1A CN115174942A (en) | 2022-07-08 | 2022-07-08 | Free visual angle switching method and interactive free visual angle playing system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115174942A true CN115174942A (en) | 2022-10-11 |
Family
ID=83490852
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210798544.1A Pending CN115174942A (en) | 2022-07-08 | 2022-07-08 | Free visual angle switching method and interactive free visual angle playing system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115174942A (en) |
Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101155298A (en) * | 2006-09-25 | 2008-04-02 | 华为技术有限公司 | Method and system for implementing fast switching television channel |
| CN102510541A (en) * | 2011-12-30 | 2012-06-20 | Tcl数码科技（深圳）有限责任公司 | Multi-screen interaction video and audio content switching method and media player |
| CN102761773A (en) * | 2012-07-03 | 2012-10-31 | Ut斯达康通讯有限公司 | Method for improving user experience of IPTV (Internet Protocol Television) service by using I frames |
| CN103856806A (en) * | 2012-11-28 | 2014-06-11 | 腾讯科技（北京）有限公司 | Video stream switching method, device and system |
| CN111355954A (en) * | 2018-12-20 | 2020-06-30 | 皇家Kpn公司 | Processing video data for a video player device |
| CN111355967A (en) * | 2020-03-11 | 2020-06-30 | 叠境数字科技（上海）有限公司 | Video live broadcast processing method, system, device and medium based on free viewpoint |
| CN111818347A (en) * | 2020-06-01 | 2020-10-23 | 深圳市佳创视讯技术股份有限公司 | Quick visual angle switching method based on reference frame replacement |
| CN111866525A (en) * | 2020-09-23 | 2020-10-30 | 腾讯科技（深圳）有限公司 | Multi-view video playing control method and device, electronic equipment and storage medium |
| CN111935557A (en) * | 2019-05-13 | 2020-11-13 | 华为技术有限公司 | Video processing method, device and system |
| CN112437319A (en) * | 2020-11-10 | 2021-03-02 | 杭州叙简科技股份有限公司 | Method for switching multiple video streams based on webrtc |
| CN113256491A (en) * | 2021-05-11 | 2021-08-13 | 北京奇艺世纪科技有限公司 | Free visual angle data processing method, device, equipment and storage medium |
| CN114035672A (en) * | 2020-07-20 | 2022-02-11 | 华为技术有限公司 | Video processing method and related equipment for virtual reality VR scene |
| CN114222156A (en) * | 2021-12-14 | 2022-03-22 | 中国建设银行股份有限公司 | Video editing method, video editing device, computer equipment and storage medium |
| CN114584769A (en) * | 2020-11-30 | 2022-06-03 | 华为技术有限公司 | Visual angle switching method and device |
-
2022
- 2022-07-08 CN CN202210798544.1A patent/CN115174942A/en active Pending
Patent Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101155298A (en) * | 2006-09-25 | 2008-04-02 | 华为技术有限公司 | Method and system for implementing fast switching television channel |
| CN102510541A (en) * | 2011-12-30 | 2012-06-20 | Tcl数码科技（深圳）有限责任公司 | Multi-screen interaction video and audio content switching method and media player |
| CN102761773A (en) * | 2012-07-03 | 2012-10-31 | Ut斯达康通讯有限公司 | Method for improving user experience of IPTV (Internet Protocol Television) service by using I frames |
| CN103856806A (en) * | 2012-11-28 | 2014-06-11 | 腾讯科技（北京）有限公司 | Video stream switching method, device and system |
| CN111355954A (en) * | 2018-12-20 | 2020-06-30 | 皇家Kpn公司 | Processing video data for a video player device |
| CN111935557A (en) * | 2019-05-13 | 2020-11-13 | 华为技术有限公司 | Video processing method, device and system |
| CN111355967A (en) * | 2020-03-11 | 2020-06-30 | 叠境数字科技（上海）有限公司 | Video live broadcast processing method, system, device and medium based on free viewpoint |
| CN111818347A (en) * | 2020-06-01 | 2020-10-23 | 深圳市佳创视讯技术股份有限公司 | Quick visual angle switching method based on reference frame replacement |
| CN114035672A (en) * | 2020-07-20 | 2022-02-11 | 华为技术有限公司 | Video processing method and related equipment for virtual reality VR scene |
| CN111866525A (en) * | 2020-09-23 | 2020-10-30 | 腾讯科技（深圳）有限公司 | Multi-view video playing control method and device, electronic equipment and storage medium |
| CN112437319A (en) * | 2020-11-10 | 2021-03-02 | 杭州叙简科技股份有限公司 | Method for switching multiple video streams based on webrtc |
| CN114584769A (en) * | 2020-11-30 | 2022-06-03 | 华为技术有限公司 | Visual angle switching method and device |
| CN113256491A (en) * | 2021-05-11 | 2021-08-13 | 北京奇艺世纪科技有限公司 | Free visual angle data processing method, device, equipment and storage medium |
| CN114222156A (en) * | 2021-12-14 | 2022-03-22 | 中国建设银行股份有限公司 | Video editing method, video editing device, computer equipment and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 张文杰: "基于Tile的全景视频自适应传输技术研究", 《中国优秀硕士学位论文全文数据库》, pages: 49 - 55 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11284055B2 (en) | Method and an apparatus and a computer program product for video encoding and decoding | |
| KR100556826B1 (en) | System and Method of Internet Broadcasting for MPEG4 based Stereoscopic Video | |
| EP2234406A1 (en) | A three dimensional video communication terminal, system and method | |
| JP2020519094A (en) | Video playback method, device, and system | |
| CN110149542B (en) | Transmission control method | |
| KR100703715B1 (en) | Multiview 3D video transmission/receiving system | |
| CN112219403B (en) | Rendering perspective metrics for immersive media | |
| KR102361314B1 (en) | Method and apparatus for providing 360 degree virtual reality broadcasting services | |
| CN111447503A (en) | Viewpoint switching method, server and system for multi-viewpoint video | |
| KR101861929B1 (en) | Providing virtual reality service considering region of interest | |
| WO2021218573A1 (en) | Video playing method, apparatus and system, and computer storage medium | |
| CN109451293B (en) | Self-adaptive stereoscopic video transmission system and method | |
| KR20170130883A (en) | Method and apparatus for virtual reality broadcasting service based on hybrid network | |
| CN102843566A (en) | Communication method and equipment for three-dimensional (3D) video data | |
| Zeng et al. | A new architecture of 8k vr fov video end-to-end technology | |
| CN115174942A (en) | Free visual angle switching method and interactive free visual angle playing system | |
| CN115174943B (en) | Free view angle playing method and system with edge cloud cooperation and client self-adaption | |
| Fautier | Next-generation video compression techniques | |
| CN107454463A (en) | A kind of method and device across the synchronous broadcasting video resource of screen based on wired connection | |
| Debono et al. | 3D video coding and transmission | |
| WO2022222533A1 (en) | Video playing method, apparatus and system, and computer-readable storage medium | |
| CN116980688A (en) | Video processing method, apparatus, computer, readable storage medium, and program product | |
| KR20130063603A (en) | Methods of coding additional frame and apparatuses for using the same |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |