WO2021208321A1 - 一种多视点视频的视点切换方法和系统 - Google Patents

一种多视点视频的视点切换方法和系统 Download PDF

Info

Publication number
WO2021208321A1
WO2021208321A1 PCT/CN2020/111970 CN2020111970W WO2021208321A1 WO 2021208321 A1 WO2021208321 A1 WO 2021208321A1 CN 2020111970 W CN2020111970 W CN 2020111970W WO 2021208321 A1 WO2021208321 A1 WO 2021208321A1
Authority
WO
WIPO (PCT)
Prior art keywords
viewpoint
terminal
server
video
frame
Prior art date
Application number
PCT/CN2020/111970
Other languages
English (en)
French (fr)
Inventor
田路
夏洪升
Original Assignee
烽火通信科技股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 烽火通信科技股份有限公司 filed Critical 烽火通信科技股份有限公司
Publication of WO2021208321A1 publication Critical patent/WO2021208321A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/81—Monomedia components thereof
    • H04N21/816—Monomedia components thereof involving special video data, e.g 3D video
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21—Server components or server architectures
    • H04N21/218—Source of audio or video content, e.g. local disk arrays
    • H04N21/21805—Source of audio or video content, e.g. local disk arrays enabling multiple viewpoints, e.g. using a plurality of cameras
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/231—Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion
    • H04N21/23106—Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion involving caching operations
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/433—Content storage operation, e.g. storage operation in response to a pause request, caching operations
    • H04N21/4331—Caching operations, e.g. of an advertisement for later insertion during playback
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/65—Transmission of management data between client and server
    • H04N21/658—Transmission by the client directed to the server
    • H04N21/6587—Control parameters, e.g. trick play commands, viewpoint selection
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/85—Assembly of content; Generation of multimedia applications
    • H04N21/854—Content authoring
    • H04N21/8547—Content authoring involving timestamps for synchronizing content

Definitions

  • the present invention belongs to the technical field of video playback, and more specifically, relates to a viewpoint switching method and system for a multi-viewpoint video.
  • Multi-view video is obtained by a camera array composed of multiple cameras shooting the same scene or object from different positions and angles.
  • multi-view video can provide information on the same scene or object from multiple angles. , So that people can get a richer visual experience.
  • the multi-viewpoint video data collected by the camera array is encoded by the encoding server and then transmitted to the CDN (Content Delivery Network) server.
  • the CDN server streams the video and sends it to the user terminal (such as TV set-top boxes, PCs, and mobile phones). Wait).
  • the user can freely switch the viewpoints in the multi-view video by means of direction keys, dragging, etc. to view the subject from a different angle, achieving a better viewing experience than traditional single-view videos.
  • the encoding server encodes and compresses the multi-view original video stream collected by the camera array to form a compressed multi-channel video stream.
  • the compressed video frames are generally divided into three types of I/P/B frames.
  • I frame is a key frame, which is the first frame that constitutes a group of frames (GOP, Group of Picture); P frame is a future single prediction frame, which only stores the difference with the previous decompressed picture, and B frame is a bidirectional prediction frame , In addition to referring to the previously decompressed picture, it will also refer to the picture information in the next frame.
  • I frames include IDR (Instantaneous Decoding Refresh) frames. The function of IDR frames is to refresh them immediately to prevent errors from propagating. Starting from the IDR frame, a new sequence is recalculated to start coding, which is different from ordinary I frames. With random access capability, from the random access video stream, the player can always play from an IDR frame.
  • the CDN server completes the streaming and packaging of multi-viewpoint video content, and according to user needs, takes out the video stream of a certain viewpoint and sends it to the terminal.
  • the CDN server generally has a buffer area for temporarily storing video data
  • the terminal also has a buffer area for temporarily storing video data.
  • the total length of the video content of the CDN server and the terminal buffer may reach the second or even ten second level, so The content played by the terminal and the content currently processed by the CDN server will have a large delay in time.
  • the purpose of the present invention is to provide a viewpoint switching method and system for multi-view video, which can play the video of the current and next viewpoints at the switching time point when the viewpoints of the multi-view video are switched. Perform processing to realize the natural connection of videos between different viewpoints at the same time point to improve the viewpoint switching effect.
  • a method for switching viewpoints of multi-view video including:
  • the terminal After receiving the user's viewpoint switching request, the terminal checks whether there are IDR frames within the preset time period of the content cached locally by the terminal of the current viewpoint;
  • the time point of the IDR frame is added to the viewpoint switching request and reported to the server;
  • the time point after the preset time period is added to the viewpoint switching request and reported to the server;
  • the time point at the end of the terminal local cache is added to the viewpoint switching request and reported to the server;
  • the terminal after the terminal reports the viewpoint switching request to the server, it also receives the video stream after the viewpoint switching sent by the server, which is specifically:
  • the server receives the viewpoint switching request sent by the terminal
  • the video frame at the corresponding point in time of the next viewpoint to be switched to on the server is combined with the relevant video frame of the GOP at that point in time to calculate Synthesize an IDR frame, replace the original non-IDR frame at this point in time, clear the server cache, stream and package from this point in time and send it to the terminal.
  • each channel of video has an IDR frame at each time node of n seconds, and the IDR frames of each channel of video are aligned at the time point, and n is a preset value .
  • said combining the video frame at the corresponding time point of the next viewpoint to be switched to on the server with the relevant video frame of the GOP at the time point to calculate and synthesize an IDR frame is specifically: All the coded frames in the GOP to which the time point belongs are decompressed to generate original video data, and the original video data is intercepted according to the offset of the time point relative to the GOP start time, and the intercepted original video data is compressed with this time point as the starting point Encoding, the first frame generates an IDR frame to replace the encoded frame in the original GOP.
  • an extension field is added before each frame of original video data of the video content for viewpoint switching, and the extension field includes camera viewpoint information and time stamp information.
  • the camera viewpoint information includes: the number of the current viewpoint, the total number of viewpoints, the frame information in the message, and the frame type.
  • the value of n is 2 seconds, and the value of the preset duration is 500 milliseconds.
  • a viewpoint switching system for multi-viewpoint video including a terminal and a server, wherein:
  • the terminal is configured to, after receiving a user viewpoint switching request, check whether there are IDR frames within a preset time period of the content cached locally by the terminal of the current viewpoint;
  • the time point of the IDR frame is added to the viewpoint switching request and reported to the server;
  • the time point after the preset time period is added to the viewpoint switching request and reported to the server;
  • the time point at the end of the terminal local cache is added to the viewpoint switching request and reported to the server;
  • the terminal after the terminal reports the viewpoint switching request to the server, it also receives the video stream after the viewpoint switching sent by the server, which is specifically:
  • the server receives the viewpoint switching request sent by the terminal
  • the video frame at the corresponding point in time of the next viewpoint to be switched to on the server is combined with the relevant video frame of the GOP at that point in time to calculate Synthesize an IDR frame, replace the original non-IDR frame at this point in time, clear the server cache, stream and package from this point in time and send it to the terminal.
  • each channel of video has an IDR frame at each time node of n seconds, and the IDR frames of each channel of video are aligned at the time point, and n is a preset value .
  • the present invention realizes the alignment of the time stamps of different viewpoints in the multi-view video switching, so that the connection of the front and rear viewpoints is synchronized in time;
  • the present invention can ensure that there are no blurring, black screens, etc. during the view switch connection process, and improve the user experience of view switch;
  • the present invention can improve the smoothness of playback in the process of view connection before and after switching by processing the video caching situation of the CDN server and the set-top box terminal.
  • Figure 1 is a schematic diagram of multi-viewpoint video collecting multi-viewpoint video through a camera array
  • FIG. 2 is a business scenario architecture diagram of multi-viewpoint video playback in an embodiment of the present invention
  • Figure 3 is a schematic diagram of IDR/P/B frames for multi-view video coding
  • FIG. 4 is a schematic diagram of a viewpoint switching method for a multi-view video in an embodiment of the present invention.
  • FIG. 5 is a flowchart of the terminal and the CDN server side when the viewpoints of the multi-view video are switched in the embodiment of the present invention
  • Fig. 6 is a schematic structural diagram of a viewpoint switching system for a multi-view video in an embodiment of the present invention.
  • the user controls the terminal to switch viewpoints (for example, through the arrow keys of the TV set-top box remote control, etc.), and the CDN server responds to the user's request to switch the screen from the current viewpoint to the screen from the next viewpoint.
  • the user expects The current viewpoint played by the terminal and the next viewpoint are aligned at the time point of switching.
  • I/P/B image frames
  • I/P/B image frames
  • the picture of may be a frame type that needs to be associated with other frames, and the next view-point playback frame has no correlation with the previous view-point frame, resulting in a blurred or black screen.
  • the video content of the current viewpoint may still be stored in the cache for a period of time, and the playback of the next viewpoint video may take a long time to be displayed.
  • Figure 2 is a business scenario architecture diagram of multi-viewpoint video playback in an embodiment of the present invention.
  • the present invention adopts a multi-viewpoint video end-to-end system, which is divided into two parts: live broadcast and on-demand.
  • Video content is compressed and encoded by the encoding server, then streamed and packaged by the CDN server, and sent to the terminal; while the on-demand content is directly injected into the CDN server as a file, processed by the CDN server, and then streamed and packaged and sent to the terminal terminal.
  • the camera array collects raw video data collected by a total of x cameras.
  • the encoding server performs compression encoding, as shown in Figure 3, IDR frames are generated for each video stream in a period of n seconds, that is, each channel
  • the video has an IDR frame at each time node of n seconds, that is, the length of the GOP (Group of Pictures) of the encoded video is n seconds, and the IDR frames of each video are aligned at the time point to generate a total of x channels of coded
  • n is a preset value. It is verified by testing that the value of n is 2 seconds.
  • the on-demand multi-view video files are injected into the CDN server and stored by the CDN server. Since the encoding methods of externally obtained video files may be different, the injected video files need to be checked and transcoded by the CDN server. The same takes n seconds. Generate IDR frames periodically and keep the IDR frames of each channel aligned at the time point.
  • the compressed video content of IDR frames is generated periodically, and the CDN server uses the streaming media protocol to stream and package the video of each viewpoint.
  • the CDN server adds an 8-byte extension field before the original video data of each frame of the video content, including camera viewpoint information and time stamp information for subsequent viewpoint switching purposes.
  • the camera viewpoint information can be: the number of the current viewpoint, the total number of viewpoints, the frame information in the message, and the frame type.
  • 1 byte is used to record the number of the current camera (viewpoint); 1 byte is used to record the total number of cameras (viewpoint); the FP field occupies 4 bits, and a value of 1 means one Frames are in the message.
  • the value is 2, it means the head of a frame, when the value is 3, it means the middle of a frame, and when the value is 4 it means the end of a frame; the FT field occupies 4 bits, and the value is 1.
  • the frame When it means that the frame is an IDR frame, when it is 2, it means it is a P-frame, when it is 3 it means it is a B-frame, and when it is 4 it means it is an I-frame other than IDR frames; the last 4 Bytes are used to record the timestamp in ms.
  • the terminal plays a video, it needs to strip the content of the extension field before playing the video frame.
  • a terminal such as a TV set-top box, a PC, a mobile phone, etc.
  • viewpoint switching through control methods such as direction buttons, in order to switch the currently playing viewpoint to another viewpoint to play the video.
  • the video frame played by the new viewpoint needs to be switched to IDR frame to ensure that the new viewpoint playback video is irrelevant to the previous viewpoint playback video. This will cause the screen to become blurred or black.
  • the front and rear viewpoints can be connected normally at the time point. Otherwise, the visual experience will be poor due to the time jump of the front and rear viewpoints.
  • the data from the previous viewpoint is processed.
  • Fig. 4 it is a method for switching viewpoints of multi-view video in an embodiment of the present invention, including:
  • the terminal After receiving the user viewpoint switching request, the terminal checks whether there are IDR frames within the preset time period of the content cached locally by the terminal of the current viewpoint;
  • the CDN when the CDN receives the handover request, there may be video content from the previous viewpoint in the CDN's local cache. This part of the content does not need to be sent to the terminal. Therefore, the CDN also needs to clear the local cache when the handover request is received.
  • the terminal When the terminal receives the user's command to switch viewpoints, it checks the extended field timestamp information of each frame of the content cached locally by the terminal. If there are IDR frames in the local cache content of the current viewpoint for the preset duration (for example, m milliseconds), it will The time point of the IDR frame is added to the viewpoint switching request and reported to the CDN server; if there is no IDR frame within m milliseconds of the local cache content of the terminal of the current viewpoint, and the cache content is greater than m milliseconds, the time point after m milliseconds is added to the viewpoint switching request , Report to the CDN server; if there is no IDR frame within the preset time period of the local cache content of the terminal of the current viewpoint, and the cache content is less than m milliseconds, the time point at the end of the terminal’s local cache is added to the viewpoint switching request and reported to the CDN server. In addition, the local cache of the terminal after the time point of reporting to the server is cleared.
  • the CDN server receives the viewpoint switch request sent by the terminal, and if it finds that there is an IDR frame in the reported terminal buffer for m milliseconds, it will directly find out the next viewpoint to be switched from stored on the CDN server according to the timestamp information in the extended field.
  • the IDR frame at the time point Corresponding to the IDR frame at the time point, after the local buffer is cleared, it is streamed and packaged from that time point and sent to the terminal; if the IDR frame does not exist before the switching time point of the terminal that sent the report, you need to switch to the saved CDN server
  • the video frame at the corresponding time point of the next viewpoint combined with the relevant video frame of the GOP at the time point, calculates and synthesizes an IDR frame, replaces the original non-IDR frame at that time point, and streams and packs it from this point in time, Send to the terminal.
  • the method of calculating and synthesizing an IDR frame can be: decompressing all the coded frames in the GOP (n seconds) to which the time point belongs to generate n seconds of original video data, and relative GOP starting point according to the time point Time offset, intercept the original video data, take this time point as the starting point, compress and encode the intercepted original video data, generate IDR frame from the first frame, and replace the coded frame in the original GOP.
  • the terminal When switching viewpoints, the terminal starts from the switching time point and plays the content of the original viewpoint up to m milliseconds, and then plays the video content of the new viewpoint, and the front and back viewpoints are aligned at the time point of the switching. Tests have verified that the value of m milliseconds is 500 milliseconds.
  • the CDN when the CDN receives the handover request, there may be video content from the previous viewpoint in the CDN local cache. This part of the content does not need to be sent to the terminal, so the CDN also needs to clear the local cache when the handover request is received. .
  • FIG. 5 is a flowchart of the terminal and CDN server side during multi-view video switching in the embodiment of the present invention; the embodiment of the present invention describes the processing performed by the terminal and the CDN server to achieve the front and rear view connection playback during the multi-view video switching.
  • the terminal (such as a TV set-top box, mobile phone, PC, etc.) is playing the video content of the multi-view video viewpoint A;
  • S102 The user initiates a viewpoint switching operation through the terminal direction keys and other methods (such as the direction keys of the set-top box remote control, mobile phone sliding, and PC mouse dragging);
  • S103 The terminal parses the user's operation request, and judges that the next viewpoint switched to is B;
  • S104 The terminal checks the local cache content for the current viewpoint A, and finds whether there are IDR frames in the video frames within m milliseconds;
  • S105 The terminal finds that the video frame content of viewpoint A locally buffered by the terminal has an IDR frame within m milliseconds, and then reports the time point information of the IDR frame and the request to switch the viewpoint to viewpoint B to the CDN server, and at the same time clears the terminal local buffer The content of the IDR frame time and after;
  • the terminal finds that the local content length is greater than m milliseconds, and then reports the time point information of the video frame at m milliseconds, the IDR frame generation request, and the viewpoint switching request to the CDN server, and at the same time clears the terminal's local cache of the content at and after the m millisecond time point;
  • S108 The terminal finds that the length of the local content is less than m milliseconds, and then reports the time point information, IDR frame generation request, and viewpoint switching request at the end of the terminal's local cache to the CDN server;
  • S109 The terminal receives and plays the content of the next viewpoint B after the switch.
  • the CDN server checks the multi-view video content to ensure that the video of each viewpoint has a period of n seconds, and IDR frames are fixed. If they do not meet the requirements, transcoding is required to generate IDR frames with a period of n seconds to replace the original frames;
  • the CDN server clears the local cache of the CDN server after receiving the terminal's switch request to switch the viewpoint to B and the time point information of the IDR frame buffered by the terminal, and directly provides the correspondence of the next viewpoint B according to the time point information Time point video stream, no need to calculate and synthesize, the first frame is the IDR frame;
  • FIG. 6 is a schematic structural diagram of a viewpoint switching system for multi-view video in an embodiment of the present invention.
  • the system includes a terminal and a server, wherein:
  • the terminal is configured to, after receiving a user viewpoint switching request, check whether there are IDR frames within a preset time period of the content cached locally by the terminal of the current viewpoint;
  • the time point of the IDR frame is added to the viewpoint switching request and reported to the server;
  • the time point after the preset time period is added to the viewpoint switching request and reported to the server;
  • the time point at the end of the terminal local cache is added to the viewpoint switching request and reported to the server;
  • the terminal reports the viewpoint switching request to the server, it also receives the video stream after the viewpoint switching sent by the server, which is specifically:
  • the server receives the viewpoint switching request sent by the terminal
  • the video frame at the corresponding point in time of the next viewpoint to be switched to on the server is combined with the relevant video frame of the GOP at that point in time to calculate Synthesize an IDR frame, replace the original non-IDR frame at this point in time, clear the server cache, stream and package from this point in time and send it to the terminal.
  • each channel of video has an IDR frame at each time node of n seconds, the IDR frames of each channel of video are aligned at the time point, and n is a preset value.
  • the computer program product includes one or more computer instructions.
  • the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.
  • the computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website, computer, server, or data center.
  • the computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or a data center integrated with one or more available media.
  • the usable medium may be a magnetic medium (for example, a floppy disk, a hard disk, and a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk (SSD)).

Abstract

本发明公开了一种多视点视频的视点切换方法，包括：终端接收到视点切换请求后，检查当前视点的终端本地缓存内容预设时长内是否存在IDR帧；如果预设时长内存在IDR帧，则将该IDR帧的时间点加入视点切换请求，上报服务器；如果预设时长内不存在IDR帧，且缓存内容大于预设时长，则将预设时长后的时间点加入视点切换请求，上报服务器；如果预设时长内不存在IDR帧，且缓存内容不足预设时长，则将缓存末端的时间点加入视点切换请求，上报服务器；清空上报服务器视点切换请求中的时间点之后的终端本地缓存。本发明实现了多视点视频切换中的不同视点时间戳的对齐，使得前后视点的衔接在时间上同步。本发明还公开了相应的多视点视频的视点切换系统。

Description

一种多视点视频的视点切换方法和系统 技术领域
本发明属于视频播放技术领域，更具体地，涉及一种多视点视频的视点切换方法和系统。
背景技术
多视点视频由多个相机组成的相机阵列从不同位置和角度对同一场景或物体进行拍摄得到，如图1所示，相对于单视点视频，多视点视频可以提供同一场景或物体多角度的信息，使人们获得更为丰富的视觉体验。相机阵列采集的多视点视频数据经过编码服务器的编码后传入CDN(Content Delivery Network，内容分发网络)服务器，由CDN服务器将视频进行流化处理，发送到用户终端(例如电视机顶盒、PC、手机等)。用户通过终端观看时，可以通过如方向键、拖动等方式自由切换多视点视频中的视点以变换角度观看被摄对象的情况，达到相比传统单个视点视频更好的观看体验。
编码服务器对相机阵列采集的多视点原始视频码流进行编码压缩后，形成压缩的多路视频码流，压缩的视频帧一般分为I/P/B三类帧。I帧为关键帧，是构成一个帧组(GOP，Group of Picture)的第一个帧；P帧是未来单项预测帧，只储存与之前一个已解压画面的差值，B帧是双向预测帧，除了参考之前解压过了的画面外，也会参考后面一帧中的画面信息。I帧中包括IDR(Instantaneous Decoding Refresh，即时解码刷新)帧，IDR帧的作用是立刻刷新，使错误不致传播，从IDR帧开始，重新算一个新的序列开始编码，其具有普通I帧所不具有的随机访问能力，从随机存取的视频流中，播放器永远可以从一个IDR帧播放。
CDN服务器完成对多视点视频内容的流化打包，并针对用户需求，取 出某一个视点的视频码流发送给终端。在发送过程中，CDN服务器一般会存在临时保存视频数据的缓存区，同时终端也存在临时保存视频数据的缓存区，CDN服务器及终端缓冲区的视频内容总长可能达到秒级甚至十秒级，故而终端播放的内容与CDN服务器当前处理的内容在时间点上会有较大延迟。
发明内容
针对现有技术的以上缺陷或改进需求，本发明的目的在于提供一种多视点视频的视点切换方法和系统，在多视点视频视点切换时，对切换时间点的当前和下个视点的视频播放进行处理，实现同一时间点上不同视点间视频自然的衔接以完善视点切换效果。
为实现上述目的，按照本发明的一个方面，提供了一种多视点视频的视点切换方法，包括：
终端接收到用户视点切换请求后，检查当前视点的终端本地缓存内容预设时长内是否存在IDR帧；
如果当前视点的终端本地缓存内容预设时长内存在IDR帧，则将该IDR帧的时间点加入视点切换请求，上报服务器；
如果当前视点的终端本地缓存内容预设时长内不存在IDR帧，且缓存内容大于预设时长，则将预设时长后的时间点加入视点切换请求，上报服务器；
如果当前视点的终端本地缓存内容预设时长内不存在IDR帧，且缓存内容不足预设时长，则将终端本地缓存末端的时间点加入视点切换请求，上报服务器；
清空上报服务器视点切换请求中的时间点之后的终端本地缓存。
本发明的一个实施例中，所述终端向服务器上报视点切换请求后，还接收服务器发送的切换视点后的视频流，具体为：
服务器接收终端发送的视点切换请求；
如果终端本地缓存内容预设时长内存在IDR帧，则直接从服务器上保存的要切换到的下个视点视频中，找出对应时间点的IDR帧，清空服务器缓存后，从该时间点流化打包并发送到终端；
如果终端本地缓存内容在上报视点切换请求中的时间点前不存在IDR帧，则将服务器上要切换到的下个视点的对应时间点视频帧结合该时间点所处GOP的相关视频帧，计算合成出一个IDR帧，替换原有的该时间点的非IDR帧，清空服务器缓存后，从该时间点流化打包并发送到终端。
本发明的一个实施例中，在所述多视点视频中，每路视频在每个n秒的时间节点上有一个IDR帧，各路视频的IDR帧在时间点上对齐，n为预设值。
本发明的一个实施例中，所述将服务器上要切换到的下个视点的对应时间点视频帧结合该时间点所处GOP的相关视频帧，计算合成出一个IDR帧，具体为：将该时间点所属的GOP内所有的编码帧进行解压缩处理生成原始视频数据，并根据该时间点相对GOP起点时间的偏移截取原始视频数据，以该时间点为起点对截取的原始视频数据进行压缩编码，首帧生成IDR帧替换原GOP内的编码帧。
本发明的一个实施例中，视频内容的每帧原始视频数据前增加扩展字段用于视点切换，所述扩展字段包括相机视点信息和时间戳信息。
本发明的一个实施例中，所述相机视点信息包括：当前视点的编号，视点的总数，报文中帧信息，帧类型。
本发明的一个实施例中，所述n取值为2秒，所述预设时长取值为500毫秒。
按照本发明的另一方面，还提供了一种多视点视频的视点切换系统，包括终端和服务器，其中：
所述终端用于在接收到用户视点切换请求后，检查当前视点的终端本地缓存内容预设时长内是否存在IDR帧；
如果当前视点的终端本地缓存内容预设时长内存在IDR帧，则将该IDR帧的时间点加入视点切换请求，上报服务器；
如果当前视点的终端本地缓存内容预设时长内不存在IDR帧，且缓存内容大于预设时长，则将预设时长后的时间点加入视点切换请求，上报服务器；
如果当前视点的终端本地缓存内容预设时长内不存在IDR帧，且缓存内容不足预设时长，则将终端本地缓存末端的时间点加入视点切换请求，上报服务器；
清空上报服务器视点切换请求中的时间点之后的终端本地缓存。
本发明的一个实施例中，所述终端向服务器上报视点切换请求后，还接收服务器发送的切换视点后的视频流，具体为：
所述服务器接收终端发送的视点切换请求；
如果终端本地缓存内容预设时长内存在IDR帧，则直接从服务器上保存的要切换到的下个视点视频中，找出对应时间点的IDR帧，清空服务器缓存后，从该时间点流化打包并发送到终端；
如果终端本地缓存内容在上报视点切换请求中的时间点前不存在IDR帧，则将服务器上要切换到的下个视点的对应时间点视频帧结合该时间点所处GOP的相关视频帧，计算合成出一个IDR帧，替换原有的该时间点的非IDR帧，清空服务器缓存后，从该时间点流化打包并发送到终端。
本发明的一个实施例中，在所述多视点视频中，每路视频在每个n秒的时间节点上有一个IDR帧，各路视频的IDR帧在时间点上对齐，n为预设值。
总体而言，通过本发明所构思的以上技术方案与现有技术相比，具有如下有益效果：
(1)本发明中实现了多视点视频切换中的不同视点时间戳的对齐，使得前后视点的衔接在时间上同步；
(2)本发明通过对IDR帧的判断和处理，可以保证在视点切换衔接过程中不出现花屏、黑屏等情况，提高视点切换的用户体验；
(3)本发明通过对CDN服务器和机顶盒终端的视频缓存情况的处理，可以提高切换前后视点衔接过程中的播放流畅度。
附图说明
图1为多视点视频通过相机阵列采集多视点视频的示意图；
图2为本发明实施例中的多视点视频播放的业务场景架构图；
图3为多视点视频编码IDR/P/B帧示意图；
图4为本发明实施例中多视点视频的视点切换方法示意图；
图5为本发明实施例中多视点视频视点切换时终端及CDN服务器侧流程图；
图6为本发明实施例中多视点视频的视点切换系统结构示意图。
具体实施方式
为了使本发明的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本发明进行进一步详细说明。应当理解，此处所描述的具体实施例仅仅用以解释本发明，并不用于限定本发明。此外，下面所描述的本发明各个实施方式中所涉及到的技术特征只要彼此之间未构成冲突就可以相互组合。
本发明要解决的技术问题是：
在多视点视频播放过程中，用户控制终端切换视点(例如通过电视机顶盒遥控器的方向键等)，CDN服务器响应用户的请求，将当前视点播放的画面切换到下一个视点播放的画面，用户期待终端播放的当前视点与下个视点在切换的时间点上对齐。
视频编码中，存在多种类型(I/P/B)的图像帧，其中只有I帧中的IDR帧是独立不参考其它帧的类型，用户请求视点切换时，下个视点对应切换时间点播放的画面可能为需要关联其它帧的帧类型情况，而下个视点播放帧 因与前一视点帧之间无关联性，造成花屏、黑屏的结果。
另外由于机顶盒终端和服务器的缓存功能，可能存在用户发出切换命令时，缓存中仍保存着一段时间当前视点的视频内容，下个视点视频的播放需要经过较长时间才能显示。
由于视频帧特性和设备缓存的因素造成前后视点的切换播放不能在同一时间点正确衔接，影响视点切换效果。
实施例1
图2为本发明实施例中的多视点视频播放的业务场景架构图，本发明采用多视点视频端到端的系统，分为直播和点播2个部分，其中直播由相机阵列实时采集对象的多视点视频内容，由编码服务器进行压缩编码，再由CDN服务器进行流化打包，发送给终端；而点播内容以文件方式直接注入到CDN服务器，由CDN服务器进行处理，再经流化打包后，发送给终端。
直播过程中，相机阵列采集共计x个相机采集的原始视频数据，在编码服务器进行压缩编码时，如图3所示，对每路的视频码流以n秒为周期生成IDR帧，即每路视频在每个n秒的时间节点上有一个IDR帧，也即编码视频的GOP(Group of Pictures)长度为n秒，各路视频的IDR帧在时间点上对齐，生成共计x路编码后的视频码流，n为预设值，经测试验证，n取值为2秒较为合适。
点播的多视点视频文件注入到CDN服务器，由CDN服务器进行存储，由于外部获取的视频文件的编码方式可能不一，需要经过CDN服务器对注入的视频文件进行检查和转码工作，同样以n秒为周期生成IDR帧，保持各路视频的IDR帧在时间点上对齐。
周期生成IDR帧的压缩视频内容，由CDN服务器对各个视点的视频使用流媒体协议进行流化打包。在打包过程中，CDN服务器在视频内容的每帧的原始视频数据前增加8个字节的扩展字段，包括相机视点信息、时 间戳信息，用于后续视点切换用途。相机视点信息可以为：当前视点的编号，视点的总数，报文中帧信息，帧类型。
例如如表1所示，1个字节用于记录当前相机(视点)的编号；1个字节用于记录相机(视点)的总数；FP字段占用4个bit位，值为1时表示一帧都在该报文中，值为2时表示一帧的头部，值为3时表示一帧的中段，值为4时表示一帧的末尾；FT字段占用4个bit位，值为1时表示该帧为IDR帧，值为2时表示该帧为P帧，值为3时表示该帧为B帧，值为4时表示该帧为除了IDR帧之外的I帧；最后4个字节用于记录以ms为单位的时间戳。终端在播放视频时，需要将扩展字段的内容剥除后播放视频帧。
表1
Figure PCTCN2020111970-appb-000001
终端(如电视机顶盒、PC、手机等)在播放某个视点视频的过程中，通过方向按键等控制方式发起视点切换，目的是将当前播放的视点切换到其他视点播放视频。在切换视点的过程中，由于不同视点的视频前后内容无参考关系，需要切换后新的视点播放的视频帧为IDR帧，才能保证新的视点播放视频与前一视点播放视频无关联性，不会因此造成花屏、黑屏的情况。同时，在切换的时间点上，需要保证前后视点在时间点上能够正常衔接，否则会因前后视点的时间跳变造成观感体验差的情况，这就需要对包括CDN服务器和终端的缓存中的前一视点的数据进行处理。
实施例2
如图4所示，为本发明实施例中一种多视点视频的视点切换方法，包 括：
S1、终端接收到用户视点切换请求后，检查当前视点的终端本地缓存内容预设时长内是否存在IDR帧；
S2、如果当前视点的终端本地缓存内容预设时长内存在IDR帧，则将该IDR帧的时间点加入视点切换请求，上报服务器；
S3、如果当前视点的终端本地缓存内容预设时长内不存在IDR帧，且缓存内容大于预设时长，则将预设时长后的时间点加入视点切换请求，上报服务器；
S4、如果当前视点的终端本地缓存内容预设时长内不存在IDR帧，且缓存内容不足预设时长，则将终端本地缓存末端的时间点加入视点切换请求，上报服务器；
S5、清空上报服务器视点切换请求中的时间点之后的终端本地缓存。
需要注意的是：上述步骤S2、S3、S4是并列关系，而非先后执行关系，即根据判断条件，S2、S3、S4三者之一执行即可。
另外，因为CDN在接收到切换请求时，可能CDN本地缓存中存在前一视点的视频内容，这部分内容无需再发送到终端，所以在在接收到切换请求时也需要CDN清空本地的缓存。
实施例3
终端接收用户切换视点的命令时，对终端本地缓存的内容，检查每一帧的扩展字段时间戳信息，如果当前视点的终端本地缓存内容预设时长(例如m毫秒)内存在IDR帧，则将该IDR帧的时间点加入视点切换请求，上报CDN服务器；如果当前视点的终端本地缓存内容m毫秒内不存在IDR帧，且缓存内容大于m毫秒，则将m毫秒后的时间点加入视点切换请求，上报CDN服务器；如果当前视点的终端本地缓存内容预设时长内不存在IDR帧，且缓存内容不足m毫秒，则将终端本地缓存末端的时间点加入视点切换请 求，上报CDN服务器。并且，清空上报服务器时间点之后的终端本地缓存。
CDN服务器接收终端发送的视点切换请求，如果发现上报的终端缓存m毫秒内存在IDR帧，则直接从CDN服务器上保存的要切换到的下个视点中，根据扩展字段的时间戳信息，找出对应时间点的IDR帧，清空本地缓存后，从该时间点流化打包，发送到终端；如果发送上报的终端切换时间点前不存在IDR帧，则需要将保存的CDN服务器上要切换到的下个视点的对应时间点视频帧，结合该时间点所处GOP的相关视频帧，计算合成出一个IDR帧，替换原有的该时间点的非IDR帧，并从该时间点流化打包，发送到终端。
计算合成出一个IDR帧所采用的方法可以是：将该时间点所属的GOP(n秒)内所有的编码帧进行解压缩处理，生成n秒的原始视频数据，并根据该时间点相对GOP起点时间的偏移，截取原始视频数据，以该时间点为起点，对截取的原始视频数据进行压缩编码，首帧生成IDR帧，替换原GOP内的编码帧。
终端在视点切换时，从切换时间点开始，播放原视点最长m毫秒的内容，继而播放新视点的视频内容，前后视点在切换的时间点上对齐。经测试验证，m毫秒取值为500毫秒较为合适。
需要注意的是：因为CDN在接收到切换请求时，可能CDN本地缓存中存在前一视点的视频内容，这部分内容无需再发送到终端，所以在接收到切换请求时也需要CDN清空本地的缓存。
实施例4
图5为本发明实施例中多视点视频视点切换时终端及CDN服务器侧流程图；本发明实施例描述了多视点视频切换时，终端与CDN服务器为达到前后视点衔接播放所做的处理。
其中，在终端侧：
S101：终端(如电视机顶盒、手机、PC等)正在播放多视点视频视点A的视频内容；
S102：用户通过终端方向键等方式(如机顶盒遥控器方向键、手机滑动、PC鼠标拖动)发起视点切换操作；
S103：终端解析用户的操作请求，判断切换到的下个视点为B；
S104：终端检查本地针对当前视点A的缓存内容，查找m毫秒内的视频帧是否有IDR帧；
S105：终端发现终端本地缓存的视点A的视频帧内容在m毫秒内有IDR帧，则将该IDR帧的时间点信息、视点切换到视点B的切换请求上报到CDN服务器，同时清空终端本地缓存内该IDR帧时间点及之后的内容；
S106：终端未在终端本地缓存的m毫秒内发现IDR帧，则检查终端本地缓存的内容长度是否大于m毫秒；
S107：终端发现本地的内容长度大于m毫秒，则将m毫秒处视频帧的时间点信息、IDR帧生成请求、视点切换请求上报CDN服务器，同时清空终端本地缓存m毫秒时间点及之后的内容；
S108：终端发现本地的内容长度小于m毫秒，则将终端本地缓存末端的时间点信息、IDR帧生成请求、视点切换请求上报CDN服务器；
S109：终端接收并播放切换后下个视点B的内容。
在CDN服务器侧：
S201：CDN服务器检查多视点视频内容，保证每个视点的视频以n秒为周期，固定存在IDR帧，如不符合，则需要进行转码，生成n秒为周期的IDR帧替换原有帧；
S202：CDN服务器在流化打包时，在每个视点的每个视频帧前，以本发明实施例1方案中的描述方式增加视点、时间戳信息的扩展字段；
S203：CDN服务器接收到终端的将视点切换到B的切换请求，以及终 端缓存IDR帧的时间点信息后，清空CDN服务器本地的缓存，并根据该时间点信息，直接提供下个视点B的对应时间点视频码流，无需计算合成，首帧即为IDR帧；
S204：CDN服务器接收到终端的视点切换到B的切换请求，以及IDR帧生成请求和生成时间点信息时，清空本地缓存，然后通过合成计算将下个视点B在该时间点的视频帧转换为IDR帧，替换B视点在该时间点的原有帧，同时将该IDR帧作为首帧发送给终端，并持续发送B视点的视频码流。
实施例5
图6所示为本发明实施例中多视点视频的视点切换系统的结构示意图。所述系统包括终端和服务器，其中：
所述终端用于在接收到用户视点切换请求后，检查当前视点的终端本地缓存内容预设时长内是否存在IDR帧；
如果当前视点的终端本地缓存内容预设时长内存在IDR帧，则将该IDR帧的时间点加入视点切换请求，上报服务器；
如果当前视点的终端本地缓存内容预设时长内不存在IDR帧，且缓存内容大于预设时长，则将预设时长后的时间点加入视点切换请求，上报服务器；
如果当前视点的终端本地缓存内容预设时长内不存在IDR帧，且缓存内容不足预设时长，则将终端本地缓存末端的时间点加入视点切换请求，上报服务器；
清空上报服务器视点切换请求中的时间点之后的终端本地缓存。
进一步地，所述终端向服务器上报视点切换请求后，还接收服务器发送的切换视点后的视频流，具体为：
所述服务器接收终端发送的视点切换请求；
如果终端本地缓存内容预设时长内存在IDR帧，则直接从服务器上保 存的要切换到的下个视点视频中，找出对应时间点的IDR帧，清空服务器缓存后，从该时间点流化打包并发送到终端；
如果终端本地缓存内容在上报视点切换请求中的时间点前不存在IDR帧，则将服务器上要切换到的下个视点的对应时间点视频帧结合该时间点所处GOP的相关视频帧，计算合成出一个IDR帧，替换原有的该时间点的非IDR帧，清空服务器缓存后，从该时间点流化打包并发送到终端。
进一步地，在所述多视点视频中，每路视频在每个n秒的时间节点上有一个IDR帧，各路视频的IDR帧在时间点上对齐，n为预设值。
在上述实施例中，可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。当使用软件实现时，可以全部或部分地以计算机程序产品的形式实现。所述计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行所述计算机程序指令时，全部或部分地产生按照本发明实施例所述的流程或功能。所述计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。所述计算机指令可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一个计算机可读存储介质传输，例如，所述计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线(DSL))或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。所述计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。所述可用介质可以是磁性介质，(例如，软盘、硬盘、磁带)、光介质(例如，DVD)、或者半导体介质(例如固态硬盘Solid State Disk(SSD))等。
在本发明的说明书和权利要求书及上述附图中的描述的一些流程中，包含了按照特定顺序出现的多个操作，但是应该清楚了解，这些操作可以不按照其在本文中出现的顺序来执行或并行执行，操作的序号仅仅是用于区 分开各个不同的操作，序号本身不代表任何的执行顺序。另外，这些流程可以包括更多或更少的操作，并且这些操作可以按顺序执行或并行执行。
本领域的技术人员容易理解，以上所述仅为本发明的较佳实施例而已，并不用以限制本发明，凡在本发明的精神和原则之内所作的任何修改、等同替换和改进等，均应包含在本发明的保护范围之内。

Claims (10)

  1. 一种多视点视频的视点切换方法，其特征在于，包括：
    终端接收到用户视点切换请求后，检查当前视点的终端本地缓存内容预设时长内是否存在IDR帧；
    如果当前视点的终端本地缓存内容预设时长内存在IDR帧，则将该IDR帧的时间点加入视点切换请求，上报服务器；
    如果当前视点的终端本地缓存内容预设时长内不存在IDR帧，且缓存内容大于预设时长，则将预设时长后的时间点加入视点切换请求，上报服务器；
    如果当前视点的终端本地缓存内容预设时长内不存在IDR帧，且缓存内容不足预设时长，则将终端本地缓存末端的时间点加入视点切换请求，上报服务器；
    清空上报服务器视点切换请求中的时间点之后的终端本地缓存。
  2. 如权利要求1所述的多视点视频的视点切换方法，其特征在于，所述终端向服务器上报视点切换请求后，还接收服务器发送的切换视点后的视频流，具体为：
    服务器接收终端发送的视点切换请求；
    如果终端本地缓存内容预设时长内存在IDR帧，则直接从服务器上保存的要切换到的下个视点视频中，找出对应时间点的IDR帧，清空服务器缓存后，从该时间点流化打包并发送到终端；
    如果终端本地缓存内容在上报视点切换请求中的时间点前不存在IDR帧，则将服务器上要切换到的下个视点的对应时间点视频帧结合该时间点所处GOP的相关视频帧，计算合成出一个IDR帧，替换原有的该时间点的非IDR帧，清空服务器缓存后，从该时间点流化打包并发送到终端。
  3. 如权利要求1或2所述的多视点视频的视点切换方法，其特征在于， 在所述多视点视频中，每路视频在每个n秒的时间节点上有一个IDR帧，各路视频的IDR帧在时间点上对齐，n为预设值。
  4. 如权利要求2所述的多视点视频的视点切换方法，其特征在于，所述将服务器上要切换到的下个视点的对应时间点视频帧结合该时间点所处GOP的相关视频帧，计算合成出一个IDR帧，具体为：将该时间点所属的GOP内所有的编码帧进行解压缩处理生成原始视频数据，并根据该时间点相对GOP起点时间的偏移截取原始视频数据，以该时间点为起点对截取的原始视频数据进行压缩编码，首帧生成IDR帧替换原GOP内的编码帧。
  5. 如权利要求1或2所述的多视点视频的视点切换方法，其特征在于，视频内容的每帧原始视频数据前增加扩展字段用于视点切换，所述扩展字段包括相机视点信息和时间戳信息。
  6. 如权利要求5述的多视点视频的视点切换方法，其特征在于，所述相机视点信息包括：当前视点的编号，视点的总数，报文中帧信息，帧类型。
  7. 如权利要求3所述的多视点视频的视点切换方法，其特征在于，所述n取值为2秒，所述预设时长取值为500毫秒。
  8. 一种多视点视频的视点切换系统，其特征在于，包括终端和服务器，其中：
    所述终端用于在接收到用户视点切换请求后，检查当前视点的终端本地缓存内容预设时长内是否存在IDR帧；
    如果当前视点的终端本地缓存内容预设时长内存在IDR帧，则将该IDR帧的时间点加入视点切换请求，上报服务器；
    如果当前视点的终端本地缓存内容预设时长内不存在IDR帧，且缓存内容大于预设时长，则将预设时长后的时间点加入视点切换请求，上报服务器；
    如果当前视点的终端本地缓存内容预设时长内不存在IDR帧，且缓存内容不足预设时长，则将终端本地缓存末端的时间点加入视点切换请求，上 报服务器；
    清空上报服务器视点切换请求中的时间点之后的终端本地缓存。
  9. 如权利要求8所述的多视点视频的视点切换系统，其特征在于，所述终端向服务器上报视点切换请求后，还接收服务器发送的切换视点后的视频流，具体为：
    所述服务器接收终端发送的视点切换请求；
    如果终端本地缓存内容预设时长内存在IDR帧，则直接从服务器上保存的要切换到的下个视点视频中，找出对应时间点的IDR帧，清空服务器缓存后，从该时间点流化打包并发送到终端；
    如果终端本地缓存内容在上报视点切换请求中的时间点前不存在IDR帧，则将服务器上要切换到的下个视点的对应时间点视频帧结合该时间点所处GOP的相关视频帧，计算合成出一个IDR帧，替换原有的该时间点的非IDR帧，清空服务器缓存后，从该时间点流化打包并发送到终端。
  10. 如权利要求8或9所述的多视点视频的视点切换系统，其特征在于，在所述多视点视频中，每路视频在每个n秒的时间节点上有一个IDR帧，各路视频的IDR帧在时间点上对齐，n为预设值。
PCT/CN2020/111970 2020-04-15 2020-08-28 一种多视点视频的视点切换方法和系统 WO2021208321A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010297270.9A CN111372145B (zh) 2020-04-15 2020-04-15 一种多视点视频的视点切换方法和系统
CN202010297270.9 2020-04-15

Publications (1)

Publication Number Publication Date
WO2021208321A1 true WO2021208321A1 (zh) 2021-10-21

Family

ID=71212591

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/111970 WO2021208321A1 (zh) 2020-04-15 2020-08-28 一种多视点视频的视点切换方法和系统

Country Status (2)

Country Link
CN (1) CN111372145B (zh)
WO (1) WO2021208321A1 (zh)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111372145B (zh) * 2020-04-15 2021-07-27 烽火通信科技股份有限公司 一种多视点视频的视点切换方法和系统
CN114035672A (zh) * 2020-07-20 2022-02-11 华为技术有限公司 一种虚拟现实vr场景的视频处理方法及相关设备
CN111800653B (zh) * 2020-07-29 2021-06-11 北京大学深圳研究生院 视频解码方法、系统、设备及计算机可读存储介质
CN112135198A (zh) * 2020-09-11 2020-12-25 烽火通信科技股份有限公司 一种复用多视点切换视频的方法、系统和服务器
CN112261418B (zh) * 2020-09-18 2022-09-30 网宿科技股份有限公司 一种传输直播视频数据的方法和直播加速系统
CN111866525A (zh) * 2020-09-23 2020-10-30 腾讯科技（深圳）有限公司 多视点视频的播放控制方法及装置、电子设备、存储介质
CN114584769A (zh) * 2020-11-30 2022-06-03 华为技术有限公司 一种视角切换方法及装置

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1694529A (zh) * 2005-06-28 2005-11-09 清华大学 交互式多视点视频系统中视频流的传输方法
CN1874520A (zh) * 2006-06-23 2006-12-06 清华大学 一种交互式多视点视频的编码方法
US20130223812A1 (en) * 2012-02-26 2013-08-29 Antonio Rossi Streaming video navigation systems and methods
CN111372145A (zh) * 2020-04-15 2020-07-03 烽火通信科技股份有限公司 一种多视点视频的视点切换方法和系统

Family Cites Families (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101459837B (zh) * 2009-01-09 2010-06-09 清华大学 一种交互式多视点视频流媒体服务中控制延迟的方法
CN102055967B (zh) * 2009-10-28 2012-07-04 中国移动通信集团公司 多视点视频的视角切换以及编码方法和装置
KR102219907B1 (ko) * 2012-04-23 2021-02-25 삼성전자주식회사 다시점 비디오 부호화 방법 및 장치, 다시점 비디오 복호화 방법 및 장치
EP2870766B1 (en) * 2012-07-09 2020-04-08 Mitsubishi Electric Corporation Method and system for processing multiview videos for view synthesis using motion vector predictor list
CN102970540B (zh) * 2012-11-21 2016-03-02 宁波大学 基于关键帧码率-量化模型的多视点视频码率控制方法
CN102982159B (zh) * 2012-12-05 2016-07-06 上海创图网络科技发展有限公司 一种三维网页多场景快速切换方法
US9848212B2 (en) * 2015-07-10 2017-12-19 Futurewei Technologies, Inc. Multi-view video streaming with fast and smooth view switch
CN105933343B (zh) * 2016-06-29 2019-01-08 深圳市优象计算技术有限公司 一种用于720度全景视频网络播放的码流缓存方法
FR3066672B1 (fr) * 2017-05-19 2020-05-22 Sagemcom Broadband Sas Procede de communication d'une video immersive
CN109218848A (zh) * 2017-07-06 2019-01-15 阿里巴巴集团控股有限公司 视频流的视角切换方法、装置、设备和计算机存储介质
CN109698949B (zh) * 2017-10-20 2020-08-21 腾讯科技（深圳）有限公司 基于虚拟现实场景的视频处理方法、装置和系统
CN108319362B (zh) * 2018-01-02 2020-06-23 联想(北京)有限公司 一种全景信息显示方法、电子设备和计算机存储介质
CN110351607B (zh) * 2018-04-04 2022-01-14 阿里巴巴（中国）有限公司 一种全景视频场景切换的方法、计算机存储介质及客户端

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1694529A (zh) * 2005-06-28 2005-11-09 清华大学 交互式多视点视频系统中视频流的传输方法
CN1874520A (zh) * 2006-06-23 2006-12-06 清华大学 一种交互式多视点视频的编码方法
US20130223812A1 (en) * 2012-02-26 2013-08-29 Antonio Rossi Streaming video navigation systems and methods
CN111372145A (zh) * 2020-04-15 2020-07-03 烽火通信科技股份有限公司 一种多视点视频的视点切换方法和系统

Also Published As

Publication number Publication date
CN111372145A (zh) 2020-07-03
CN111372145B (zh) 2021-07-27

Similar Documents

Publication Publication Date Title
WO2021208321A1 (zh) 一种多视点视频的视点切换方法和系统
KR100711635B1 (ko) 화상 부호화 방법
KR101737325B1 (ko) 멀티미디어 시스템에서 멀티미디어 서비스의 경험 품질 감소를 줄이는 방법 및 장치
JP5788101B2 (ja) メディアデータのネットワークストリーミング
JP3801984B2 (ja) 帯域幅スケーラを含むマルチキャスト伝送システム
US7436454B2 (en) Method and apparatus for transmitting encoded information based upon piority data in the encoded information
KR101453239B1 (ko) 인코딩된 비디오 데이터의 스트리밍
RU2385541C2 (ru) Изменение размера буфера в кодере и декодере
KR101687640B1 (ko) 동기화된 컨텐츠 재생 방법
CN103843301A (zh) 经译码多媒体数据的网络串流期间的表示之间的切换
CN107566918A (zh) 一种视频分发场景下的低延时取流秒开方法
CN101917613A (zh) 一种流媒体采集编码服务系统
KR100678891B1 (ko) Ａｖ데이터 수신시 버퍼량을 컨텐츠 속성에 따라탄력적으로 조절하는 방법 및 장치
CN112770122B (zh) 一种在云导播台视频同步的方法及系统
CN112752115A (zh) 直播数据传输方法、装置、设备及介质
WO2009103343A1 (en) Method and apparatus for distributing media over a communications network
US20060161676A1 (en) Apparatus for IP streaming capable of smoothing multimedia stream
US20140321556A1 (en) Reducing amount of data in video encoding
Shirai et al. 4K SHD real-time video streaming system with JPEG 2000 parallel codec
CN112565670B (zh) 云会议多层视频快速平滑出图的方法
KR20130122117A (ko) 실시간 동영상 전송 방법 및 장치
CN115883855B (zh) 播放数据处理方法、装置、计算机设备和存储介质
KR101992533B1 (ko) 실시간 iptv 방송에서의 채널 전환 시스템
JP2004349743A (ja) 映像ストリーム切替システム、方法、映像ストリーム切替システムを含む映像監視、映像配信システム
CN116055810A (zh) 音视频播放方法、装置、系统、电子设备及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20931356

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20931356

Country of ref document: EP

Kind code of ref document: A1