WO2025213828A1 - 直播方法、装置、电子设备及存储介质 - Google Patents

直播方法、装置、电子设备及存储介质

Info

Publication number
WO2025213828A1
WO2025213828A1 PCT/CN2024/139663 CN2024139663W WO2025213828A1 WO 2025213828 A1 WO2025213828 A1 WO 2025213828A1 CN 2024139663 W CN2024139663 W CN 2024139663W WO 2025213828 A1 WO2025213828 A1 WO 2025213828A1
Authority
WO
WIPO (PCT)
Prior art keywords
video stream
target
video
projection format
live video
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
PCT/CN2024/139663
Other languages
English (en)
French (fr)
Inventor
胡武林
张涛
辛彦哲
陈泽鑫
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Zitiao Network Technology Co Ltd
Original Assignee
Beijing Zitiao Network Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Zitiao Network Technology Co Ltd filed Critical Beijing Zitiao Network Technology Co Ltd
Publication of WO2025213828A1 publication Critical patent/WO2025213828A1/zh
Pending legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234309—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4 or from Quicktime to Realvideo
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21—Server components or server architectures
    • H04N21/218—Source of audio or video content, e.g. local disk arrays
    • H04N21/2187—Live feed
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440218—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4

Definitions

  • the present disclosure relates to the field of data processing technology, and in particular to a live broadcast method, device, electronic device, and storage medium.
  • VR Virtual Reality
  • the present disclosure provides a live broadcast method, device, electronic device and storage medium.
  • a live broadcast method comprising:
  • the video projection format of the source live video stream is a first projection format
  • converting the video projection format of the source live video stream into a second projection format to obtain a target live video stream
  • a live broadcast method comprising:
  • the video projection format of the target live video stream is a second projection format, obtaining video layout information and a video type of the target live video stream; wherein the second projection format is used to project a spherical image onto each face of an unfolded cube, the cube being circumscribed to the spherical image;
  • the target live video stream is divided into a plurality of sub-pictures, where the plurality of sub-pictures include a plurality of viewing angles;
  • the multiple sub-pictures are respectively mapped to a target three-dimensional sphere, and the video picture on the target three-dimensional sphere is displayed on the terminal.
  • a live broadcast device comprising:
  • a video projection format acquisition module is used to acquire a source live video stream and obtain a video projection format of the source live video stream
  • a video projection format conversion module configured to, when the video projection format of the source live video stream is a first projection format, convert the video projection format of the source live video stream into a second projection format to obtain a target live video stream; wherein the first projection format is used to project a spherical image into a rectangular image, and the second projection format is used to project the spherical image onto each face of an unfolded cube, the cube being circumscribed to the spherical image;
  • the video stream sending module is used to push the target live video stream to the terminal.
  • a live broadcast device comprising:
  • a format acquisition module is used to acquire a target live video stream and obtain a video projection format of the target live video stream
  • a video information acquisition module configured to acquire video layout information and a video type of the target live video stream when the video projection format of the target live video stream is a second projection format; wherein the second projection format is used to project a spherical image onto each face of an unfolded cube, the cube being circumscribed to the spherical image;
  • a segmentation module configured to segment the target live video stream into a plurality of sub-pictures based on the video layout information and the video type, wherein the plurality of sub-pictures include a plurality of viewing angles;
  • the video display module is used to map the multiple sub-pictures to a target three-dimensional sphere respectively, and display the video pictures on the target three-dimensional sphere on the terminal.
  • an electronic device comprising: a memory and a processor, wherein the memory stores a computer program, and the processor implements the above method when executing the program.
  • a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the above method of the present disclosure is implemented.
  • FIG1 is a schematic diagram of a VR live broadcast scene provided by an exemplary embodiment of the present disclosure
  • FIG2 is a schematic diagram of converting an image in ERP format into an image in EAC format according to an exemplary embodiment of the present disclosure
  • FIG3 is a schematic diagram of converting an image in ERP format into an image in EAC format provided by another exemplary embodiment of the present disclosure
  • FIG4 is a schematic diagram of converting an image in ERP format into an image in EAC format provided by another exemplary embodiment of the present disclosure
  • FIG5 is a schematic diagram of converting an image in ERP format into an image in EAC format provided by another exemplary embodiment of the present disclosure
  • FIG6 is a schematic diagram of converting an image in ERP format into an image in EAC format provided by yet another exemplary embodiment of the present disclosure
  • FIG7 is a schematic diagram of a three-dimensional grid provided by an exemplary embodiment of the present disclosure.
  • FIG8 is a schematic diagram of mapping an image to a three-dimensional grid according to an exemplary embodiment of the present disclosure
  • FIG9 is a flowchart of a live broadcast method provided by an exemplary embodiment of the present disclosure.
  • FIG10 is a schematic block diagram of functional modules of a live broadcast device provided by an exemplary embodiment of the present disclosure.
  • FIG11 is a structural block diagram of an electronic device provided by an exemplary embodiment of the present disclosure.
  • FIG12 is a structural block diagram of a computer system provided by an exemplary embodiment of the present disclosure.
  • a prompt message is sent to the user to clearly inform the user that the operation requested will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the electronic device, application, server, storage medium, or other software or hardware that performs the operations of the disclosed technical solution based on the prompt message.
  • the method of sending a prompt message to the user can be, for example, a pop-up window, and the prompt message can be presented in the form of text in the pop-up window.
  • the pop-up window can also carry a selection control for the user to choose "agree” or "disagree” to provide personal information to the electronic device.
  • the top and bottom parts (i.e., the edges corresponding to the top and bottom of the VR screen) of the live video stream in the ERP projection format have denser pixels allocated to the top and bottom, which will result in a large amount of pixel waste. This will cause a waste of CDN (Content Delivery Network) encoding and decoding resources and bandwidth in VR live streaming, and degrade indicators such as the first frame time, second opening rate, freeze rate, stream bit rate or frame rate of the VR live broadcast link, thereby wasting video encoding and decoding resources and bandwidth in VR live streaming, and having a negative impact on video transmission and playback.
  • CDN Content Delivery Network
  • the server can receive the VR live video stream sent by the anchor end.
  • the VR live video stream can be obtained by stitching the video images taken by the anchor end through a panoramic camera or multiple different angles to obtain a 360-degree or 180-degree panoramic image, and obtain a VR live video stream.
  • the anchor end sends the obtained VR live video stream to the server.
  • a panoramic camera can be used to obtain video images shot by multiple cameras and stitch them together to form a 360-degree or 180-degree panoramic image in ERP format.
  • a spherical image is obtained, and a VR live video stream can be obtained based on the spherical image.
  • the host terminal can also convert the above-mentioned spherical image into a two-dimensional plane image in ERP format, and then obtain a VR live video stream based on the two-dimensional plane image.
  • the VR live video stream sent by the above-mentioned anchor terminal can also be a VR live video stream in other projection formats, and the embodiment is not limited to this.
  • the VR live video stream when a VR live video stream is received, during the transcoding process of the VR live video stream, the VR live video stream can be first decapsulated by the decapsulation module, and decrypted by the audio decoding module to obtain the audio stream.
  • the decoded live video stream can be obtained by the video decoding module.
  • the VR live video stream in ERP format is converted into a VR live video stream in EAC (Equi-Angular Cubemap projection) format by the projection format conversion module, and the VR live video stream is encoded by the video encoding module.
  • the audio stream is encoded by the audio encoding module, and the encoded audio stream and video stream are encapsulated by the encapsulation module.
  • the encapsulated VR live video stream is distributed to each edge node through the content distribution network.
  • the video decoding module, projection format conversion module and video encoding module are respectively located in the GPU (Graphics Processing Unit), and the video encoding and decoding and projection format conversion are performed by the GPU.
  • the GPU Graphics Processing Unit
  • the server when the server receives the VR live video stream sent by the host, it can first determine whether the VR live video stream is in ERP format. If it is, it needs to convert the ERP format VR live video stream to EAC format. Because EAC format video can evenly distribute pixels, it can avoid a large amount of pixel redundancy in the non-central part of the VR live video.
  • the VR live video stream when receiving a VR live video stream, after detecting that the projection format of the VR live video stream is the ERP format, it is also possible to determine whether the VR live video stream is a spherical image or a two-dimensional plane image based on the video type of the VR live video stream, so as to convert the VR live video stream in the ERP format into the VR live video stream in the EAC format.
  • the VR live video stream in ERP format can be directly converted into a VR live video stream in EAC format.
  • the VR live video stream is a spherical image, for example, a 360-degree spherical image, it can be sampled in six directions at the center of the sphere to obtain six images of the cube, and then each pixel of each image in the cube is calculated according to the following formula (1), and the six faces obtained are the EAC images; and when the VR live video stream is a 180-degree spherical image, it can be sampled in five directions at the center of the sphere to obtain images in five directions.
  • Figure 2 is a schematic diagram of converting an image in ERP format into an image in EAC format.
  • the left side of Figure 2 is a face in the RPR cube, and there is a pixel point p on the face.
  • the pixel point p is transformed into the coordinate q of the pixel point by formula (1), which can be converted into an image in EAC format.
  • the coordinates of the pixel point q (u, v) are as follows:
  • layout information of a VR live video stream can be obtained, and a VR live video stream in ERP format can be converted into a VR live video stream in EAC format.
  • the VR live video stream can be a video file in FLV (FLASH VIDEO) format, and can specifically include four layouts: 2D 180 degrees, i.e., a 180-degree FOV (field of view) monocular image; 2D 360 degrees, i.e., a 360-degree FOV (field of view) monocular image; 3D 180 degrees, i.e., a 180-degree FOV (field of view) binocular image; and 3D 360 degrees, i.e., a 360-degree FOV (field of view) binocular image.
  • 2D 180 degrees i.e., a 180-degree FOV (field of view) monocular image
  • 2D 360 degrees i.e., a 360-degree FOV (field of view) monocular image
  • 3D 180 degrees i.e., a 180-degree FO
  • a terminal when converting a VR live video stream in ERP format to a VR live video stream in EAC format, SEI (Supplemental Enhancement Information) can be added to the video frame.
  • SEI can carry an EAC identifier to indicate that the video frame or the VR live video stream is an EAC format video stream, so that the VR terminal can map the EAC format VR live video stream to spherical coordinates according to the EAC format of the VR live video stream to obtain a panoramic spherical video image.
  • the above-mentioned SEI information may be added to the starting frame of the generated VR live video stream in the EAC format, but the embodiment is not limited thereto.
  • the stream can be pushed to the CDN, and the projection format field information can be added to the metadata to indicate that the transmitted VR live video stream is a video stream in EAC format.
  • the client on the VR terminal When the client on the VR terminal receives the VR live video stream pushed by the edge node, it can decapsulate the VR live video stream through the decapsulation module, obtain and decode the VR live video stream through the audio decoding module and the video decoding module to obtain audio data volume and video data stream.
  • EAC sub-pictures of multiple perspectives can be cut out according to the EAC layout information and SEI information of the VR live video stream, and some EAC sub-pictures can be rotated.
  • the EAC layout information of the VR live video stream is 2D360 degrees, it is divided into 6 sub-screens: left, front, right, bottom, back, and top.
  • the bottom and top sub-screens are rotated 90 degrees counterclockwise, and the 'back' sub-screen is rotated 90 degrees clockwise.
  • the EAC layout information of the VR live video stream is 3D180 degrees, it is divided into 10 sub-screens according to the layout, namely left eye left, left eye front, left eye right, left eye bottom, left eye top, right eye left, right eye front, right eye right, right eye bottom, and right eye top.
  • the four sub-screens of left eye bottom, left eye top, right eye bottom, and right eye top are rotated 90 degrees counterclockwise.
  • the EAC layout information of the VR live video stream is 3D360 degrees
  • it is divided into 12 sub-screens according to the layout, namely left eye left, left eye front, left eye right, left eye bottom, left eye back, left eye top, right eye left, right eye front, right eye right, right eye bottom, right eye back, and right eye top.
  • the four sub-screens of left eye bottom, left eye top, right eye bottom, and right eye top are rotated 90 degrees counterclockwise, and the two sub-screens behind the left eye and behind the right eye are rotated 90 degrees clockwise.
  • each EAC sub-picture obtained as described above may be mapped onto a three-dimensional spherical surface, and the three-dimensional spherical surface picture may be displayed on a VR terminal.
  • a 3D mesh is regenerated for the hemisphere. Since EAC is in five directions, this mesh also consists of five parts, which are spliced together by five sub-meshes. As shown in Figure 7, each sub-mesh takes on the EAC sub-image in the corresponding direction, so the EAC sub-images in the five directions can be mapped to five sub-meshes. It should be noted that if it is 360 degrees, EAC will be obtained by using images in six directions, so the corresponding spherical mesh will include six sub-meshes.
  • the edges of each sub-mesh in three-dimensional space are curved.
  • the left and right edges of the upper left sub-mesh and the leftmost mesh are curved. Therefore, the sub-mesh precision cannot be too low, otherwise the edges will appear uneven and curved.
  • the precision cannot be infinitely high and can be set to M ⁇ N based on business needs. That is, the precision of each face can be limited to M ⁇ N. The specific setting can be based on needs and the embodiment is not limited to this.
  • the mesh can be split according to the EAC screen distribution instead of the traditional spindle-shaped organization vertices.
  • the spindle-shaped vertices if the six faces of the EAC sub-screen are fitted to the spherical surface of the spindle-shaped vertices, the vertex density of the upper and lower faces will be higher than the vertices of the four front, back, left and right faces, that is, the vertex density of the six faces is inconsistent.
  • the upper and lower faces cannot cut out regular regular polygons, which will consume more performance during rendering calculations.
  • the embodiment of the present disclosure can ensure uniform density of the six faces by splitting according to the EAC distribution, which can generate less calculation.
  • UV texture mapping coordinates
  • arctan calculation of EAC mapping to the sphere is also included, so that the five surfaces (or six surfaces) are spliced together. Since the collinear parts each use their own UV, there will be no visible seams at the splicing points.
  • the EAC calculation results can be written to vertices. If it is a seam, it is written twice. For example, for the upper and front seams in Figure 8, the calculated value of the seam is written once for the upper calculation and once for the front seam. Each side saves the conversion value of the same position and performs its own calculation processing during rendering to achieve the processing of the seam.
  • the present disclosure further provides a live broadcast method, which can be applied to a server.
  • the method may include the following steps:
  • step S110 a source live video stream is obtained, and a video projection format of the source live video stream is obtained.
  • the video projection format is ERP
  • a large amount of pixel redundancy will be generated in the non-central portion of the video stream.
  • a large number of pixels are concentrated at the top and bottom edges of the VR screen, while the user's attention is generally focused on the central portion of the video. This will result in a large amount of pixel waste, which in turn causes excessive encoding and decoding resources and bandwidth resources to be occupied during video transmission.
  • the source live video stream can be the VR live video stream in the above embodiment.
  • step S120 when the video projection format of the source live video stream is the first projection format, the video projection format of the source live video stream is converted into the second projection format to obtain the target live video stream.
  • the first projection format is used to project the spherical image into a rectangular image
  • the second projection format is used to project the spherical image onto each surface of the unfolded cube, with the cube circumscribing the spherical image.
  • the first projection format may be the ERP projection format in the above embodiment
  • the second projection format may be the EAC projection format in the above embodiment, or may be a CMP (CubeMap, cube map projection) projection format.
  • the second projection format is described as the EAC projection format as an example, but the embodiment is not limited thereto.
  • step S130 the target live video stream is pushed to the terminal.
  • the live broadcast method detects the video projection format of the source live video stream, and when the video projection format of the source live video stream is an equirectangular cylindrical projection, converts the video projection format of the source live video stream into an equiangular cube map projection. In this way, the target video stream obtained can reduce the excessive encoding and decoding resources and bandwidth resources occupied during transmission, thereby improving the transmission efficiency of the video.
  • the server can receive streaming information sent by the terminal, and the streaming information can carry the location information of the terminal.
  • the server can push the live video stream through the edge node closest to the terminal through the CDN based on the location information of the terminal to improve the transmission efficiency of the video. Therefore, the server can send the target live video stream to the content distribution network; when receiving the play request sent by the terminal, the server can send the target live video stream to the target edge node through the content distribution network, and push the target live video stream to the terminal through the target edge node; wherein, the target edge node is determined based on the current location of the terminal.
  • the method may further include the following steps:
  • step S140 target supplementary enhancement information is obtained, wherein the target supplementary enhancement information carries a video projection format identifier of the second projection format.
  • step S150 target supplementary enhancement information is added to the target video frame of the target live video stream.
  • the target supplementary enhancement information can carry the EAC video projection format identifier.
  • the target supplementary enhancement information can specifically be the SEI information in the above embodiment.
  • the SEI information can be added to each video frame, and the SEI information can also be added to the target video frame.
  • the starting video frame of the target live video stream can be obtained, and the starting video frame can be used as the target video frame. In this way, by adding SEI information to the starting video frame, the starting frame and subsequent video frames are processed, for example, divided into multiple sub-pictures, and mapped to spherical coordinates respectively.
  • the target supplementary enhancement information may also be added to a key frame of the target live video stream, but the embodiment is not limited thereto.
  • the source live video stream includes audio data and video data; the above step S120 may further include the following steps:
  • step S121 the audio data in the source live video stream is encoded, and the video data converted into the second projection format is encoded.
  • step S122 the encoded audio data and the encoded video data are encapsulated to obtain encapsulated audio and video stream data, and video projection format field information of the second projection format is added to the metadata of the encapsulated audio and video stream data to obtain the target live video stream.
  • the second projection format is still described as the EAC projection format.
  • EAC projection format By encoding the audio data and video data in the live video stream separately, and encapsulating the encoded audio data and the encoded video data, encapsulated audio and video stream data can be obtained, and the video projection format field information of the equiangular cube map projection EAC is added to the metadata of the encapsulated audio and video stream data.
  • the terminal receives the target live video stream, it can directly determine whether the target live video stream is a live video stream in the EAC format based on the video projection format field information carried in the metadata, which facilitates the processing of the video according to the projection format of the live video stream, thereby improving the video processing efficiency.
  • the source live video stream may include a panoramic video
  • the above step S120 may further include the following steps:
  • step S123 the video type of the panoramic video is obtained, and the video projection format of the source live video stream is converted into a second projection format based on the video type.
  • the video projection format of the source live video stream can be converted into a second projection format based on the video type, and the second projection format is still described as the EAC projection format.
  • the video type can be a spherical image video or a two-dimensional plane image video.
  • the VR live video stream in ERP format can be directly converted into a VR live video stream in EAC format.
  • the VR live video stream is a spherical image, it can be sampled in multiple directions at the center of the sphere to obtain plane images in multiple directions, and the plane images in multiple directions are converted from ERP format to EAC format.
  • a live broadcast method is further provided.
  • the method can be applied to a terminal, such as a VR terminal, and the method may include the following steps:
  • step S210 a target live video stream is obtained, and a video projection format of the target live video stream is obtained.
  • the terminal can send pull stream information to the push stream end, and the push stream end pushes the target live video stream, such as a VR live video stream, to the terminal.
  • the target live video stream such as a VR live video stream
  • step S220 when the video projection format of the target live video stream is a second projection format, the video layout information and video type of the target live video stream are obtained.
  • the second projection format is used to project the spherical image onto each face of a cube after expansion, and the cube is circumscribed to the spherical image.
  • the second projection format is the EAC projection format as an example for description.
  • the target supplementary enhancement information carried by the target live video stream is obtained, and the projection format of the target live video stream is determined according to the target supplementary enhancement information.
  • the target supplementary enhancement information may be included in the target video frame, which may specifically be the starting frame or key frame of the live video stream.
  • the target supplementary enhancement information may also be carried in each video frame, but the embodiment is not limited thereto.
  • a target video frame can be obtained from a target live video stream, the target video frame carrying target supplemental enhancement information; the target supplemental enhancement information carries a video projection format identifier of an equi-angle cubemap projection (EAC); and the video projection format of the target live video stream is determined based on the video projection format identifier.
  • EAC equi-angle cubemap projection
  • the video layout information and video type of the target live video stream can be obtained.
  • the video layout information can specifically include 2D180 degrees, 2D360 degrees, 3D180 degrees or 3D360 degrees
  • the video type can include a spherical image or a flat image.
  • step S230 based on the video layout information and the video type, the target live video stream is divided into multiple sub-pictures, and the multiple sub-pictures include multiple viewing angles.
  • ERP format video can be converted to EAC format video based on the video type.
  • a 2D 180-degree, 2D 360-degree, 3D 180-degree, or 3D 360-degree image can be converted to a corresponding EAC format image, thereby obtaining multiple sub-EAC images.
  • step S240 the plurality of sub-pictures are mapped to the target three-dimensional sphere respectively, and the video picture on the target three-dimensional sphere is displayed on the terminal.
  • the EPR format video is converted to the EAC format video.
  • the generated multiple EAC sub-images are mapped to the spherical surface, thereby realizing the display of the video image on the target 3D spherical surface. This can avoid the problem of excessive encoding and decoding resources and bandwidth resources being consumed during video transmission due to the large number of pixels at the top and bottom of the ERP format video.
  • step S240 may further include the following steps:
  • step S241 a target three-dimensional sphere is generated based on the video type, and a target three-dimensional sphere mesh is generated based on the target three-dimensional sphere, wherein the target three-dimensional sphere mesh includes a plurality of three-dimensional mesh surfaces.
  • step S242 the plurality of sub-pictures are mapped to a plurality of three-dimensional mesh surfaces respectively, to obtain a target three-dimensional spherical surface including the plurality of sub-pictures.
  • a mesh of multiple parts can be generated based on the video layout information. Taking the display of EAC 3D 180-degree picture as an example, a three-dimensional grid mesh is regenerated for the hemisphere. Since EAC is in five directions, the mesh also includes five parts, which are spliced together by five sub-meshes. As shown in FIG7 , each sub-mesh takes over the EAC sub-picture in the corresponding direction, so that the EAC sub-pictures in the five directions can be mapped to the five sub-meshes. It should be noted that if it is 360 degrees, EAC will be obtained by adopting the pictures in six directions, so the corresponding spherical mesh will include six sub-meshes. In this way, multiple EAC sub-pictures can be mapped to the sub-meshes corresponding to each part, and the sub-pictures in each mesh can be spliced and seamed to obtain the video picture on the target three-dimensional sphere.
  • an embodiment of the present disclosure provides a live broadcast device, which can be a server, a terminal, or a chip applied to a server.
  • Figure 10 is a schematic block diagram of the functional modules of the live broadcast device provided by an exemplary embodiment of the present disclosure. As shown in Figure 10, the live broadcast device includes:
  • the video projection format acquisition module 10 is used to acquire a source live video stream and obtain a video projection format of the source live video stream;
  • the video projection format conversion module 20 is configured to convert the video projection format of the source live video stream into a second projection format when the video projection format of the source live video stream is a first projection format, thereby obtaining a target live video stream; wherein the first projection format is used to project a spherical image into a rectangular image, and the second projection format is used to project the spherical image onto each face of an unfolded cube, wherein the cube is circumscribed to the spherical image;
  • the video stream sending module 30 is used to push the target live video stream to the terminal.
  • the apparatus further includes:
  • a supplementary enhancement information acquisition module configured to acquire target supplementary enhancement information, wherein the target supplementary enhancement information carries a video projection format identifier of the second projection format;
  • An information adding module is used to add the target supplementary enhancement information to the target video frame of the target live video stream.
  • the apparatus further includes:
  • the target video frame determination module is used to obtain the starting video frame of the target live video stream and use the starting video frame as the target video frame.
  • the source live video stream includes audio data and video data; the video projection format conversion module is specifically configured to:
  • the encoded audio data and the encoded video data are encapsulated to obtain encapsulated audio and video stream data, and video projection format field information of the second projection format is added to the metadata of the encapsulated audio and video stream data to obtain the target live video stream.
  • the apparatus further includes: a target live video stream sending module, wherein the target live video stream sending module is configured to:
  • the target live video stream Upon receiving a playback request from a terminal, the target live video stream is sent to a target edge node via the content distribution network, and the target live video stream is pushed to the terminal via the target edge node; wherein the target edge node is determined based on the current location of the terminal.
  • the source live video stream includes a panoramic video
  • the video projection format conversion module is specifically configured to:
  • the video type of the panoramic video is obtained, and based on the video type, the video projection format of the source live video stream is converted into a second projection format.
  • the relevant device part corresponds to the above method. Please refer to the description of the corresponding method for details and will not be repeated here.
  • the live broadcast device detects the video projection format of the source live video stream, and when the video projection format of the source live video stream is the first projection format, converts the video projection format of the source live video stream into the second projection format. In this way, the target video stream obtained can reduce the excessive encoding and decoding resources and bandwidth resources occupied during transmission, thereby improving the transmission efficiency of the video.
  • the embodiment of the present disclosure provides a live broadcast device, which can be a server, a terminal, or a chip applied to a server.
  • the live broadcast device includes:
  • a format acquisition module is used to acquire a target live video stream and obtain a video projection format of the target live video stream
  • a video information acquisition module configured to acquire video layout information and a video type of the target live video stream when the video projection format of the target live video stream is a second projection format; wherein the second projection format is used to project a spherical image onto each face of an unfolded cube, the cube being circumscribed to the spherical image;
  • a segmentation module configured to segment the target live video stream into a plurality of sub-pictures based on the video layout information and the video type, wherein the plurality of sub-pictures include a plurality of viewing angles;
  • the video display module is used to map the multiple sub-pictures to a target three-dimensional sphere respectively, and display the video pictures on the target three-dimensional sphere on the terminal.
  • the video display module is specifically configured to:
  • the target three-dimensional spherical mesh includes a plurality of three-dimensional mesh surfaces
  • the multiple sub-pictures are mapped to the multiple three-dimensional mesh surfaces respectively to obtain a target three-dimensional spherical surface containing the multiple sub-pictures.
  • the format acquisition module is specifically configured to:
  • the target video frame carries target supplementary enhancement information
  • the target supplementary enhancement information carries a video projection format identifier of the second projection format
  • the video projection format of the target live video stream is determined based on the video projection format identifier.
  • the relevant device part corresponds to the above method. Please refer to the description of the corresponding method for details and will not be repeated here.
  • the video in the first projection format is converted into a video in the second projection format.
  • the generated multiple sub-images in the second projection format are mapped onto a spherical surface, thereby achieving video display on the target three-dimensional spherical surface. This avoids the problem of excessive codec and bandwidth resources being consumed during video transmission due to the large number of pixels at the top and bottom of the video in the first projection format.
  • An embodiment of the present disclosure further provides an electronic device, comprising: at least one processor; a memory for storing instructions executable by the at least one processor; wherein the at least one processor is configured to execute the instructions to implement the above method disclosed in the embodiment of the present disclosure.
  • Figure 11 is a schematic diagram of the structure of an electronic device provided by an exemplary embodiment of the present disclosure.
  • the electronic device 1800 includes at least one processor 1801 and a memory 1802 coupled to the processor 1801.
  • the processor 1801 can execute the corresponding steps of the above method disclosed in the embodiment of the present disclosure.
  • the processor 1801 can also be referred to as a central processing unit (CPU), which can be an integrated circuit chip with signal processing capabilities. Each step in the method disclosed in the embodiment of the present disclosure can be completed by hardware integrated logic circuits in the processor 1801 or by software instructions.
  • the processor 1801 can be a general-purpose processor, a digital signal processor (DSP), an ASIC (Application Specific Integrated Circuit), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.
  • the general-purpose processor can be a microprocessor or any conventional processor.
  • the steps of the method disclosed in conjunction with the embodiment of the present disclosure can be directly implemented as being executed by a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor.
  • the software module can be located in the memory 1802, such as a random access memory, flash memory, read-only memory, programmable read-only memory, or electrically erasable programmable memory, registers, and other mature storage media in the art.
  • the processor 1801 reads the information in the memory 1802 and completes the steps of the above method in combination with its hardware.
  • FIG12 is a block diagram of the structure of a computer system provided by an exemplary embodiment of the present disclosure.
  • Computer system 1900 is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers.
  • Electronic devices may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices.
  • the components shown herein, their connections and relationships, and their functions are intended to be examples only and are not intended to limit the implementation of the present disclosure described and/or claimed herein.
  • computer system 1900 includes a computing unit 1901, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1902 or a computer program loaded from a storage unit 1908 into a random access memory (RAM) 1903.
  • RAM 1903 may also store various programs and data required for the operation of computer system 1900.
  • Computing unit 1901, ROM 1902, and RAM 1903 are connected to each other via a bus 1904.
  • An input/output (I/O) interface 1905 is also connected to bus 1904.
  • I/O interface 1905 Several components within computer system 1900 are connected to I/O interface 1905, including an input unit 1906, an output unit 1907, a storage unit 1908, and a communication unit 1909.
  • Input unit 1906 can be any type of device capable of inputting information into computer system 1900.
  • Input unit 1906 can receive input numeric or character information and generate key input signals related to user settings and/or function control of an electronic device.
  • Output unit 1907 can be any type of device capable of presenting information and may include, but is not limited to, a display, a speaker, a video/audio output terminal, a vibrator, and/or a printer.
  • Storage unit 1908 may include, but is not limited to, a magnetic disk or an optical disk.
  • Communication unit 1909 allows computer system 1900 to exchange information/data with other devices over a network, such as the Internet, and may include, but is not limited to, a modem, a network card, an infrared communication device, a wireless communication transceiver and/or chipset, such as a BluetoothTM device, a WiFi device, a WiMax device, a cellular communication device, and/or the like.
  • the computing unit 1901 can be various general-purpose and/or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc.
  • the computing unit 1901 performs the various methods and processes described above. For example, in some embodiments, the above-mentioned methods disclosed in the embodiments of the present disclosure may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 1908.
  • part or all of the computer program can be loaded and/or installed on the electronic device via the ROM 1902 and/or the communication unit 1909.
  • the computing unit 1901 can be configured to perform the above-mentioned methods disclosed in the embodiments of the present disclosure by any other appropriate means (for example, by means of firmware).
  • An embodiment of the present disclosure further provides a computer-readable storage medium, wherein, when instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the above method disclosed in the embodiment of the present disclosure.
  • the computer-readable storage medium in the disclosed embodiments can be a tangible medium, which can contain or store a program for use by an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment.
  • the above-mentioned computer-readable storage medium can include but is not limited to electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the above.
  • the above-mentioned computer-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (ERPOM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device or any suitable combination of the above.
  • the computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.
  • the embodiments of the present disclosure further provide a computer program product, including a computer program, wherein when the computer program is executed by a processor, the method disclosed in the embodiments of the present disclosure is implemented.
  • computer program code for performing the operations of the present disclosure may be written in one or more programming languages or combinations thereof, including but not limited to object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages.
  • the program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server.
  • the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer.
  • LAN local area network
  • WAN wide area network
  • each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function.
  • the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved.
  • each box in the block diagram and/or flowchart, and the combination of the boxes in the block diagram and/or flowchart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
  • the modules, components, or units described in the embodiments of the present disclosure may be implemented in software or hardware.
  • the names of the modules, components, or units do not necessarily limit the modules, components, or units themselves.
  • exemplary hardware logic components include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.
  • FPGAs field programmable gate arrays
  • ASICs application specific integrated circuits
  • ASSPs application specific standard products
  • SOCs systems on chip
  • CPLDs complex programmable logic devices

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Databases & Information Systems (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

本公开涉及直播方法、装置、电子设备及存储介质，上述方法包括:获取源直播视频流，并获取所述源直播视频流的视频投影格式；在所述源直播视频流的视频投影格式为第一投影格式的情况下，将所述源直播视频流的视频投影格式转换为第二投影格式，获得目标直播视频流；其中，所述第一投影格式用于将球面图像投影为矩形图像，所述第二投影格式用于将球面图像投影到立方体展开后的各面上，所述立方体外切于所述球面图像；向终端推送所述目标直播视频流。

Description

直播方法、装置、电子设备及存储介质
相关申请的交叉引用
本申请要求申请号为202410437684.5，题为“直播方法、装置、电子设备及存储介质”、申请日为2024年4月11日的中国发明专利申请的优先权，通过引用方式将该申请整体并入本文。
技术领域
本公开涉及数据处理技术领域，尤其涉及直播方法、装置、电子设备及存储介质。
背景技术
随着技术的不断发展，VR(Virtual Reality，虚拟现实)直播也得到很大程度发展，用户可以通过VR设备观看VR直播视频。
发明内容
本公开提供了一种直播方法、装置、电子设备及存储介质。
根据本公开的第一方面，提供了一种直播方法，所述方法包括：
获取源直播视频流，并获取所述源直播视频流的视频投影格式；
在所述源直播视频流的视频投影格式为第一投影格式的情况下，将所述源直播视频流的视频投影格式转换为第二投影格式，获得目标直播视频流；
向终端推送所述目标直播视频流。
根据本公开的第二方面，提供了一种直播方法，所述方法包括：
获取目标直播视频流，并获取所述目标直播视频流的视频投影格式；
在所述目标直播视频流的视频投影格式为第二投影格式的情况下，获取所述目标直播视频流的视频布局信息和视频类型；其中，所述第二投影格式用于将球面图像投影到立方体展开后的各面上，所述立方体外切于所述球面图像；
基于所述视频布局信息和所述视频类型，将所述目标直播视频流切分多个子画面，所述多个子画面包括多个视角；
将所述多个子画面分别映射到目标三维球面，并在终端上显示所述目标三维球面上的视频画面。
根据本公开的第三方面，提供了一种直播装置，所述装置包括：
视频投影格式获取模块，用于获取源直播视频流，并获取所述源直播视频流的视频投影格式；
视频投影格式转换模块，用于在所述源直播视频流的视频投影格式为第一投影格式的情况下，将所述源直播视频流的视频投影格式转换为第二投影格式，获得目标直播视频流；其中，所述第一投影格式用于将球面图像投影为矩形图像，所述第二投影格式用于将球面图像投影到立方体展开后的各面上，所述立方体外切于所述球面图像；
视频流发送模块，用于向终端推送所述目标直播视频流。
根据本公开的第四方面，提供了一种直播装置，所述装置包括：
格式获取模块，用于获取目标直播视频流，并获取所述目标直播视频流的视频投影格式；
视频信息获取模块，用于在所述目标直播视频流的视频投影格式为第二投影格式的情况下，获取所述目标直播视频流的视频布局信息和视频类型；其中，所述第二投影格式用于将球面图像投影到立方体展开后的各面上，所述立方体外切于所述球面图像；
切分模块，用于基于所述视频布局信息和所述视频类型，将所述目标直播视频流切分多个子画面，所述多个子画面包括多个视角；
视频显示模块，用于将所述多个子画面分别映射到目标三维球面，并在终端上显示所述目标三维球面上的视频画面。
根据本公开的第五方面，提供了一种电子设备。该电子设备包括：存储器和处理器，所述存储器上存储有计算机程序，所述处理器执行所述程序时实现如以上所述的方法。
根据本公开的第六方面，提供了一种计算机可读存储介质，其上存储有计算机程序，所述程序被处理器执行时实现本公开的上述方法。
附图说明
在下面结合附图对于示例性实施例的描述中，本公开的更多细节、特征和优点被公开，在附图中：
图1为本公开一示例性实施例提供的VR直播场景示意图；
图2为本公开一示例性实施例提供的将ERP格式的图像转换为EAC格式的图像的示意图；
图3为本公开另一示例性实施例提供的将ERP格式的图像转换为EAC格式的图像的示意图；
图4为本公开又一示例性实施例提供的将ERP格式的图像转换为EAC格式的图像的示意图；
图5为本公开又一示例性实施例提供的将ERP格式的图像转换为EAC格式的图像的示意图；
图6为本公开又一示例性实施例提供的将ERP格式的图像转换为EAC格式的图像的示意图；
图7为本公开一示例性实施例提供的三维网格示意图；
图8为本公开一示例性实施例提供的图像映射到三维网格的示意图；
图9为本公开一示例性实施例提供的直播方法的流程图；
图10为本公开一示例性实施例提供的直播装置的功能模块示意性框图；
图11为本公开一示例性实施例提供的电子设备的结构框图；
图12为本公开一示例性实施例提供的计算机系统的结构框图。
具体实施方式
下面将参照附图更详细地描述本公开的实施例。虽然附图中显示了本公开的某些实施例，然而应当理解的是，本公开可以通过各种形式来实现，而且不应该被解释为限于这里阐述的实施例，相反提供这些实施例是为了更加透彻和完整地理解本公开。应当理解的是，本公开的附图及实施例仅用于示例性作用，并非用于限制本公开的保护范围。
应当理解，本公开的方法实施方式中记载的各个步骤可以按照不同的顺序执行，和/或并行执行。此外，方法实施方式可以包括附加的步骤和/或省略执行示出的步骤。本公开的范围在此方面不受限制。
本文使用的术语“包括”及其变形是开放性包括，即“包括但不限于”。术语“基于”是“至少部分地基于”。术语“一个实施例”表示“至少一个实施例”；术语“另一实施例”表示“至少一个另外的实施例”；术语“一些实施例”表示“至少一些实施例”。其他术语的相关定义将在下文描述中给出。需要注意，本公开中提及的“第一”、“第二”等概念仅用于对不同的装置、模块或单元进行区分，并非用于限定这些装置、模块或单元所执行的功能的顺序或者相互依存关系。
需要注意，本公开中提及的“一个”、“多个”的修饰是示意性而非限制性的，本领域技术人员应当理解，除非在上下文另有明确指出，否则应该理解为“一个或多个”。
本公开实施方式中的多个装置之间所交互的消息或者信息的名称仅用于说明性的目的，而并不是用于对这些消息或信息的范围进行限制。
可以理解的是，在使用本公开各实施例公开的技术方案之前，均应当依据相关法律法规通过恰当的方式对本公开所涉及个人信息的类型、使用范围、使用场景等告知用户并获得用户的授权。
例如，在响应于接收到用户的主动请求时，向用户发送提示信息，以明确地提示用户，其请求执行的操作将需要获取和使用到用户的个人信息。从而，使得用户可以根据提示信息来自主地选择是否向执行本公开技术方案的操作的电子设备、应用程序、服务器或存储介质等软件或硬件提供个人信息。
作为一种可选的但非限定性的实现方式，响应于接收到用户的主动请求，向用户发送提示信息的方式例如可以是弹窗的方式，弹窗中可以以文字的方式呈现提示信息。此外，弹窗中还可以承载供用户选择“同意”或者“不同意”向电子设备提供个人信息的选择控件。可以理解的是，上述通知和获取用户授权过程仅是示意性的，不对本公开的实现方式构成限定，其它满足相关法律法规的方式也可应用于本公开的实现方式中。
用户在通过VR设备观看VR直播视频时，通常会将注意力集中在视频的中间部分，而相关技术中在VR直播视频的非中间部分会产生大量的像素冗余，进而造成VR直播中视频编解码资源和带宽的浪费，对视频的传输和播放造成负面影响。例如，相关技术中通常采用ERP(Equirectangular Projection，等距柱状投影)投影格式的直播视频流。而ERP投影格式的视频会对画面的顶部和底部产生严重的拉伸，且存在大量像素冗余。但在VR直播中，用户的注意力主要集中的区域是VR画面的中间部分，ERP投影格式的直播视频流中顶部和底部(即对应VR画面顶部和底部的边缘)部分，相较于ERP投影格式的直播视频流的中心部分，由于顶部和底部分配了更密集的像素，会产生像素的大量浪费，从而造成VR直播中CDN(Content Delivery Network，内容分发网络)的编解码资源和带宽的浪费，对VR直播链路的首帧时间、秒开率、卡顿率、拉流码率或帧率等指标造成劣化，进而造成VR直播中视频编解码资源和带宽的浪费，对视频的传输和播放造成负面影响。
在本公开提供的实施例中，为解决上述技术问题，如图1所示，服务器可以接收主播端发送的VR直播视频流，该VR直播视频流可以是对主播端通过全景相机或者多个不同角度的视频图像进行拼接，得到360度或者180度的全景图像，获得VR直播视频流，主播端将获得VR直播视频流向服务器发送。
实施例中，可以通过全景相机获取多个摄像头拍摄的视频图像拼接合成为360度或者180度的ERP格式的全景图像，通过将该ERP格式的全景图像投影到球面坐标，得到球面图像，可以根据该球面图像得到VR直播视频流。
实施例中，主播端还可以将上述球面图像转换为ERP格式的二维平面图像，进而基于该二维平面图像获得VR直播视频流。
需要说明的是，上述主播端发送的VR直播视频流，具体还可以是其它投影格式的VR直播视频流，实施例不限于此。
结合图1所示，在接收到VR直播视频流时，在对该VR直播视频流进行转码过程中，可以先对该VR直播视频流通过解封装模块进行解封装，并通过音频解码模块进行解密得到音频流，通过视频解码模块可以得到解码后的直播视频流，通过投影格式转换模块将ERP格式的VR直播视频流转换为EAC(Equi-Angular Cubemap projection，等角度立方体贴图投影)格式的VR直播视频流，并通过视频编码模块对该VR直播视频流进行编码，通过音频编码模块对音频流进行编码，并对编码后的音频流和视频流通过封装模块进行封装，将封装后得到的VR直播视频流通过内容分发网络向各边缘节点分发该VR直播视频流。
其中，视频解码模块、投影格式转换模块和视频编码模块分别位于GPU(Graphics Processing Unit，图形处理器)中，通过GPU执行对视频编解码和投影格式的转换。
因此，服务器在接收到主播端发送的VR直播视频流时，可以首先判断该VR直播视频流是否为ERP格式，如果是ERP格式，就需要将该ERP格式的VR直播视频流转换为EAC格式的VR直播视频流。由于EAC格式的视频可以把像素均匀分散，能够避免在VR直播视频的非中间部分会产生大量的像素冗余。
因此，在接收到VR直播视频流时，在检测到该VR直播视频流的投影格式为ERP格式，还可以根据VR直播视频流的视频类型，判断该VR直播视频流是球面图像还是二维平面图像，以便将ERP格式的VR直播视频流转化为EAC格式的VR直播视频流。
具体的，在VR直播视频流是二维平面图像时，可以直接将ERP格式的VR直播视频流转化为EAC格式的VR直播视频流。在VR直播视频流是球面图像时，例如为360度球面图像，可以在球的中心，向六个方向分别采样，得到立方体的六个画面，再对立方体中每个画面的每个像素都按照下述公式(1)的计算，得到的六个面即为EAC的画面；而在VR直播视频流为180度的球面图像时，可以在球的中心，向五个方向分别采样，得到五个方向的画面。如图2所示，图2是将ERP格式的图像转换为EAC格式的图像的示意图，图2左边是RPR立方体中的一个面，该面上有像素点p，将该像素点p通过公式(1)进行坐标变换得到像素点的坐标q，可以转换为EAC格式的图像。其中，像素点q(u，v)的坐标如下：
实施例中，可以获得VR直播视频流的布局信息，将ERP格式的VR直播视频流转化为EAC格式的VR直播视频流。其中，VR直播视频流可以为FLV(FLASH VIDEO)格式的视频文件，具体可以包括四种布局：2D180度，即180度fov(视角)单目画面；2D360度，即360度fvo单目画面；3D180度，即180度fvo双目画面；以及3D360度，即360度fvo双目画面。
在将ERP格式的VR直播视频流转化为EAC格式的VR直播视频流，如图3所示，如果VR直播视频流为2D180度，一帧视频ERP格式的视频可以转换为五个EAC格式的子画面。
在将ERP格式的VR直播视频流转化为EAC格式的VR直播视频流，如图4所示，如果VR直播视频流为2D360度，一帧视频ERP格式的视频可以转换为六个EAC格式的子画面。
在将ERP格式的VR直播视频流转化为EAC格式的VR直播视频流，如图5所示，如果VR直播视频流为3D180度，一帧左眼和右眼视频的ERP格式的视频画面可以分别转换为五个EAC格式的子画面。
在将ERP格式的VR直播视频流转化为EAC格式的VR直播视频流，如图6所示，如果VR直播视频流为3D180度，一帧左眼和右眼视频的ERP格式的视频画面可以分别转换为六个EAC格式的子画面。
实施例中，由于终端在接收到VR直播视频流时，需要将接收到二维平面图像投影到球面，生成球面视频图像，以便用户可以通过VR终端观看全景视频。因此，在将ERP格式的VR直播视频流转化为EAC格式的VR直播视频流时，可以在视频帧上添加SEI(Supplemental Enhancement Information，补充增强信息)，该SEI可以携带EAC标识，用于表征该视频帧或者该VR直播视频流为EAC格式的视频流，便于VR终端根据VR直播视频流的EAC格式，将EAC格式的VR直播视频流映射到球面坐标，得到全景球面视频图像。
实施例中，可以在生成的EAC格式的VR直播视频流的起始帧添加上述SEI信息，实施例不限于此。
结合图1所示，在完成对音视频流编码诗句的协议层封装之后，可以推流到CDN，还可以在元数据metadata中添加投影格式的字段信息，用于表征该传输的VR直播视频流为EAC格式的视频流。
VR终端上的客户端在接收到边缘节点推送的VR直播视频流时，可以通过解封装模块将VR直播视频流进行解封装处理，得到并通过音频解码模块和视频解码模块对VR直播视频流进行解码处理，得到音频数据量和视频数据流。
在检测到该视频数据流为EAC格式的视频流时，可以根据VR直播视频流EAC布局信息和SEI信息，切分出的多个视角的EAC子画面，并将部分EAC子画面进行旋转。
具体的，如图3所示，在VR直播视频流的EAC布局信息为2D180度时，按照布局切分为左、前、右、下和上共5个子画面，并对其中的上和下两个子画面逆时针旋转90度。
如图4所示，在VR直播视频流的EAC布局信息为2D360度时，按照布局切分为左、前、右、下、后、上共6个子画面，对其中下、上两个子画面逆时针旋转90度对'后’子画面顺时针旋转90度。
如图5所示，在VR直播视频流的EAC布局信息为3D180度时，按照布局切分为成左眼左、左眼前、左眼右、左眼下、左眼上、右眼左、右眼前、右眼右、右眼下、右眼上共10个子画面，对其中左眼下、左眼上、右眼下、右眼上四个子画面逆时针旋转90度。
如图6所示，在VR直播视频流的EAC布局信息为3D360度时，按照布局切分为成左眼左、左眼前、左眼右、左眼下、左眼后、左眼上、右眼左、右眼前、右眼右、右眼下、右眼后、右眼上共12个子画面，对其中左眼下、左眼上、右眼下、右眼上四个子画面逆时针旋转90度，对左眼后、右眼后两个子画面顺时针旋转90度。
实施例中，可以将上述获得的各个EAC子画面映射到三维球面上，并在VR终端上显示三维球面画面。
具体的，以显示EAC 3D180度画面为例，对半球重新生成三维网格mesh，由于EAC是五个方向，该mesh也包括五个部分，通过五个子mesh拼接而成，如图7所示，每个子mesh分别承接对应方向的EAC子画面，这样可以将五个方向的EAC子画面映射到五个子mesh。需要说明的是，如果是360度，EAC会是六个方向的画面采用得到，这样对应的球面mesh会包括六个子mesh。
实施例中，由于上述子mesh承接的是一个四边形的画面，但每个子mesh在三维空间中的边缘是弯曲的，如图7所示，图7中上左侧的子mesh，以及最左侧的mesh的左右两条边是弧形。因此，子mesh的精度不能太低，否则边缘会出现不平滑的弧形。实施例中，可以出于对性能的考虑，精度也不能无限的高，根据业务需求设定为M×N，即可以限制每个面精度为M×N，具体可以根据需要进行设定，实施例不限于此。
在本公开提供的实施例中，mesh可以不按照传统的纺锤形组织顶点，而是按EAC的画面分布提交做拆分。例如，按照纺锤形组织的顶点，如果把EAC子画面的六个面贴合到纺锤形组织顶点的球面时，上下两个面的顶点密度就会比前后左右四个面的顶点高，即六个面的顶点密度不一致。同时上下两个面无法切割出规则的正边形，这样在渲染计算的时候会消耗更多的性能。而本公开实施例通过按EAC分布进行拆分，可以保证六个面的密度均匀，能够产生更少的计算量。
实施例中，UV(纹理映射坐标)可以按照EAC的layout做旋转和偏移，并把EAC映射到球面的arctan计算也包含进去，将五个面(或者六个面)拼接到一起，由于共线部分各自使用自身的UV，所以拼接处也不会出现肉眼可见的接缝。
具体的，实施例中可以在顶点上写入EAC计算的结果，如果是接缝部分，则写入两次。如图8中上面和前面的接缝，接缝的计算值上面计算时写入一次，前面的接缝写入一次，两边各自保存同一位置的转换值，渲染时进行各自计算处理，实现对接缝的处理。
基于上述实施例，本公开实施例还提供了一种直播方法，该方法可以应用于服务器，如图9所示，该方法可以包括如下步骤：
在步骤S110中，获取源直播视频流，并获取源直播视频流的视频投影格式。
实施例中，由于视频投影格式为ERP时的视频流会在非中间部分会产生大量的像素冗余，例如在VR画面顶部和底部的边缘集中了大量的像素，而用户的注意力一般会集中在视频的中间部分，这会导致像素的大量浪费，进而造成视频传输过程中占用过多的编解码资源和带宽资源。其中，该源直播视频流可以上述实施例中的VR直播视频流。
因此，在获得源直播视频流时，需要获得源直播视频流的视频投影格式，如果是ERP格式，就需要进行格式转换，以避免上述在视频传输过程中占用过多的编解码资源和带宽资源。
在步骤S120中，在源直播视频流的视频投影格式为第一投影格式的情况下，将源直播视频流的视频投影格式转换为第二投影格式，获得目标直播视频流。
其中，第一投影格式用于将球面图像投影为矩形图像，第二投影格式用于将球面图像投影到立方体展开后的各面上，立方体外切于球面图像。
实施例中，该第一投影格式可以是上述实施例中的ERP投影格式，该第二投影格式可以为上述实施例中的EAC投影格式，还可以是CMP(CubeMap，立方体贴图投影)投影格式，实施例中以第二投影格式为EAC投影格式为例进行说明，实施例不限于此。
在步骤S130中，向终端推送目标直播视频流。
本公开实施例提供的直播方法，通过检测源直播视频流的视频投影格式，并且在源直播视频流的视频投影格式为等矩柱状投影的情况下，将源直播视频流的视频投影格式转换为等角度立方体贴图投影，这样获得的目标视频流在传输时，可以减少占用过多的编解码资源和带宽资源，进而提高视频的传输效率。
在本公开提供的实施例中，服务器可以接收终端发送的拉流信息，该拉流信息可以携带终端的位置信息。服务器在接收到该拉流信息时，可以根据终端的位置信息，通过CDN与该终端距离最近的边缘节点进行直播视频流的推送，以提高视频的传输效率。因此，服务器可以向内容分发网络发送目标直播视频流；在接收到终端发送的播放请求的情况下，通过内容分发网络向目标边缘节点发送目标直播视频流，并通过目标边缘节点向终端推送目标直播视频流；其中，目标边缘节点基于终端的当前位置确定。
基于上述实施例，在本公开提供的又一实施例中，该方法还可以包括以下步骤：
在步骤S140中，获取目标补充增强信息。其中，该目标补充增强信息携带第二投影格式的视频投影格式标识。
在步骤S150中，在目标直播视频流的目标视频帧中添加目标补充增强信息。
实施例中，以第二投影格式为EAC投影格式为例进行说明，该目标补充增强信息可以携带EAC的视频投影格式标识。该目标补充增强信息具体可以是上述实施例中的SEI信息。可以在各个视频帧中添加该SEI信息，还可以在目标视频帧中添加该SEI信息。例如，可以获取目标直播视频流的起始视频帧，并将起始视频帧作为目标视频帧。这样通过在起始视频帧中添加SEI信息，这样对该起始帧及之后的视频帧进行处理，例如切分为多个子画面，并分别映射到球面坐标。
实施例中，还可以在目标直播视频流的关键帧中添加该目标补充增强信息，实施例不限于此。
基于上述实施例中，结合图1所示，该源直播视频流包括音频数据和视频数据；上述步骤S120具体还可以包括以下步骤：
在步骤S121中，对源直播视频流中的音频数据进行编码，并对转换为第二投影格式的视频数据进行编码。
在步骤S122中，对编码后的音频数据和编码后的视频数据进行封装，得到封装后的音视频流数据，并在封装后的音视频流数据的元数据中添加第二投影格式的视频投影格式字段信息，得到目标直播视频流。
实施例中，仍旧以第二投影格式为EAC投影格式为例进行说明。通过对直播视频流中的音频数据和视频数据分别进行编码，以及对编码后的音频数据和编码后的视频数据进行封装，可以得到封装后的音视频流数据，并在封装后的音视频流数据的元数据中添加等角度立方体贴图投影EAC的视频投影格式字段信息，这样终端在接收到目标直播视频流时，可以直接根据元数据中携带的视频投影格式字段信息判断该目标直播视频流是否为EAC格式的直播视频流，便于根据直播视频流的投影格式对视频进行处理，进而提高视频的处理效率。
基于上述实施例，上述源直播视频流可以包括全景视频，上述步骤S120具体还可以包括以下步骤：
在步骤S123中，获取全景视频的视频类型，并基于视频类型将源直播视频流的视频投影格式转换为第二投影格式。
实施例中，可以基于视频类型将源直播视频流的视频投影格式转换为第二投影格式，仍旧以第二投影格式为EAC投影格式为例进行说明。该视频类型可以为球面图像视频或者二维平面图像视频，在VR直播视频流是二维平面图像时，可以直接将ERP格式的VR直播视频流转化为EAC格式的VR直播视频流。在VR直播视频流是球面图像时，可以在球的中心，向多个方向分别采样，得到多个方向的平面图像，并将该多个方向的平面图像由ERP格式转换为EAC格式。
基于上述实施例，在本公开提供的又一实施例中，还提供了一种直播方法，该方法可以应用终端，例如VR终端，该方法可以包括以下步骤：
在步骤S210中，获取目标直播视频流，并获取目标直播视频流的视频投影格式。
实施例中，终端可以向推流端发送拉流信息，推流端向该终端推送目标直播视频流，例如VR直播视频流，为了将该直播视频流在球面上进行显示，需要获得该目标直播视频流的视频投影格式。
在步骤S220中，在目标直播视频流的视频投影格式为第二投影格式的情况下，获取目标直播视频流的视频布局信息和视频类型。其中，该第二投影格式用于将球面图像投影到立方体展开后的各面上，该立方体外切于球面图像；
实施例中，以第二投影格式为EAC投影格式为例进行说明。
通过获取该目标直播视频流携带的目标补充增强信息，并根据该目标补充增强信息确定该目标直播视频流的投影格式。其中，该目标补充增强信息可以包含在目标视频帧中，该目标视频帧具体可以为该直播视频流的起始帧或者关键帧，还可以在各个视频帧中携带有该目标补充增强信息，实施例不限于此。
因此，实施例中，可以获取目标直播视频流中的目标视频帧，目标视频帧携带目标补充增强信息；目标补充增强信息携带等角度立方体贴图投影EAC的视频投影格式标识；基于视频投影格式标识确定目标直播视频流的视频投影格式。这样通过获取目标视频帧中携带的目标补充增强信息，确定目标直播视频流的视频投影格式，便于终端可以基于该视频投影格式对目标直播视频流进行格式转换处理。
如果该直播视频流为EAC格式，可以获取该目标直播视频流的视频布局信息和视频类型，具体可以参见上述实施例中的描述，该视频布局信息具体可以包括2D180度、2D360度、3D180度或者3D360度，该视频类型可以包括球面图像或者平面图像。
在步骤S230中，基于视频布局信息和视频类型，将目标直播视频流切分多个子画面，多个子画面包括多个视角。
实施例中，可以根据视频类型，对ERP格式的视频转换为EAC格式的视频。还可以结合上述图3～图6所示，根据直播视频流的视频类型，可以将2D180度、2D360度、3D180度或者3D360度的图像换为对应的EAC格式的图像，这样可以得到多个子EAC图像。
在步骤S240中，将多个子画面分别映射到目标三维球面，并在终端上显示目标三维球面上的视频画面。
通过获取目标直播视频流的视频布局信息和视频类型，对EPR格式的视频转换为EAC格式的视频，并根据视频布局信息，将生成的多个EAC子画面分别映射到球面，进而实现在目标三维球面上显示视频画面。这样可以避免因ERP格式的视频在视频的顶部和底部几种大量的像素，造成视频传输过程中占用过多的编解码资源和带宽资源的问题。
基于上述实施例，在本公开提供的又一实施例中，上述步骤S240具体还可以包括以下步骤：
在步骤S241中，基于视频类型生成目标三维球面，并基于目标三维球面生成目标三维球面网格。其中，该目标三维球面网格包括多个三维网格面。
在步骤S242中，将多个子画面分别对应映射到多个三维网格面，得到包含多个子画面的目标三维球面。
实施例中，可以根据视频布局信息，生成多个部分的mesh，以显示EAC 3D180度画面为例，对半球重新生成三维网格mesh，由于EAC是五个方向，该mesh也包括五个部分，通过五个子mesh拼接而成，如图7所示，每个子mesh分别承接对应方向的EAC子画面，这样可以将五个方向的EAC子画面映射到五个子mesh。需要说明的是，如果是360度，EAC会是六个方向的画面采用得到，这样对应的球面mesh会包括六个子mesh。这样可以将多个EAC子画面分别映射到各部分对应的子mesh，并将各mesh中的子画面进行拼接和接缝处理，可以得到目标三维球面上的视频画面。
在采用对应各个功能划分各个功能模块的情况下，本公开实施例提供了一种直播装置，该直播装置可以为服务器、终端或应用于服务器的芯片。图10为本公开一示例性实施例提供的直播装置的功能模块示意性框图。如图10所示，该直播装置包括：
视频投影格式获取模块10，用于获取源直播视频流，并获取所述源直播视频流的视频投影格式；
视频投影格式转换模块20，用于在所述源直播视频流的视频投影格式为第一投影格式的情况下，将所述源直播视频流的视频投影格式转换为第二投影格式，获得目标直播视频流；其中，所述第一投影格式用于将球面图像投影为矩形图像，所述第二投影格式用于将球面图像投影到立方体展开后的各面上，所述立方体外切于所述球面图像；
视频流发送模块30，用于向终端推送所述目标直播视频流。
在本公开提供的又一实施例中，所述装置还包括：
补充增强信息获取模块，用于获取目标补充增强信息，所述目标补充增强信息携带所述第二投影格式的视频投影格式标识；
信息添加模块，用于在所述目标直播视频流的目标视频帧中添加所述目标补充增强信息。
在本公开提供的又一实施例中，所述装置还包括：
目标视频帧确定模块，用于获取所述目标直播视频流的起始视频帧，并将所述起始视频帧作为所述目标视频帧。
在本公开提供的又一实施例中，所述源直播视频流包括音频数据和视频数据；所述视频投影格式转换模块，具体用于：
对所述源直播视频流中的音频数据进行编码，并对转换为所述第二投影格式的视频数据进行编码；
对编码后的所述音频数据和编码后的视频数据进行封装，得到封装后的音视频流数据，并在所述封装后的音视频流数据的元数据中添加所述第二投影格式的视频投影格式字段信息，得到所述目标直播视频流。
在本公开提供的又一实施例中，所述装置还包括：目标直播视频流发送模块，所述目标直播视频流发送模块用于：
向内容分发网络发送所述目标直播视频流；
在接收到终端发送的播放请求的情况下，通过所述内容分发网络向目标边缘节点发送所述目标直播视频流，并通过所述目标边缘节点向所述终端推送所述目标直播视频流；其中，所述目标边缘节点基于所述终端的当前位置确定。
在本公开提供的又一实施例中，所述源直播视频流包括全景视频；
在本公开提供的又一实施例中，所述视频投影格式转换模块，具体用于：
获取所述全景视频的视频类型，并基于所述视频类型将所述源直播视频流的视频投影格式转换为第二投影格式。
有关装置部分，与上述方法相对应，具体参见对应方法的描述，这里不再赘述。
本公开实施例提供的直播装置，通过检测源直播视频流的视频投影格式，并且在源直播视频流的视频投影格式为第一投影格式的情况下，将源直播视频流的视频投影格式转换为第二投影格式，这样获得的目标视频流在传输时，可以减少占用过多的编解码资源和带宽资源，进而提高视频的传输效率。
在采用对应各个功能划分各个功能模块的情况下，本公开实施例提供了一种直播装置，该直播装置可以为服务器、终端或应用于服务器的芯片。该直播装置包括：
格式获取模块，用于获取目标直播视频流，并获取所述目标直播视频流的视频投影格式；
视频信息获取模块，用于在所述目标直播视频流的视频投影格式为第二投影格式的情况下，获取所述目标直播视频流的视频布局信息和视频类型；其中，所述第二投影格式用于将球面图像投影到立方体展开后的各面上，所述立方体外切于所述球面图像；
切分模块，用于基于所述视频布局信息和所述视频类型，将所述目标直播视频流切分多个子画面，所述多个子画面包括多个视角；
视频显示模块，用于将所述多个子画面分别映射到目标三维球面，并在终端上显示所述目标三维球面上的视频画面。
在本公开提供的又一实施例中，所述视频显示模块，具体用于：
基于所述视频类型生成目标三维球面，并基于所述目标三维球面生成目标三维球面网格；所述目标三维球面网格包括多个三维网格面；
将所述多个子画面分别对应映射到所述多个三维网格面，得到包含所述多个子画面的目标三维球面。
在本公开提供的又一实施例中，所述格式获取模块，具体用于：
获取所述目标直播视频流中的目标视频帧，所述目标视频帧携带目标补充增强信息；所述目标补充增强信息携带所述第二投影格式的视频投影格式标识；
基于所述视频投影格式标识确定所述目标直播视频流的视频投影格式。
有关装置部分，与上述方法相对应，具体参见对应方法的描述，这里不再赘述。
实施例中，通过获取目标直播视频流的视频布局信息和视频类型，对第一投影格式的视频转换为第二投影格式的视频，并根据视频布局信息，将生成的多个第二投影格式的子画面分别映射到球面，进而实现在目标三维球面上显示视频画面。这样可以避免因第一投影格式的视频在视频的顶部和底部几种大量的像素，造成视频传输过程中占用过多的编解码资源和带宽资源的问题。
本公开实施例还提供一种电子设备，包括：至少一个处理器；用于存储所述至少一个处理器可执行指令的存储器；其中，所述至少一个处理器被配置为执行所述指令，以实现本公开实施例公开的上述方法。
图11为本公开一示例性实施例提供的电子设备的结构示意图。如图11所示，该电子设备1800包括至少一个处理器1801以及耦接至处理器1801的存储器1802，该处理器1801可以执行本公开实施例公开的上述方法中的相应步骤。
上述处理器1801还可以称为中央处理单元(central processing unit，CPU)，其可以是一种集成电路芯片，具有信号的处理能力。本公开实施例公开的上述方法中的各步骤可以通过处理器1801中的硬件的集成逻辑电路或者软件形式的指令完成。上述的处理器1801可以是通用处理器、数字信号处理器(digital signal processing，DSP)、ASIC(Application Specific Integrated Circuit，专用集成电路)、现成可编程门阵列(field-programmable gate array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。结合本公开实施例所公开的方法的步骤可以直接体现为硬件译码处理器执行完成，或者用译码处理器中的硬件及软件模块组合执行完成。软件模块可以位于存储器1802中，例如随机存储器，闪存、只读存储器，可编程只读存储器或者电可擦写可编程存储器、寄存器等本领域成熟的存储介质。处理器1801读取存储器1802中的信息，结合其硬件完成上述方法的步骤。
另外，根据本公开的各种操作/处理在通过软件和/或固件实现的情况下，可从存储介质或网络向具有专用硬件结构的计算机系统，例如图12所示的计算机系统1900安装构成该软件的程序，该计算机系统在安装有各种程序时，能够执行各种功能，包括诸如前文所述的功能等等。图12为本公开一示例性实施例提供的计算机系统的结构框图。
计算机系统1900旨在表示各种形式的数字电子的计算机设备，诸如，膝上型计算机、台式计算机、工作台、个人数字助理、服务器、刀片式服务器、大型计算机、和其它适合的计算机。电子设备还可以表示各种形式的移动装置，诸如，个人数字处理、蜂窝电话、智能电话、可穿戴设备和其它类似的计算装置。本文所示的部件、它们的连接和关系、以及它们的功能仅仅作为示例，并且不意在限制本文中描述的和/或者要求的本公开的实现。
如图12所示，计算机系统1900包括计算单元1901，该计算单元1901可以根据存储在只读存储器(ROM)1902中的计算机程序或者从存储单元1908加载到随机存取存储器(RAM)1903中的计算机程序，来执行各种适当的动作和处理。在RAM 1903中，还可存储计算机系统1900操作所需的各种程序和数据。计算单元1901、ROM 1902以及RAM 1903通过总线1904彼此相连。输入/输出(I/O)接口1905也连接至总线1904。
计算机系统1900中的多个部件连接至I/O接口1905，包括：输入单元1906、输出单元1907、存储单元1908以及通信单元1909。输入单元1906可以是能向计算机系统1900输入信息的任何类型的设备，输入单元1906可以接收输入的数字或字符信息，以及产生与电子设备的用户设置和/或功能控制有关的键信号输入。输出单元1907可以是能呈现信息的任何类型的设备，并且可以包括但不限于显示器、扬声器、视频/音频输出终端、振动器和/或打印机。存储单元1908可以包括但不限于磁盘、光盘。通信单元1909允许计算机系统1900通过网络诸如因特网的与其他设备交换信息/数据，并且可以包括但不限于调制解调器、网卡、红外通信设备、无线通信收发机和/或芯片组，例如蓝牙TM设备、WiFi设备、WiMax设备、蜂窝通信设备和/或类似物。
计算单元1901可以是各种具有处理和计算能力的通用和/或专用处理组件。计算单元1901的一些示例包括但不限于中央处理单元(CPU)、图形处理单元(GPU)、各种专用的人工智能(AI)计算芯片、各种运行机器学习模型算法的计算单元、数字信号处理器(DSP)、以及任何适当的处理器、控制器、微控制器等。计算单元1901执行上文所描述的各个方法和处理。例如，在一些实施例中，本公开实施例公开的上述方法可被实现为计算机软件程序，其被有形地包含于机器可读介质，例如存储单元1908。在一些实施例中，计算机程序的部分或者全部可以经由ROM 1902和/或通信单元1909而被载入和/或安装到电子设备上。在一些实施例中，计算单元1901可以通过其他任何适当的方式(例如，借助于固件)而被配置为执行本公开实施例公开的上述方法。
本公开实施例还提供一种计算机可读存储介质，其中，当所述计算机可读存储介质中的指令由电子设备的处理器执行时，使得所述电子设备能够执行本公开实施例公开的上述方法。
本公开实施例中的计算机可读存储介质可以是有形的介质，其可以包含或存储以供指令执行系统、装置或设备使用或与指令执行系统、装置或设备结合地使用的程序。上述计算机可读存储介质可以包括但不限于电子的、磁性的、光学的、电磁的、红外的、或半导体系统、装置或设备，或者上述内容的任何合适组合。更具体的，上述计算机可读存储介质可以包括基于一个或多个线的电气连接、便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦除可编程只读存储器(ERPOM或快闪存储器)、光纤、便捷式紧凑盘只读存储器(CD-ROM)、光学储存设备、磁储存设备、或上述内容的任何合适组合。
上述计算机可读介质可以是上述电子设备中所包含的；也可以是单独存在，而未装配入该电子设备中。
本公开实施例还提供一种计算机程序产品，包括计算机程序，其中，所述计算机程序被处理器执行时实现本公开实施例公开的上述方法。
在本公开的实施例中，可以以一种或多种程序设计语言或其组合来编写用于执行本公开的操作的计算机程序代码，上述程序设计语言包括但不限于面向对象的程序设计语言，诸如Java、Smalltalk、C++，还包括常规的过程式程序设计语言，诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络(包括局域网(LAN)或广域网(WAN))连接到用户计算机，或者，可以连接到外部计算机。
附图中的流程图和框图，图示了按照本公开各种实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段、或代码的一部分，该模块、程序段、或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意，在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个接连地表示的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或操作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。
描述于本公开实施例中所涉及到的模块、部件或单元可以通过软件的方式实现，也可以通过硬件的方式来实现。其中，模块、部件或单元的名称在某种情况下并不构成对该模块、部件或单元本身的限定。
本文中以上描述的功能可以至少部分地由一个或多个硬件逻辑部件来执行。例如，非限制性地，可以使用的示例性的硬件逻辑部件包括：现场可编程门阵列(FPGA)、专用集成电路(ASIC)、专用标准产品(ASSP)、片上系统(SOC)、复杂可编程逻辑设备(CPLD)等等。
以上描述仅为本公开的一些实施例以及对所运用技术原理的说明。本领域技术人员应当理解，本公开中所涉及的公开范围，并不限于上述技术特征的特定组合而成的技术方案，同时也应涵盖在不脱离上述公开构思的情况下，由上述技术特征或其等同特征进行任意组合而形成的其它技术方案。例如上述特征与本公开中公开的(但不限于)具有类似功能的技术特征进行互相替换而形成的技术方案。
虽然已经通过示例对本公开的一些特定实施例进行了详细说明，但是本领域的技术人员应该理解，以上示例仅是为了进行说明，而不是为了限制本公开的范围。本领域的技术人员应该理解，可在不脱离本公开的范围和精神的情况下，对以上实施例进行修改。本公开的范围由所附权利要求来限定。

Claims (13)

  1. 一种直播方法，其中所述方法包括：
    获取源直播视频流，并获取所述源直播视频流的视频投影格式；
    在所述源直播视频流的视频投影格式为第一投影格式的情况下，将所述源直播视频流的视频投影格式转换为第二投影格式，获得目标直播视频流；其中，所述第一投影格式用于将球面图像投影为矩形图像，所述第二投影格式用于将球面图像投影到立方体展开后的各面上，所述立方体外切于所述球面图像；
    向终端推送所述目标直播视频流。
  2. 根据权利要求1所述的方法，其中所述方法还包括：
    获取目标补充增强信息，所述目标补充增强信息携带所述第二投影格式的视频投影格式标识；
    在所述目标直播视频流的目标视频帧中添加所述目标补充增强信息。
  3. 根据权利要求2所述的方法，其中所述方法还包括：
    获取所述目标直播视频流的起始视频帧，并将所述起始视频帧作为所述目标视频帧。
  4. 根据权利要求1所述的方法，其中所述源直播视频流包括音频数据和视频数据；
    所述获得目标直播视频流，包括：
    对所述源直播视频流中的音频数据进行编码，并对转换为所述第二投影格式的视频数据进行编码；
    对编码后的所述音频数据和编码后的视频数据进行封装，得到封装后的音视频流数据，并在所述封装后的音视频流数据的元数据中添加所述第二投影格式的视频投影格式字段信息，得到所述目标直播视频流。
  5. 根据权利要求1所述的方法，其中所述方法还包括：
    向内容分发网络发送所述目标直播视频流；
    在接收到终端发送的播放请求的情况下，通过所述内容分发网络向目标边缘节点发送所述目标直播视频流，并通过所述目标边缘节点向所述终端推送所述目标直播视频流；其中，所述目标边缘节点基于所述终端的当前位置确定。
  6. 根据权利要求1所述的方法，其中所述源直播视频流包括全景视频；
    所述将所述源直播视频流的视频投影格式转换为第二投影格式，包括：
    获取所述全景视频的视频类型，并基于所述视频类型将所述源直播视频流的视频投影格式转换为第二投影格式。
  7. 一种直播方法，其中所述方法包括：
    获取目标直播视频流，并获取所述目标直播视频流的视频投影格式；
    在所述目标直播视频流的视频投影格式为第二投影格式的情况下，获取所述目标直播视频流的视频布局信息和视频类型；其中，所述第二投影格式用于将球面图像投影到立方体展开后的各面上，所述立方体外切于所述球面图像；
    基于所述视频布局信息和所述视频类型，将所述目标直播视频流切分多个子画面，所述多个子画面包括多个视角；
    将所述多个子画面分别映射到目标三维球面，并在终端上显示所述目标三维球面上的视频画面。
  8. 根据权利要求7所述的方法，其中所述将所述多个子画面分别映射到目标三维球面，包括：
    基于所述视频类型生成目标三维球面，并基于所述目标三维球面生成目标三维球面网格；所述目标三维球面网格包括多个三维网格面；
    将所述多个子画面分别对应映射到所述多个三维网格面，得到包含所述多个子画面的目标三维球面。
  9. 根据权利要求7所述的方法，其中所述获取所述目标直播视频流的视频投影格式，包括：
    获取所述目标直播视频流中的目标视频帧，所述目标视频帧携带目标补充增强信息；所述目标补充增强信息携带所述第二投影格式的视频投影格式标识；
    基于所述视频投影格式标识确定所述目标直播视频流的视频投影格式。
  10. 一种直播装置，其中所述装置包括：
    视频投影格式获取模块，用于获取源直播视频流，并获取所述源直播视频流的视频投影格式；
    视频投影格式转换模块，用于在所述源直播视频流的视频投影格式为第一投影格式的情况下，将所述源直播视频流的视频投影格式转换为第二投影格式，获得目标直播视频流；其中，所述第一投影格式用于将球面图像投影为矩形图像，所述第二投影格式用于将球面图像投影到立方体展开后的各面上，所述立方体外切于所述球面图像；
    视频流发送模块，用于向终端推送所述目标直播视频流。
  11. 一种直播装置，其中所述装置包括：
    格式获取模块，用于获取目标直播视频流，并获取所述目标直播视频流的视频投影格式；
    视频信息获取模块，用于在所述目标直播视频流的视频投影格式为第二投影格式的情况下，获取所述目标直播视频流的视频布局信息和视频类型；其中，所述第二投影格式用于将球面图像投影到立方体展开后的各面上，所述立方体外切于所述球面图像；
    切分模块，用于基于所述视频布局信息和所述视频类型，将所述目标直播视频流切分多个子画面，所述多个子画面包括多个视角；
    视频显示模块，用于将所述多个子画面分别映射到目标三维球面，并在终端上显示所述目标三维球面上的视频画面。
  12. 一种电子设备，其中所述设备包括：
    至少一个处理器；
    用于存储所述至少一个处理器可执行指令的存储器；
    其中，所述至少一个处理器被配置为执行所述指令，以实现如权利要求1-9中任一项所述的方法。
  13. 一种计算机可读存储介质，其中当所述计算机可读存储介质中的指令由电子设备的处理器执行时，使得所述电子设备能够执行如权利要求1-9中任一项所述的方法。
PCT/CN2024/139663 2024-04-11 2024-12-16 直播方法、装置、电子设备及存储介质 Pending WO2025213828A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202410437684.5A CN118175354A (zh) 2024-04-11 2024-04-11 直播方法、装置、电子设备及存储介质
CN202410437684.5 2024-04-11

Publications (1)

Publication Number Publication Date
WO2025213828A1 true WO2025213828A1 (zh) 2025-10-16

Family

ID=91360202

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2024/139663 Pending WO2025213828A1 (zh) 2024-04-11 2024-12-16 直播方法、装置、电子设备及存储介质

Country Status (2)

Country Link
CN (1) CN118175354A (zh)
WO (1) WO2025213828A1 (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118175354A (zh) * 2024-04-11 2024-06-11 北京字跳网络技术有限公司 直播方法、装置、电子设备及存储介质
WO2026000369A1 (zh) * 2024-06-28 2026-01-02 影石创新科技股份有限公司 影像处理方法、云端服务器、电子设备及存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200053393A1 (en) * 2017-02-27 2020-02-13 Koninklijke Kpn N.V. Processing Spherical Video Data
CN112565736A (zh) * 2020-11-25 2021-03-26 聚好看科技股份有限公司 一种全景视频显示方法及显示设备
CN113014924A (zh) * 2019-12-20 2021-06-22 中国电信股份有限公司 视频编码方法、服务器、装置及计算机可读存储介质
CN113206992A (zh) * 2021-04-20 2021-08-03 聚好看科技股份有限公司 一种转换全景视频投影格式的方法及显示设备
CN117793356A (zh) * 2022-09-20 2024-03-29 抖音视界有限公司 一种视频编码方法及装置
CN118175354A (zh) * 2024-04-11 2024-06-11 北京字跳网络技术有限公司 直播方法、装置、电子设备及存储介质

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108616747A (zh) * 2016-12-23 2018-10-02 上海微湃电子科技有限公司 全景视频及图片的播放方法
MX2022015674A (es) * 2020-06-09 2023-01-16 Bytedance Inc Informacion de decodificador de video de referencia hipotetica anidada no escalable de se?alizacion.
CN114339268B (zh) * 2020-10-10 2023-08-29 腾讯科技（深圳）有限公司 一种直播数据处理方法、装置和计算机可读存储介质

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200053393A1 (en) * 2017-02-27 2020-02-13 Koninklijke Kpn N.V. Processing Spherical Video Data
CN113014924A (zh) * 2019-12-20 2021-06-22 中国电信股份有限公司 视频编码方法、服务器、装置及计算机可读存储介质
CN112565736A (zh) * 2020-11-25 2021-03-26 聚好看科技股份有限公司 一种全景视频显示方法及显示设备
CN113206992A (zh) * 2021-04-20 2021-08-03 聚好看科技股份有限公司 一种转换全景视频投影格式的方法及显示设备
CN117793356A (zh) * 2022-09-20 2024-03-29 抖音视界有限公司 一种视频编码方法及装置
CN118175354A (zh) * 2024-04-11 2024-06-11 北京字跳网络技术有限公司 直播方法、装置、电子设备及存储介质

Also Published As

Publication number Publication date
CN118175354A (zh) 2024-06-11

Similar Documents

Publication Publication Date Title
TWI619088B (zh) 圖像資料處理系統和相關方法以及相關圖像融合方法
WO2025213828A1 (zh) 直播方法、装置、电子设备及存储介质
CN107454468B (zh) 对沉浸式视频进行格式化的方法、装置和流
US20140092439A1 (en) Encoding images using a 3d mesh of polygons and corresponding textures
WO2018121333A1 (zh) 一种360度vr全景图形图像及视频实时生成方法
WO2017113681A1 (zh) 一种基于虚拟现实技术的视频图像处理方法及装置
WO2024104248A1 (zh) 虚拟全景图的渲染方法、装置、设备及存储介质
CN111447504B (zh) 三维视频的处理方法、装置、可读存储介质和电子设备
CN108780584A (zh) 用于流传输和渲染的球形视频的转换和预处理
CN112843676B (zh) 数据处理方法、装置、终端、服务器及存储介质
CN108174265A (zh) 一种360度全景视频的播放方法、装置及系统
WO2019007120A1 (zh) 一种媒体数据的处理方法和装置
CN114866760A (zh) 一种虚拟现实显示方法、设备、系统和可读存储介质
WO2023066054A1 (zh) 图像处理方法、云服务器、vr终端、存储介质
CN115002442B (zh) 一种图像展示方法、装置、电子设备及存储介质
JP7471731B2 (ja) メディアファイルのカプセル化方法、メディアファイルのカプセル化解除方法及び関連機器
CN111640191A (zh) 基于vr一体机的投录屏画面采集处理方法
CN114567742A (zh) 全景视频的传输方法、装置及存储介质
WO2024027611A1 (zh) 视频直播方法、装置、电子设备以及存储介质
CN114820660B (zh) 图像处理方法、装置、电子设备、存储介质及程序产品
US20240331202A1 (en) Fisheye image compression method, fisheye video stream compression method and panoramic video generation method
WO2025218295A1 (zh) 播放全景视频的方法、装置、设备及存储介质
CN112954452A (zh) 视频生成方法、装置、终端及存储介质
CN113014924A (zh) 视频编码方法、服务器、装置及计算机可读存储介质
JP2022513487A (ja) 没入型ビデオビットストリーム処理

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 24934891

Country of ref document: EP

Kind code of ref document: A1