WO2023130893A1 - 流媒体传输方法、装置、电子设备及计算机可读存储介质 - Google Patents

流媒体传输方法、装置、电子设备及计算机可读存储介质 Download PDF

Info

Publication number
WO2023130893A1
WO2023130893A1 PCT/CN2022/137754 CN2022137754W WO2023130893A1 WO 2023130893 A1 WO2023130893 A1 WO 2023130893A1 CN 2022137754 W CN2022137754 W CN 2022137754W WO 2023130893 A1 WO2023130893 A1 WO 2023130893A1
Authority
WO
WIPO (PCT)
Prior art keywords
streaming media
sorting
objects
media object
relationship
Prior art date
Application number
PCT/CN2022/137754
Other languages
English (en)
French (fr)
Inventor
胡颖
Original Assignee
腾讯科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 腾讯科技（深圳）有限公司 filed Critical 腾讯科技（深圳）有限公司
Publication of WO2023130893A1 publication Critical patent/WO2023130893A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/80—Responding to QoS
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets

Definitions

  • the present application belongs to the technical field of audio and video, and specifically relates to a streaming media transmission method, device, electronic equipment, computer readable storage medium and computer program product.
  • Streaming media refers to a technology and process that compresses a series of media data, sends the data in segments through the Internet, and instantly transmits video and audio on the Internet for viewing. This technology enables data packets to be sent like flowing water.
  • the characteristics of real-time segmental transmission of streaming media determine that users can obtain media data in real time without downloading complete media files, but at the same time, it also puts forward high requirements on user equipment performance and network conditions. Once the performance of the device is degraded or the network status is not ideal, the problem of media data transmission is likely to be blocked. Therefore, streaming media transmission generally has problems such as low transmission efficiency and poor stability.
  • the purpose of the present application is to provide a streaming media transmission method, device, electronic equipment, computer readable storage medium and computer program product.
  • a streaming media transmission method executed by an electronic device comprising:
  • the sorting relationship descriptor includes a sorting identification field and a sorting information field; the sorting identification field is used to indicate the sorting range; the sorting information field is used as a sorting basis ;
  • At least two streaming media objects in the sorting relationship set are sorted to obtain a streaming media object sequence
  • the target streaming media object to be received is selected from the streaming media object sequence, and a streaming media transmission request for the target streaming media object is sent, and the streaming media transmission request is used to request to acquire the target streaming media object.
  • a streaming media transmission method executed by an electronic device comprising:
  • the sorting range respectively configure the sorting identification field of the sorting relationship descriptor of at least one streaming media object
  • the sorting basis respectively configure the sorting information fields of the sorting relationship descriptors of at least one streaming media object
  • a streaming media transmission device comprising:
  • the signaling parsing module is configured to analyze the streaming media transmission signaling to obtain the respective sorting relationship descriptors of each streaming media object.
  • the sorting relationship descriptor includes a sorting identification field to sort the information field; the sorting identification field is used to indicate the sorting range; The sorting information field is used as a sorting basis;
  • the object collection module is configured to form at least two streaming media objects with the same sort identification field into a sort relationship set
  • the object sorting module is configured to sort at least two streaming media objects in the sorting relationship set according to the sorting information field to obtain a streaming media object sequence;
  • the request sending module is configured to select a target streaming media object to be received from the streaming media object sequence, and send a streaming media transmission request for the target streaming media object, and the streaming media transmission request is used to request to acquire the target streaming media object.
  • a streaming media transmission device comprising:
  • An acquisition module configured to acquire the respective sorting ranges and respective sorting basis of at least one streaming media object
  • the range configuration module is configured to configure the sort identification fields of the sort relationship descriptors of at least one streaming media object respectively according to the sort range;
  • the basis configuration module is configured to respectively configure the sorting information fields of the sorting relationship descriptors of at least one streaming media object according to the sorting basis;
  • the signaling sending module is configured to generate streaming media transmission instructions based on the ordering relationship descriptors of at least one streaming media object, and send the streaming media transmission signaling to the data receiver;
  • the object returning module is configured to respond to the streaming media transmission request corresponding to the streaming media transmission signaling sent by the data receiver, and return to the data receiver a target streaming media object that matches the streaming media transmission request .
  • a computer-readable storage medium stores at least one computer-readable instruction, and the at least one computer-readable instruction is loaded by one or more processors and Execute to implement the operations performed in the streaming media transmission method in the embodiment of the present application.
  • an electronic device includes: one or more processors and a memory, the memory is used to store at least one computer-readable instruction, and the at least one computer-readable instruction is executed by the One or more processors are loaded and executed to implement the operations performed in the streaming media transmission method in the embodiments of the present application.
  • a computer program product includes computer-readable instructions, and the computer-readable instructions are stored in a computer-readable storage medium.
  • One or more processors of the computer device read the computer-readable instructions from the computer-readable storage medium, and the processors execute the computer-readable instructions, so that the computer device executes the streaming media transmission method in the above technical solution.
  • Fig. 1 shows a schematic diagram of an exemplary system architecture to which the technical solutions of the embodiments of the present application can be applied.
  • Figure 2 illustrates the placement of a video encoding device and a video decoding device in a streaming environment.
  • Figure 3 shows a basic flow diagram of a video encoder.
  • Fig. 4 shows a flow chart of the steps of the streaming media transmission method executed by the decoding side in some embodiments of the present application.
  • Fig. 5 shows a flowchart of the steps of the streaming media transmission method executed by the encoding side in some embodiments of the present application.
  • FIG. 6 shows a structural block diagram of a streaming media transmission device on a decoding side provided by an embodiment of the present application.
  • Fig. 7 shows a structural block diagram of a streaming media transmission device on the encoding side in some embodiments of the present application.
  • FIG. 8 shows a structural block diagram of a computer system suitable for implementing an electronic device according to an embodiment of the present application.
  • Example embodiments will now be described more fully with reference to the accompanying drawings.
  • Example embodiments may, however, be embodied in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be thorough and complete, and will fully convey the concepts of example embodiments to those skilled in the art.
  • the "plurality” mentioned in this article refers to two or more than two.
  • “And/or” describes the association relationship of associated objects, indicating that there may be three types of relationships. For example, A and/or B may indicate: A exists alone, A and B exist simultaneously, and B exists independently. The character “/” generally indicates that the contextual objects are an "or” relationship.
  • user-related data such as streaming media resources are involved.
  • user permission or consent When each embodiment of this application is applied to a specific product or technology, it is necessary to obtain user permission or consent, and the collection of relevant data, Use and processing need to comply with relevant laws, regulations and standards of relevant countries and regions.
  • DASH dynamic adaptive streaming over HTTP (dynamic adaptive streaming based on HTTP (Hyper Text Transfer Protocol, hypertext transfer protocol)), HTTP-based dynamic adaptive streaming is an adaptive bitrate streaming technology that enables high-quality streaming media Can be delivered over the Internet via conventional HTTP web servers.
  • HTTP Dynamic adaptive streaming based on HTTP (Hyper Text Transfer Protocol, hypertext transfer protocol)
  • HTTP-based dynamic adaptive streaming is an adaptive bitrate streaming technology that enables high-quality streaming media Can be delivered over the Internet via conventional HTTP web servers.
  • MPD media presentation description, media presentation description signaling in DASH, used to describe media segment information.
  • DASH In DASH, a combination of one or more media components, such as a video file with a certain resolution, can be regarded as a Representation.
  • Adaptation Sets In DASH, a collection of one or more video streams, one Adaptation Sets can contain multiple Representations.
  • Multi-view/multi-view video refers to a video with depth information taken from multiple angles using multiple camera arrays.
  • Multi-view/multi-viewpoint video also called free-view/free-viewpoint video, is an immersive media that provides a six-degree-of-freedom experience.
  • Point cloud is a set of discrete point sets randomly distributed in space that express the spatial structure and surface properties of a three-dimensional object or scene. Each point in the point cloud has at least three-dimensional position information, and may also have color, material or other information depending on the application scenario. Typically, each point in a point cloud has the same number of additional attributes.
  • V3C volumetric media visual volumetric video-based coding media, refers to immersive media that captures visual content in three-dimensional space and provides 3DoF+ and 6DoF viewing experience, is encoded with traditional video, and contains volumetric video tracks in the file package, including multiple Perspective video, video encoded point cloud, etc.
  • DoF Degree of Freedom, degree of freedom. In this application, it refers to the freedom of motion supported by the user when viewing immersive media and resulting in content interaction.
  • 3DoF Three degrees of freedom, referring to the three degrees of freedom in which the user's head rotates around the x, y, and z axes.
  • 3DoF+ On the basis of three degrees of freedom, the user also has limited freedom of movement along the x, y, and z axes.
  • 6DoF On the basis of three degrees of freedom, the user also has the freedom to move freely along the x, y, and z axes.
  • Fig. 1 shows a schematic diagram of an exemplary system architecture to which the technical solutions of the embodiments of the present application can be applied.
  • the system architecture 100 includes a plurality of end devices that can communicate with each other through, for example, a network 150 .
  • the system architecture 100 may include a first terminal device 110 and a second terminal device 120 interconnected by a network 150 .
  • the first terminal device 110 and the second terminal device 120 perform unidirectional data transmission.
  • the first terminal device 110 can encode video data (such as a video picture stream collected by the first terminal device 110) to be transmitted to the second terminal device 120 through the network 150, and the encoded video data is represented by one or more
  • the second terminal device 120 can receive the encoded video data from the network 150, decode the encoded video data to restore the video data, and display the video picture according to the restored video data.
  • the system architecture 100 may include a third terminal device 130 and a fourth terminal device 140 performing bi-directional transmission of encoded video data, such as may occur during a video conference.
  • the third terminal device 130 can encode video data (such as a video picture stream captured by the third terminal device) for transmission to the fourth terminal device 140 via the network 150 .
  • the third terminal device 130 may also receive encoded video data transmitted by the fourth terminal device 140, decode the encoded video data to restore the video data, and display video pictures on an accessible display device according to the restored video data.
  • the first terminal device 110, the second terminal device 120, the third terminal device 130 and the fourth terminal device 140 may be servers, personal computers and smart phones, but the principles disclosed in this application may not be limited thereto .
  • Embodiments disclosed herein are suitable for use with laptop computers, tablet computers, media players, and/or dedicated video conferencing equipment.
  • Network 150 represents any number of networks, including for example wired and/or wireless communication networks, that communicate encoded video data between first terminal device 110, second terminal device 120, third terminal device 130, and fourth terminal device 140.
  • Communication network 150 may exchange data in circuit-switched and/or packet-switched channels.
  • the network may include a telecommunications network, a local area network, a wide area network and/or the Internet.
  • the architecture and topology of network 150 may be immaterial to the operation of the present disclosure.
  • the server in this embodiment of the application may be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services.
  • the terminal may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, a vehicle terminal, a smart TV, etc., but is not limited thereto.
  • the terminal and the server may be connected directly or indirectly through wired or wireless communication, which is not limited in this application.
  • FIG. 2 shows how a video encoding device and a video decoding device are placed in a streaming environment.
  • the subject matter disclosed herein is equally applicable to other video-enabled applications including, for example, videoconferencing, digital TV (television), storing compressed video on digital media including CDs, DVDs, memory sticks, and the like.
  • the streaming transmission system includes an acquisition subsystem 213 , and the acquisition subsystem 213 includes a video source 201 such as a digital camera, and the video source 201 creates an uncompressed video picture stream 202 .
  • video picture stream 202 includes samples captured by a digital camera.
  • the video picture stream 202 is depicted as a thick line to emphasize the video picture stream of high data volume.
  • the video picture stream 202 can be processed by the electronic device 220, the electronic device 220
  • the device 220 comprises a video encoding device 203 coupled to the video source 201 .
  • Video encoding device 203 includes hardware, software, or a combination of hardware and software to implement or implement aspects of the disclosed subject matter as described in more detail below.
  • the encoded video data 204 (or encoded video code stream 204 ) is depicted as a thinner line to emphasize the lower data volume of the encoded video data 204 (or encoded video code stream 204 ) compared to the video picture stream 202 . 204), which may be stored on the streaming server 205 for future use.
  • One or more streaming client subsystems such as client subsystem 206 and client subsystem 208 in FIG. 2 , may access streaming server 205 to retrieve copies 207 and 209 of encoded video data 204 .
  • Client subsystem 206 includes electronic device 230 , and electronic device 230 includes video decoding device 210 .
  • a video decoding device 210 decodes an incoming copy 207 of encoded video data, producing an output video picture stream 211 that can be presented on a display 212 (eg, a screen) or another presentation device.
  • the encoded video data 204, video data 207, and video data 209 may be encoded according to a video encoding/compression standard.
  • examples of applicable standards include ITU-TH.265 and the like.
  • the electronic device 220 and the electronic device 230 may include other components not shown in the figure.
  • the electronic device 220 may include a video decoding device
  • the electronic device 230 may also include a video encoding device.
  • the video frame image will be divided into several non-overlapping processing units according to a block size, and each processing unit will perform a similar compression operation.
  • This processing unit is called CTU (Coding Tree Unit, coding tree unit), or LCU (Largest Coding Unit, largest coding unit).
  • the CTU can be further divided into finer divisions to obtain one or more basic coding units CU, which are the most basic elements in a coding process.
  • Predictive coding includes intra-frame prediction and inter-frame prediction. After the original video signal is predicted by the selected reconstructed video signal, the residual video signal is obtained. The encoder needs to decide which predictive coding mode to choose for the current CU, and inform the decoder. Among them, intra-frame prediction means that the predicted signal comes from the area that has been coded and reconstructed in the same image; inter-frame prediction means that the predicted signal comes from another image (called a reference image) that has been encoded and is different from the current image. .
  • Transform&Quantization After the residual video signal undergoes transformation operations such as DFT (Discrete Fourier Transform, Discrete Fourier Transform), DCT (Discrete Cosine Transform, Discrete Cosine Transform), the signal is converted into the transform domain, called is the transformation coefficient.
  • the transform coefficients are further subjected to a lossy quantization operation, and certain information is lost, so that the quantized signal is conducive to compressed expression.
  • there may be more than one transformation method for selection so the encoder also needs to select one of the transformation methods for the current CU and inform the decoder.
  • Entropy Coding or Statistical Coding The quantized transform domain signal will be statistically compressed and coded according to the frequency of occurrence of each value, and finally output a binary (0 or 1) compressed code stream. At the same time, other information generated by encoding, such as the selected encoding mode, motion vector data, etc., also needs to be entropy encoded to reduce the bit rate.
  • Statistical coding is a lossless coding method that can effectively reduce the bit rate required to express the same signal. Common statistical coding methods include variable length coding (Variable Length Coding, VLC for short) or context-based binary arithmetic coding ( Content Adaptive Binary Arithmetic Coding, referred to as CABAC).
  • Loop Filtering The changed and quantized signal will be reconstructed through inverse quantization, inverse transformation and prediction compensation operations. Compared with the original image, due to the influence of quantization, some information of the reconstructed image is different from the original image, that is, the reconstructed image will produce distortion (Distortion). Therefore, filtering operations can be performed on the reconstructed image, such as filters such as Deblocking filter (DB for short), SAO (Sample Adaptive Offset, adaptive pixel compensation) or ALF (Adaptive Loop Filter, adaptive loop filter) , which can effectively reduce the degree of distortion generated by quantization. Since these filtered reconstructed images will be used as references for subsequent encoded images to predict future image signals, the above filtering operation is also called in-loop filtering, that is, a filtering operation in an encoding loop.
  • DB Deblocking filter
  • SAO Sample Adaptive Offset, adaptive pixel compensation
  • ALF Adaptive Loop Filter, adaptive loop filter
  • FIG. 3 shows a basic flow chart of a video encoder, in which intra prediction is taken as an example for illustration.
  • the difference between the original image signal and the predicted image signal will be calculated to obtain the residual signal, and then the residual signal will be transformed and quantized to obtain the quantized coefficient, and the encoded bit stream corresponding to the quantized coefficient will be obtained through entropy coding.
  • Quantization and inverse transformation process to obtain the reconstructed residual signal corresponding to the quantized coefficient, superimpose the predicted image signal and the reconstructed residual signal to generate an image signal, and input the image signal to the intra-frame mode decision module and the intra-frame prediction module for intra-frame prediction processing , obtain the intra-frame prediction result, and perform loop filtering on the image signal to obtain a reconstructed image signal, which can be used as a reference image for the next frame for motion estimation and motion compensation prediction, and motion compensation prediction based on the reconstructed image signal , obtain the result of motion compensation prediction, obtain the predicted image signal of the next frame based on the result of motion compensation prediction and intra-frame prediction, and continue to repeat the above process until the encoding is completed.
  • the compressed code stream will be entropy decoded to obtain various mode information and quantization Coefficients, and then perform inverse quantization and inverse transformation processing on the quantized coefficients to obtain the residual signal.
  • the prediction signal corresponding to the coding unit
  • the residual signal is then subjected to operations such as loop filtering to generate a final output signal.
  • the streaming transmission technology is usually used to handle the transmission of media resources between the server and the client.
  • Common media streaming transmission technologies include DASH (Dynamic Adaptive Streaming over HTTP), HLS (HTTP Live Streaming, HTTP-based streaming media network transmission protocol), SMT (Smart Media Transport, intelligent media transmission) and other technologies.
  • DASH is an adaptive bitrate streaming technology that enables high-quality streaming media to be delivered over the Internet through traditional HTTP web servers.
  • DASH breaks down content into a series of small HTTP-based file segments, each segment containing a short length of playable content that could be several hours in length (such as a movie or live sports event). Content will be segmented into multiple bitrate alternatives to provide multiple bitrate versions for selection.
  • the client will automatically choose which alternative to download and play according to the current network conditions. The client will choose the highest bitrate segment that can be downloaded in time to play, avoiding playback stuttering or rebuffering events. Because of this, the DASH client can seamlessly adapt to changing network conditions and provide a high-quality playback experience with less stuttering and re-buffering.
  • DASH uses existing HTTP web server infrastructure. It allows devices such as Internet TVs, TV set-top boxes, desktop computers, smartphones, tablets, etc. to consume multimedia content (such as video, TV, radio, etc.) delivered over the Internet and can cope with changing Internet reception conditions.
  • multimedia content such as video, TV, radio, etc.
  • this embodiment of the present application provides a method for indicating information such as quality and priority of streaming media data.
  • relevant information such as the quality and priority of streaming media data within a specific range can be flexibly defined and indicated, so that the client can request corresponding streaming media data according to its own needs.
  • Readable storage medium The various technical solutions of the embodiments of the present application can be applied to links such as the server end, player end, and intermediate nodes of the streaming media transmission system.
  • Figure 4 shows a flow chart of the steps of the streaming media transmission method performed by the decoding side in some embodiments of the present application
  • the decoding side can be understood as the data receiver receiving streaming media data
  • the method can be applied to the user's terminal equipment or can be Applied to the intermediate node for data communication between the server and the terminal device.
  • the streaming media transmission method may include the following steps S410 to S440.
  • step S410 the streaming media transmission signaling is analyzed to obtain the sorting relationship descriptors of each streaming media object.
  • the sorting relationship descriptor includes a sorting identification field and a sorting information field.
  • the sorting identification field is used to indicate the sorting range, and the sorting information field Used as sort by.
  • Streaming media transmission signaling is a message transmitted between the data sender and data receiver to coordinate the communication process.
  • streaming media transmission signaling can be DASH signaling based on the DASH protocol, or SMT signaling .
  • the sorting relationship descriptor is used to define the quality sorting information, priority sorting or other dimension sorting information between different streaming media objects, and the sorting relationship descriptor can be defined by different elements in different types of signaling.
  • the ranking relationship descriptor may be defined as an element RankingRelationship in DASH signaling
  • the ranking relationship descriptor may be defined as an element Ranking_set_descriptor in SMT signaling.
  • the data receiver can obtain the sorting relationship descriptors of each streaming media object by analyzing the streaming media transmission signaling.
  • the sorting relationship descriptor includes a sorting identification field and a sorting information field. Based on the sorting identification field, the Each streaming media object is divided into at least one sorting range, and for each sorting range in the at least one sorting range, the streaming media objects can be sorted within the targeted sorting range based on the sorting information field.
  • step S420 at least two streaming media objects having the same sort identification field are combined into a sort relationship set.
  • the streaming media transmission signaling carries multiple sorting relationship descriptors of streaming media objects.
  • the data receiver can determine the sorting relationship between each streaming media object by comparing the sorting relationship descriptors.
  • the sorting relationship includes sorting range and sorting There are two levels of position, where the sorting scope is determined according to the sorting identification field in the sorting relationship descriptor. When any two streaming media objects have the same sorting identification field, it means that the two streaming media objects belong to the same sorting range, and the streaming media objects within the same sorting range can be combined into a sorting relationship set.
  • streaming media object A and streaming media object B both take a value of 1
  • the sorting identifier fields of streaming media object C and streaming media object D both take a value of 2
  • streaming media object A and streaming media Object B forms a sorting relationship set
  • streaming media object C and streaming media object D can form another sorting relationship set.
  • a streaming media object may have an ordering relationship descriptor, thus, the streaming media object may have an ordering identification field, based on the ordering identification field of the streaming media object, the streaming media object may be Combined into the set of sort relations corresponding to the sort identifier field.
  • a streaming media object may also have multiple different sorting relationship descriptors at the same time, and the values of the sorting identifier fields included in each sorting relationship descriptor may be the same or different.
  • the streaming media object has at least two sorting identification fields with different values, and accordingly, the streaming media object can be combined into In the set of sorting relationships corresponding to at least two sorting identification fields with different values respectively.
  • streaming media object A can have two sorting relationship descriptors (assumed to be sorting relationship descriptor 1 and sorting relationship description 2) at the same time, and the value of the sorting identification field included in sorting relationship descriptor 1 is 1, and the sorting relationship descriptor The value of the sorting identification field included in the relationship descriptor 2 is 2, then the streaming media object A can be combined into the sorting relationship set 1 corresponding to the value of the sorting identification field 1, and the streaming media object A can be combined into In the sort relationship set 2 corresponding to the value 2 of the sort identifier field.
  • step S430 at least two streaming media objects in the sorting relationship set are sorted according to the sorting information field to obtain a sequence of streaming media objects.
  • the data receiver can sort the at least two streaming media objects distributed in the sorting relationship set according to the value of the sorting information field, and obtain the stream media objects arranged in the specified order A sequence of streaming media objects.
  • the data receiver can sort the at least two streaming media objects in the sorting relationship set according to the value of the sorting information field from small to large, or can also sort according to the value of the sorting information field from large to large.
  • the small order sorts at least two streaming media objects in the sort relation set.
  • the sorting information field may include at least one of quality sorting information, priority sorting information, or undefined sorting information
  • the quality sorting information is used to indicate the sorting of streaming media objects according to the streaming media quality Level
  • the priority sorting information is used to indicate the sorting level for sorting the streaming media objects according to the selected priority
  • the undefined sorting information is used to indicate the sorting level for sorting the streaming media objects according to the specified rules.
  • the specified rules can be configured according to actual application scenarios.
  • the flexibility and diversity of streaming media object sorting can be improved, thereby providing a variety of options for network transmission of streaming media objects, and ensuring the stability of streaming media data transmission performance and reliability, and improve the transmission efficiency of streaming media data.
  • the sorting information field may include one type of specified information among quality sorting information, priority sorting information, or undefined sorting information, or may include two or more types of specified information.
  • the data receiver can sort at least two streaming media objects in the sorting relationship set according to the same type of specified information, or can sort at least two stream media objects in the sorting relationship set according to multiple types of specified information. Streaming objects are sorted comprehensively.
  • step S440 a target streaming media object to be received is selected from the streaming media object sequence, and a streaming media transmission request for the target streaming media object is sent, and the streaming media transmission request is used to request acquisition of the target streaming media object.
  • the data receiver can select an object that meets its own data requirements from the streaming media object sequence as the target streaming media object to be received.
  • the target streaming media object can be a single streaming media object, or It can be all streaming media objects in a composite object.
  • the streaming media object with higher quality or higher priority can be preferentially selected as the target streaming media object to be received.
  • a streaming media object with a lower quality ranking or a lower priority ranking may be selected as the target streaming media object to be received.
  • the data receiver can send a streaming media transmission request for the target streaming media object to the data sender.
  • the receiver transmits the corresponding target streaming media object.
  • the streaming media transmission method by dividing the streaming media objects into multiple different sorting ranges, and sorting the streaming media objects according to different sorting criteria within a specific sorting range, the streaming media objects can be improved.
  • the flexibility and diversity of sorting can provide a variety of options for the network transmission of streaming media objects, and then enable the data receiver to analyze the streaming media transmission signaling to obtain the sorting relationship descriptors of each streaming media object.
  • At least two streaming media objects with the same sorting identification field form a sorting relationship set, sort at least two streaming media objects in the sorting relationship set according to the sorting information field, and obtain the sequence of streaming media objects according to the performance of the device itself Select the target streaming media object to be received from the streaming media object sequence and send the streaming media transmission request for the target streaming media object to obtain the target streaming media object, which can ensure the stability and reliability of streaming media data transmission and improve Transmission efficiency of streaming media data.
  • the sorting relationship descriptor also includes a combination judgment field for indicating whether to perform combined sorting on the streaming media objects, and according to the sorting information field, sort at least two streaming media objects in the sorting relationship set to obtain
  • the streaming media object sequence includes: based on the combination judgment fields of at least two streaming media objects in the sorting relationship set, obtaining the combination judgment result, based on the combination judgment result and the sorting information field, at least two streaming media objects in the sorting relationship set Sort to get the sequence of streaming media objects.
  • the combination determination field may be used to determine whether the streaming media objects need to be combined and sorted. For each streaming media object in at least two streaming media objects, the combination of the targeted streaming media objects has
  • the value of the determination field is the preset target field value, it means that the targeted streaming media object needs to form a sorting and binding combined object with at least one other streaming media object in the sorting relationship set, and the combined object will be taken as a whole Participate in sorting.
  • the value of the combination judgment field of the targeted streaming media object is not the value of the target field, it means that the targeted streaming media object does not need to be combined with other streaming media objects in the sorting relationship set to participate in the sorting, but to separate Participate in sorting.
  • the value of the target field can be configured according to the actual application scenario. For example, the value of the target field may be 1.
  • the combination judgment field it is possible to realize whether it is necessary to perform combination and sorting on the streaming media objects, and obtain the combination judgment result, and then by combining the combination judgment result and the sorting information field, at least two streams in the sorting relationship set can be The media objects are accurately sorted to obtain a sequence of streaming media objects.
  • the value of the combination determination field of any streaming media object in the sorting relationship set is the value of the target field, it indicates that a combination determination is required, and the data receiver will obtain the representation that needs to perform a combination determination
  • the value of the combination judgment field of at least two streaming media objects is not the target field value, it means that no combination judgment is required, and the data receiver will obtain the characterization and does not need to perform Combined judgment result of combined judgment.
  • the value of the combination determination field of the streaming media object when the value of the combination determination field of the streaming media object is not the value of the target field, its corresponding combination identification field can be omitted, that is, the combination Identification field.
  • the corresponding combination identification field When the value of the combination determination field of the streaming media object is the value of the target field, the corresponding combination identification field must be shown in the ordering relationship descriptor.
  • the data receiver when the result of the combination determination indicates that a combination determination is required, will combine the at least two streaming media objects based on the combination identification fields each of the at least two streaming media objects , to obtain a combination result, and then sort at least two streaming media objects based on the combination result and the sorting information field to obtain a sequence of streaming media objects.
  • At least two streaming media objects are combined based on the combination identification fields each of the at least two streaming media objects, and obtaining the combination result includes: combining the streaming media objects with the same combination identification field into Combining objects to obtain a combination result, where the combination result includes at least one combination object.
  • the data receiver may, according to the combined identification fields of the plurality of streaming media objects, Combination matching is performed, and streaming media objects with the same combination identification field can form a combination object with a sorted binding relationship.
  • the combined identification fields of streaming media object E and streaming media object F both take a value of 100
  • the combined identification fields of streaming media object G and streaming media object H both take a value of 101.
  • streaming media object E and streaming The media object F forms a combination object with a sort binding relationship
  • the streaming media object G and the streaming media object H can form another combination object with a sort binding relationship.
  • At least two streaming media objects can be combined based on the combination identification field to obtain a combination result, and then at least two streaming media objects can be accurately sorted through the combination result and the sorting information field to obtain a streaming media object. sequence of objects.
  • At least two streaming media objects are sorted based on the combination result and the sorting information field, and obtaining the streaming media object sequence includes:
  • At least one combined object is sorted based on the sorting information fields of at least one combined object to obtain a first object sequence, and the first object sequence is used as a streaming media object sequence ;
  • At least one uncombined streaming media object in at least two streaming media objects based on the sorting information field, at least one combined object and at least one uncombined streaming media object are sorted to obtain a second object sequence, and the second A sequence of objects as a sequence of streaming objects.
  • the data receiver when at least two streaming media objects are combined, it means that all elements in the sorting relationship set are combined objects composed of streaming media objects, and the data receiver will According to the values of the sorting information fields of the objects, at least one combined object is sorted to obtain a first sequence of objects arranged in order, and the first sequence of objects is used as a sequence of streaming media objects.
  • the elements in the sorting relationship set include both independent uncombined streaming media objects and combined objects composed of streaming media objects.
  • the receiver will sort at least one combined object and at least one uncombined streaming media object based on the values of the sorting information fields of at least one combined object and at least one uncombined streaming media object, and obtain the second object arranged in order Sequence, using the second object sequence as a streaming media object sequence.
  • the sorting in the case of sorting according to the values of the sorting information fields, can be performed according to the order of the values of the sorting information fields from small to large, or can be sorted according to the values of the sorting information fields from large to large. Sort in order of smallest.
  • the sorting information field may include one type of specified information among quality sorting information, priority sorting information, or undefined sorting information, or may include two or more types of specified information.
  • the data receiver may sort the at least one composite object according to the same type of specified information, or may comprehensively sort the at least one composite object according to multiple types of specified information.
  • the data receiver can sort at least one combined object and at least one uncombined streaming media object according to the specified information of the same type, or can synthesize at least one combined object and at least one uncombined streaming media object according to multiple types of specified information Sort.
  • the value of the sorting information field of the combined object is determined according to the value of the sorting information field of the streaming media objects that make up the combined object.
  • the manner of determining the value of the sorting information field of the combined object according to the value of the sorting information field of the streaming media objects constituting the combined object is not limited, as long as the determination can be realized.
  • the value of the sorting information field of any streaming media object constituting the combined object may be directly used as the value of the sorting information field of the combined object.
  • the minimum value among the values of the sorting information fields of the streaming media objects constituting the composite object may be used as the value of the sorting information field of the composite object.
  • the streaming media objects constituting the combined object may share the same sorting information field, and the same sorting information field is the sorting information field of the combined object.
  • At least two streaming media objects are sorted in different ways, so that accurate sorting of at least two streaming media objects can be achieved and a sequence of streaming media objects can be obtained.
  • the data receiver when the result of the combination determination indicates that no combination determination is required, the data receiver will perform the at least two streaming media objects based on the sorting information fields each of the at least two streaming media objects have. Sort to get the sequence of streaming media objects.
  • the combination judgment result when the combination judgment result is characterized as no need to make a combination judgment, it means that at least two streaming media objects in the sorting relationship set are independent streaming media objects and participate in the sorting independently, and the data The receiver directly sorts the at least two streaming media objects based on the values of the sorting information fields of the at least two streaming media objects to obtain a sequence of streaming media objects.
  • the sorting in the case of sorting according to the values of the sorting information fields, can be performed according to the order of the values of the sorting information fields from small to large, or can be sorted according to the values of the sorting information fields from large to large. Sort in order of smallest.
  • the sorting information field may include one type of specified information among quality sorting information, priority sorting information, or undefined sorting information, or may include two or more types of specified information.
  • the data receiver may sort the at least two streaming media objects according to the same type of specified information, or may comprehensively sort the at least two streaming media objects according to multiple types of specified information.
  • the sorting relationship set includes streaming media object A, streaming media object B, and streaming media object C
  • the sorting field information of streaming media object A has The value is 1
  • the value of the sorting field information of the streaming media object B is 2
  • the value of the sorting field information of the streaming media object C is 3
  • the sorting method is in order of the values of the sorting information field from large to small Sorting at least two streaming media objects in the sorting relationship set, the sequence of streaming media objects obtained by the data receiver is streaming media object C-streaming media object B-streaming media object A.
  • the at least two streaming media objects are sorted directly based on the sorting information fields of the at least two streaming media objects, so that fast and accurate sorting can be realized. , to get the sequence of streaming media objects.
  • the sorting relationship descriptor further includes a scope quantity field, which is used to indicate the quantity of the sorting scope to which the streaming media object belongs.
  • the scope quantity field can be defined as an element ranking_set_num, indicating the number of sorting ranges to which a streaming media object belongs.
  • the streaming media object includes at least one of a code stream file Representation, an adaptation subset Adaptation Set, or a preselection set Preselection.
  • the code stream file Representation is a file object including at least one streaming media resource
  • the adaptive subset Adaptation Set is a file object including at least one code stream file Representation
  • the preselection set Preselection is a code stream file Representation or an adaptive sub-set according to preset rules.
  • the embodiments of the present application can be applied to the transmission of streaming media objects in various dimensions, have strong universal applicability, can adapt to various versions of hardware devices, and can also be applied to different types of streaming media transmission protocols.
  • the streaming media resource is a point cloud media resource;
  • the streaming media object includes a geometric component and an attribute component, the geometric component is an adaptive subset used to represent the geometric components of the point cloud, and the attribute component is used to represent An adaptive subset of point cloud attribute components.
  • a point cloud is a set of discrete point sets randomly distributed in space that express the spatial structure and surface properties of a three-dimensional object or scene. Each point in the point cloud has at least three-dimensional position information, and may also have color, material or other information depending on the application scenario. Typically, each point in a point cloud has the same number of additional attributes.
  • Point cloud media can be divided into point cloud media (Video-based Point Cloud Compression, VPCC) compressed based on traditional video coding methods and point cloud media compressed based on geometric features (Geometry-based Point Cloud Compression, GPCC).
  • the three-dimensional position information is usually called the geometry component (Geometry Component) of the point cloud file
  • the attribute information is called the attribute component (Attribute Component) of the point cloud file.
  • a point cloud file has only one geometric component, but there can be one or more attribute components.
  • the encoded data stream After encoding the point cloud media resource, the encoded data stream needs to be encapsulated and transmitted to the user. Correspondingly, on the point cloud media player side, it is necessary to decapsulate the point cloud file first, then decode it, and finally present the decoded data stream.
  • the streaming media resource may be an immersive media resource, such as a multi-view video, a video encoding point cloud, and the like.
  • the streaming media object may include volumetric media with different degrees of freedom, for example, immersive media with a degree of freedom of 3DoF, 3DoF+ or 6DoF containing a volumetric video track.
  • various options can be provided for network transmission of streaming media objects, which can ensure the stability and reliability of streaming media data transmission, and improve the transmission efficiency of streaming media data.
  • Figure 5 shows a flow chart of the steps of the streaming media transmission method performed by the coding side in some embodiments of the present application
  • the coding side can be understood as the data sender sending streaming media data
  • the method can be applied to the server or can be applied to the An intermediate node for data communication between the server and the terminal device.
  • the streaming media transmission method may include the following steps S510 to S550.
  • step S510 the respective sorting ranges and respective sorting criteria of at least one streaming media object are obtained.
  • At least one streaming media object is classified to obtain the respective sorting ranges of at least one streaming media object, and the streaming media objects belonging to the same sorting range are sorted to obtain the respective sorting ranges of at least one streaming media object. corresponding sort by.
  • the sorting relationship between various streaming media objects includes two levels of sorting range and sorting position. Streaming media objects belonging to the same sorting range can be sorted according to the sorting basis, while streaming media objects belonging to different sorting ranges cannot. sort each other.
  • the data sender will obtain the corresponding quality evaluation results of at least one streaming media object, classify the at least one streaming media object according to the quality evaluation results and the pre-configured classification rules, and obtain the corresponding sorting range of the at least one streaming media object.
  • the quality evaluation result refers to the result obtained by performing quality evaluation on the streaming media object, and the quality of the streaming media object can be determined based on the quality evaluation result.
  • Preconfigured classification rules can be configured according to actual application scenarios. For example, the preconfigured classification rule may be to classify high-quality streaming media objects into one category, and classify low-quality streaming media objects into one category. It should be noted that the so-called high quality and low quality are relative concepts.
  • the quality of the streaming media object can be determined based on the quality evaluation result and a pre-configured quality evaluation threshold.
  • the quality evaluation result is specific It may be a quality evaluation score.
  • a streaming media object may belong to at least one sorting range.
  • the corresponding sorting basis of at least one streaming media object may be obtained based on a pre-configured sorting rule after obtaining the corresponding sorting range of at least one streaming media object, and the pre-configured sorting rule may be obtained according to Configured in actual application scenarios, the pre-configured sorting rules define the way to sort the streaming media objects within the sorting range. It should be noted that multiple streaming media objects belonging to the same sorting scope may have a single type of sorting basis, or may have multiple types of sorting criteria.
  • the flexibility and diversity of streaming media object sorting can be improved, so that It provides a variety of options for the network transmission of streaming media objects, which can ensure the stability and reliability of streaming media data transmission and improve the transmission efficiency of streaming media data.
  • step S520 respectively configure the sorting identifier fields of the sorting relationship descriptors of at least one streaming media object.
  • the streaming media object when two streaming media objects belong to the same sorting range, they can be configured with the same sorting identifier field; when two streaming media objects belong to two different sorting ranges , you can configure sorting identifier fields with different values for them respectively.
  • the streaming media object when a streaming media object belongs to two or more sorting ranges at the same time, the streaming media object can be configured with sorting identification field values corresponding to each sorting range.
  • streaming media object A and streaming media object B belong to one sorting range
  • streaming media object C and streaming media object D belong to another sorting range
  • the sorting identifier fields of streaming media object A and streaming media object B can be both It is configured as 1, and at the same time, both the sorting identification fields of streaming media object C and streaming media object D can be configured as 2.
  • the first sorting identification field corresponding to the first sorting range can be configured for streaming media object A
  • the stream media object A can be configured with the first sorting field corresponding to the second sorting range.
  • step S530 according to the sorting criteria, the sorting information fields of the sorting relationship descriptors of at least one streaming media object are respectively configured.
  • the data sender may configure the sorting information fields of the sorting relationship descriptors of at least one streaming media object directly according to the sorting basis.
  • the number of ranges of the corresponding sorting ranges of at least one streaming media object can be counted, and then the ranges of the sorting relationship descriptors can be sorted according to the number of ranges Quantity field to configure.
  • the range number field is used to indicate the number of ranking ranges to which the streaming media object belongs.
  • the range number field can be defined as an element ranking_set_num, which indicates the number of ranking ranges to which a streaming media object belongs.
  • the data receiver before sending the streaming media transmission signaling to the data receiver, it also includes:
  • the sorting binding determination result indicates that there are streaming media objects to be bound that have sorting binding relationships
  • the sorting binding relationships configure the combination identifier fields of the sorting relationship descriptors of the streaming media objects to be bound respectively.
  • the combination determination field of the ordering relationship descriptor of the streaming media objects to be bound according to the value of the target field.
  • the component type is used to represent the type of streaming media object.
  • the component type can be specifically geometric, and the streaming media object is a geometric component.
  • the component type can be specifically attribute type, then Streaming media objects are property components. Based on the component type, it is possible to determine which streaming media objects in at least one streaming media object can be bound. After determining the streaming media objects that can be bound, based on the pre-configured binding rules, the The stream media object to be bound with the sort binding relationship is determined among the objects, and the sort binding determination result is obtained.
  • only streaming media objects belonging to different component types can be bound, that is, only streaming media objects of different types can be bound.
  • geometry components can be bound to property components, but geometry components cannot be bound to each other, and property components cannot be bound to each other.
  • the pre-configured binding rules can be configured according to actual application scenarios, and the pre-configured binding rules are used to determine what quality of streaming media objects to bind.
  • the preconfigured binding rule may specifically bind streaming media objects of the same quality belonging to different component types.
  • high-quality geometry components can be bound to attribute components
  • low-quality geometry components can be bound to attribute components.
  • the preconfigured binding rule may specifically bind streaming media objects of different qualities belonging to different component types.
  • high-quality geometry components can be bound to low-quality property components, or low-quality geometry components can be bound to high-quality property components.
  • the same field value can be assigned to the combination identification field of the streaming media object to be bound, and at the same time, it can be set according to the target field value Configure the combination judgment field of the sequence relationship descriptor of the stream media object to be bound.
  • the combination identification field and the combination determination field it is convenient for the decoding side to efficiently determine whether the streaming media objects can be combined. If the streaming media objects with sorting binding relationship can be combined into composite objects, the composite objects can be sorted as a whole when sorting objects, so the calculation amount of object sorting can be reduced and the efficiency of object sorting can be improved.
  • the combined object can also be transmitted as a whole, so the transmission efficiency of the streaming media data can be further improved.
  • step S540 a streaming media transmission instruction is generated based on the ordering relationship descriptors of at least one streaming media object, and the streaming media transmission signaling is sent to the data receiver.
  • step S550 in response to the streaming media transmission request corresponding to the streaming media transmission signaling sent by the data receiver, return the target streaming media object matching the streaming media transmission request to the data receiver.
  • the data receiver can select an object that meets its own data requirements from the streaming media object sequence as the target streaming media object to be received.
  • the target streaming media object can be a separate streaming media object, or can be is all streaming objects in one composite object.
  • the streaming media object with higher quality ranking or higher priority ranking can be preferentially selected as the target streaming media object to be received; when the device performance of the data receiving party When the network status is poor or poor, a streaming media object with a lower quality ranking or a lower priority ranking may be selected as the target streaming media object to be received.
  • the data receiver can send a streaming media transmission request for the target streaming media object to the data sender.
  • the party transfer matches the target streaming object for the streaming transfer request.
  • the streaming media transmission method by dividing the streaming media objects into multiple different sorting ranges, and sorting the streaming media objects according to different sorting criteria within a specific sorting range, the streaming media objects can be improved.
  • the flexibility and diversity of sorting can provide a variety of options for network transmission of streaming media objects, and then can generate streaming media transmission instructions based on the sorting relationship descriptors of at least one streaming media object, and stream media transmission instructions
  • the data receiver can select the target streaming media to be received from the streaming media object sequence according to its own equipment performance and network conditions after analyzing the streaming media transmission signaling to obtain the streaming media object sequence Object, sending a streaming media transmission request for the target streaming media object to obtain the target streaming media object, can ensure the stability and reliability of streaming media data transmission, and improve the transmission efficiency of streaming media data.
  • the server organizes the media files in the form of video streams and generates corresponding signaling files according to the structure of the media files and the quality and priority information contained therein. According to the quality and priority information in the file, indicate the quality and priority information of the video stream within a specific range.
  • the server sends the signaling file to the client.
  • the client requests the appropriate video stream and decodes it for consumption according to its own needs, network conditions or decoding capabilities, combined with the quality and priority information in the signaling file.
  • an indication method of related sorting information such as video stream quality and priority is defined by taking the form of extending the existing DASH signaling as an example.
  • a SupplementalProperty element with a @schemeIdUri attribute value of "urn:avs:ims:2020:rk-relation” is used to represent a sorting relationship descriptor.
  • This descriptor is used to define quality sorting or priority sorting information between specific Representations and specific Adaptation Sets. This descriptor can describe media resources at the Representation, Adaptation Set, or Preselection level.
  • One or more ordering relationship descriptors may exist in one MPD signaling. The syntax and semantics of this descriptor are shown in Table 1 below.
  • the RankingRelationship@qualityRanking field in the above descriptor can be replaced by the @qualityRanking field.
  • the RankingRelationship@priorityRanking field in the above descriptor can be replaced by the @selectionPriority field.
  • the sorting relationship descriptor can also be extended on the basis of the existing fields of the streaming media transmission signaling, so as to achieve flexible quality of media resources such as Representation, Adaptation Set, and Preselection within a specific range. and prioritization purposes.
  • Table 2 shows the syntax and semantics of extending the quality sorting equivalent descriptor in some embodiments of the present application
  • Table 3 shows the syntax and semantics of extending the priority selection information descriptor in some embodiments of the present application Syntax and semantics
  • Table 4 shows the syntax and semantics of the extension of the replaceable information descriptor in some embodiments of the present application.
  • the ranking association descriptor can be defined as a ranking set descriptor Ranking_set_descriptor.
  • the ranking set descriptor Ranking_set_descriptor is used to indicate the information of the ranking set to which the current media resource belongs, and a media resource may belong to one or more ranking sets.
  • Table 5 shows the syntax information of the ranking set descriptor Ranking_set_descriptor.
  • descriptor_tag identifier, used to indicate the type of descriptor.
  • descriptor_length The length of the identifier, in bytes.
  • ranking_set_num Indicates the number of ranking sets the current media resource belongs to.
  • ranking_set_id Ranking set identifier, the media resources with the same ranking_set_id together form the range of quality ranking, priority ranking and undefined ranking. Media resources with different ranking_set_ids cannot be compared based on the corresponding quality_ranking, priority_ranking, and undefined_ranking.
  • combine_ranking_flag A value of 1 indicates that all resource sets with the same combine_ranking_id participate in quality ranking, priority ranking or undefined ranking as a whole.
  • combine_ranking_id All resource sets carrying the same combine_ranking_id participate in quality ranking, priority ranking or undefined ranking as a whole.
  • quality_ranking Specifies the quality ranking information of the corresponding resource or resource collection. The smaller the value of this field, the higher the quality level.
  • priority_ranking Specifies the priority ranking information of the corresponding resource or resource collection. The smaller the value of this field, the higher the priority.
  • undefined_ranking When different resources or resource sets are classified according to special rules (such as algorithm type) and cannot be sorted by conventional quality level or priority, this field specifies the classification method under special rules, and the value of this field is the same
  • the resource or resource collection of is the resource or resource collection of the same level under the classification method.
  • point cloud media resources as an example to describe the process of streaming media transmission of point cloud media resources in an application scenario according to the embodiment of the present application.
  • the server organizes the media files in the form of video streams and generates corresponding signaling files according to the structure of the media files and the quality information contained therein. Indicates the quality information of the video stream within a specific range according to the quality information in the file.
  • point cloud file F1 contains geometry component, attribute component-color.
  • the geometry component and attribute component have two different quality versions respectively, and the combination of different quality versions constitutes the overall quality information of point cloud consumption. Accordingly, the DASH signaling is generated as follows:
  • the server sends the signaling file to the client.
  • the client can know that for the point cloud file F1, the server contains two geometric components of different qualities (Representation1, Representation2) and two attribute components of different qualities (Representation3, Representation4). And Representation1+Representation3 and Representation2+Representation4 are combined to perform quality sorting.
  • the client requests an appropriate video stream and decodes it for consumption according to its own needs, network conditions or decoding capabilities, combined with the quality information in the signaling file.
  • the embodiment of the present application proposes a quality and priority information indication method for video streaming, especially immersive media (point cloud, multi-view video, panoramic video, etc.).
  • immersive media point cloud, multi-view video, panoramic video, etc.
  • FIG. 6 shows a structural block diagram of a streaming media transmission device on a decoding side provided by an embodiment of the present application.
  • the streaming media transmission device 600 on the decoding side may mainly include:
  • the signaling analysis module 610 is configured to analyze the streaming media transmission signaling to obtain the sorting relationship descriptors of each streaming media object.
  • the sorting relationship descriptor includes a sorting identification field and a sorting information field; the sorting identification field is used to indicate the sorting range ; The sorting information field is used as the sorting basis;
  • the object collection module 620 is configured to form at least two streaming media objects with the same sort identification field into a sort relationship set;
  • the object sorting module 630 is configured to sort at least two streaming media objects in the sorting relationship set according to the sorting information field to obtain a streaming media object sequence;
  • the request sending module 640 is configured to select a target streaming media object to be received from the streaming media object sequence, and send a streaming media transmission request for the target streaming media object, and the streaming media transmission request is used to request to acquire the target streaming media object.
  • the streaming media transmission device by dividing the streaming media objects into multiple different sorting ranges, and sorting the streaming media objects according to different sorting criteria within a specific sorting range, the streaming media objects can be improved.
  • the flexibility and diversity of sorting can provide a variety of options for the network transmission of streaming media objects, and then enable the data receiver to analyze the streaming media transmission signaling to obtain the sorting relationship descriptors of each streaming media object.
  • At least two streaming media objects with the same sorting identification field form a sorting relationship set, sort at least two streaming media objects in the sorting relationship set according to the sorting information field, and obtain the sequence of streaming media objects according to the performance of the device itself Select the target streaming media object to be received from the streaming media object sequence and send the streaming media transmission request for the target streaming media object to obtain the target streaming media object, which can ensure the stability and reliability of streaming media data transmission and improve Transmission efficiency of streaming media data.
  • the object sorting module is further configured to obtain a combined judgment result based on the combined judgment fields of at least two streaming media objects in the sorted relationship set, and to sort the sorted relationship based on the combined judgment result and the sorted information field. At least two streaming media objects in the collection are sorted to obtain a sequence of streaming media objects.
  • the sorting relationship descriptor further includes a combination identification field used to indicate the scope of the combined sorting; the object sorting module is also configured to, when the combination determination result indicates that a combination determination is required, based on at least two Combine the at least two streaming media objects to obtain a combination result based on the combination identification field of each streaming media object, and sort the at least two streaming media objects based on the combination result and the sorting information field to obtain a sequence of streaming media objects.
  • the object sorting module is also used to combine streaming media objects with the same combination identification field into a combined object to obtain a combined result
  • the combined result includes at least one combined object
  • at least two streaming media objects are In the case of being combined, based on the sorting information fields of at least one combined object, at least one combined object is sorted to obtain a first object sequence, and the first object sequence is used as a streaming media object sequence, and at least two streaming media objects
  • at least one uncombined streaming media object based on the sorting information field, at least one combined object and at least one uncombined streaming media object are sorted to obtain a second object sequence, and the second object sequence is used as a streaming media object sequence .
  • the object sorting module is further configured to sort at least two stream media Objects are sorted to obtain a sequence of streaming media objects.
  • Fig. 7 shows a structural block diagram of a streaming media transmission device on the encoding side in some embodiments of the present application.
  • the streaming media transmission device 700 on the encoding side may mainly include:
  • the obtaining module 710 is configured to obtain the respective sorting ranges and respective sorting basis of at least one streaming media object
  • the range configuration module 720 is configured to configure the sorting identification fields of the sorting relationship descriptors of at least one streaming media object respectively according to the sorting range;
  • the configuration module 730 it is configured to configure the sorting information fields of the sorting relationship descriptors of the at least one streaming media object respectively according to the sorting basis;
  • the signaling sending module 740 is configured to generate a streaming media transmission instruction based on the ordering relationship descriptors of at least one streaming media object, and send the streaming media transmission signaling to the data receiver;
  • the object returning module 750 is configured to, in response to the streaming media transmission request sent by the data receiver and corresponding to the streaming media transmission signaling, return the target streaming media object matching the streaming media transmission request to the data receiver.
  • the streaming media transmission device by dividing the streaming media objects into multiple different sorting ranges, and sorting the streaming media objects according to different sorting criteria within a specific sorting range, the streaming media objects can be improved.
  • the flexibility and diversity of sorting can provide a variety of options for network transmission of streaming media objects, and then can generate streaming media transmission instructions based on the sorting relationship descriptors of at least one streaming media object, and stream media transmission instructions
  • the data receiver can select the target streaming media to be received from the streaming media object sequence according to its own equipment performance and network conditions after analyzing the streaming media transmission signaling to obtain the streaming media object sequence Object, sending a streaming media transmission request for the target streaming media object to obtain the target streaming media object, can ensure the stability and reliability of streaming media data transmission, and improve the transmission efficiency of streaming media data.
  • the obtaining module 710 may include:
  • the classification sub-module 711 is configured to classify at least one streaming media object, and obtain the respective sorting ranges of at least one streaming media object;
  • the obtaining sub-module 712 is configured to sort the streaming media objects belonging to the same sorting range, and obtain the respective sorting basis of at least one streaming media object.
  • the streaming media transmission device 700 on the encoding side also includes:
  • the quantity counting module 760 is configured to count the number of ranges of the respective sorting ranges of at least one streaming media object
  • the quantity configuration module 770 is configured to respectively configure the scope quantity field of the ordering relationship descriptor of at least one streaming media object according to the scope quantity.
  • the streaming media transmission device 700 on the encoding side also includes:
  • the binding comparison module 780 is configured to obtain the corresponding component types of at least one streaming media object, and based on the component type, perform sorting binding determination on at least one streaming media object, and obtain the sorting binding determination result;
  • the binding configuration module 790 is configured to, in the case that the sorting binding determination result indicates that there are streaming media objects to be bound with a sorting binding relationship, according to the sorting binding relationship, the respective sorting of the streaming media objects to be bound Configure the combination identification field of the relationship descriptor, and configure the combination judgment field of the ordering relationship descriptor that the streaming media objects to be bound have respectively according to the value of the target field.
  • Fig. 8 schematically shows a structural block diagram of a computer system for implementing an electronic device according to an embodiment of the present application
  • the electronic device includes one or more processors and a memory
  • a computer program is stored in the memory
  • the processor executes the computer program
  • the computer system 900 includes a central processing unit 901 (Central Processing Unit, CPU), which can be stored in a program in a read-only memory 802 (Read-Only Memory, ROM) or loaded from a storage part 808 to a random Various appropriate actions and processes are executed by accessing programs in the memory 803 (Random Access Memory, RAM). In the random access memory 803, various programs and data necessary for system operation are also stored.
  • the CPU 801 , the read only memory 802 and the random access memory 803 are connected to each other through a bus 804 .
  • An input/output interface 805 (Input/Output interface, ie, an I/O interface) is also connected to the bus 804 .
  • the following components are connected to the input/output interface 805: an input part 806 including a keyboard, a mouse, etc.; an output part 807 including a cathode ray tube (Cathode Ray Tube, CRT), a liquid crystal display (Liquid Crystal Display, LCD), etc., and a speaker ; a storage section 808 including a hard disk or the like; and a communication section 809 including a network interface card such as a LAN card, a modem, or the like. The communication section 809 performs communication processing via a network such as the Internet.
  • a driver 810 is also connected to the input/output interface 805 as needed.
  • a removable medium 811 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is mounted on the drive 810 as necessary so that a computer program read therefrom is installed into the storage section 808 as necessary.
  • the processes described in the respective method flowcharts can be implemented as computer software programs.
  • the embodiments of the present application include a computer program product, which includes computer-readable instructions carried on a computer-readable storage medium, where the computer-readable instructions include program codes for executing a streaming media transmission method.
  • the computer readable instructions may be downloaded and installed from a network via communications portion 809 and/or installed from removable media 811 .
  • the central processing unit 801 When the computer readable instructions are executed by the central processing unit 801, various functions defined in the system of the present application are performed.
  • the present application also provides one or more computer storage media, and at least one computer-readable instruction is stored in the computer storage medium, and the computer-readable instruction includes a program instruction, when the processor executes the above-mentioned program instruction , the streaming media transmission method in the above-mentioned corresponding embodiments can be implemented, so details will not be repeated here.
  • a computer program product or computer readable instructions comprising computer readable instruction code, the computer readable instruction code being stored in a computer readable storage medium .
  • the processor of the computer device reads the computer-readable instruction code from the computer-readable storage medium, and the processor executes the computer-readable instruction code, so that the computer device can execute the streaming media transmission method in the above-mentioned corresponding embodiments, therefore, No further details will be given here.
  • any reference to storage, database or other media used in the various embodiments provided in the present application may include at least one of non-volatile and volatile storage.
  • Non-volatile memory can include read-only memory (Read-Only Memory, ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive variable memory (ReRAM), magnetic variable memory (Magnetoresistive Random Access Memory, MRAM), Ferroelectric Random Access Memory (FRAM), Phase Change Memory (Phase Change Memory, PCM), graphene memory, etc.
  • the volatile memory may include random access memory (Random Access Memory, RAM) or external cache memory, etc.
  • RAM Random Access Memory
  • RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

本申请属于音视频技术领域，具体涉及一种流媒体传输方法、装置、电子设备、计算机可读存储介质以及计算机程序产品，该方法包括：解析流媒体传输信令，得到各流媒体对象各自具有的排序关系描述符，排序关系描述符包括排序标识字段以及排序信息字段；排序标识字段用于表示排序范围；排序信息字段用于作为排序依据（410）；将具有相同排序标识字段的至少两个流媒体对象组成排序关系集合（420）；根据排序信息字段，对排序关系集合中至少两个流媒体对象进行排序，得到流媒体对象序列（430）；从流媒体对象序列中选取待接收的目标流媒体对象，发送针对目标流媒体对象的流媒体传输请求，流媒体传输请求用于请求获取目标流媒体对象（440）。

Description

流媒体传输方法、装置、电子设备及计算机可读存储介质
相关申请
本申请要求于2022年01月10日申请的，申请号为202210022663.8，名称为“流媒体传输方法及相关产品”的中国专利申请的优先权，在此将其全文引入作为参考。
技术领域
本申请属于音视频技术领域，具体涉及一种流媒体传输方法、装置、电子设备、计算机可读存储介质以及计算机程序产品。
背景技术
流媒体(streaming media)是指将一连串的媒体数据压缩后，经过网上分段发送数据，在网上即时传输影音以供观赏的一种技术与过程，此技术使得数据包得以像流水一样发送。流媒体实时分段传输的特点决定了用户在无需下载完整媒体文件的情况下便可以实时获得媒体数据，但同时也对用户的设备性能和网络条件提出了很高的要求。一旦出现设备性能下降或者网络状态不理想等情况，便很容易出现媒体数据传输受阻的问题，因此流媒体传输普遍存在传输效率低、稳定性差等问题。
发明内容
本申请的目的在于提供一种流媒体传输方法、装置、电子设备、计算机可读存储介质以及计算机程序产品。
根据本申请实施例的一个方面，提供一种流媒体传输方法，由电子设备执行，该方法包括：
解析流媒体传输信令，得到各流媒体对象各自具有的排序关系描述符，排序关系描述符包括排序标识字段以及排序信息字段；排序标识字段用于表示排序范围；排序信息字段用于作为排序依据；
将具有相同排序标识字段的至少两个流媒体对象组成排序关系集合；
根据排序信息字段，对排序关系集合中至少两个流媒体对象进行排序，得到流媒体对象序列；
从流媒体对象序列中选取待接收的目标流媒体对象，发送针对目标流媒体对象的流媒体传输请求，流媒体传输请求用于请求获取目标流媒体对象。
根据本申请实施例的一个方面，提供一种流媒体传输方法，由电子设备执行，该方法包括：
获取至少一个流媒体对象各自相应的排序范围以及各自相应的排序依据；
根据排序范围，分别对至少一个流媒体对象各自具有的排序关系描述符的排序标识字段进行配置；
根据排序依据，分别对至少一个流媒体对象各自具有的排序关系描述符的排序信息字段进行配置；
基于至少一个流媒体对象各自具有的排序关系描述符，生成流媒体传输指令，将流媒体传输信令发送至数据接收方；
响应于数据接收方发送的对应于所述流媒体传输信令的流媒体传输请求，向所述数据接收方返回与所述流媒体传输请求相匹配的目标流媒体对象。
根据本申请实施例的一个方面，提供一种流媒体传输装置，该装置包括：
信令解析模块，被配置为解析流媒体传输信令，得到各流媒体对象各自具有的排序关系描述符，排序关系描述符包括排序标识字段以排序信息字段；排序标识字段用于表示排序范围；排序信息字段用于作为排序依据；
对象集合模块，被配置为将具有相同排序标识字段的至少两个流媒体对象组成排序关系集合；
对象排序模块，被配置为根据排序信息字段，对排序关系集合中至少两个流媒体对象进行排序，得到流媒体对象序列；
请求发送模块，被配置为从流媒体对象序列中选取待接收的目标流媒体对象，发送针对目标流媒体对象的流媒体传输请求，流媒体传输请求用于请求获取目标流媒体对象。
根据本申请实施例的一个方面，提供一种流媒体传输装置，该装置包括：
获取模块，被配置为获取至少一个流媒体对象各自相应的排序范围以及各自相应的排序依据；
范围配置模块，被配置为根据排序范围，分别对至少一个流媒体对象各自具有的排序关系描述符的排序标识字段进行配置；
依据配置模块，被配置为根据排序依据，分别对至少一个流媒体对象各自具有的排序关系描述符的排序信息字段进行配置；
信令发送模块，被配置为基于至少一个流媒体对象各自具有的排序关系描述符，生成流媒体传输指令，将流媒体传输信令发送至数据接收方；
对象返回模块，被配置为响应于数据接收方发送的对应于所述流媒体传输信令的流媒体传输请求，向所述数据接收方返回与所述流媒体传输请求相匹配的目标流媒体对象。
根据本申请实施例的一个方面，提供一种计算机可读存储介质，该计算机可读存储介质中存储有至少一条计算机可读指令，该至少一条计算机可读指令由一个或多个处理器加载并执行以实现如本申请实施例中流媒体传输方法中所执行的操作。
根据本申请实施例的一个方面，提供一种电子设备，该电子设备包括：一个或多个处理器以及存储器，该存储器用于存储至少一条计算机可读指令，该至少一段计算机可读指令由该一个或多个处理器加载并执行以实现本申请实施例中的流媒体传输方法中所执行的操作。
根据本申请实施例的一个方面，提供一种计算机程序产品，该计算机程序产品包括计算机可读指令，该计算机可读指令存储在计算机可读存储介质中。计算机设备的一个或多个处理器从计算机可读存储介质读取该计算机可读指令，处理器执行该计算机可读指令，使得该计算机设备执行如以上技术方案中的流媒体传输方法。
附图说明
此处的附图被并入说明书中并构成本说明书的一部分，示出了符合本申请的实施例，并与说明书一起用于解释本申请的原理。显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1示出了可以应用本申请实施例的技术方案的示例性系统架构的示意图。
图2示出视频编码装置和视频解码装置在流式传输环境中的放置方式。
图3示出了一个视频编码器的基本流程图。
图4示出了本申请一些实施例中由解码侧执行的流媒体传输方法的步骤流程图。
图5示出了本申请一些实施例中由编码侧执行的流媒体传输方法的步骤流程图。
图6示出了本申请实施例提供的解码侧的流媒体传输装置的结构框图。
图7示出了本申请一些实施例中的编码侧的流媒体传输装置的结构框图。
图8示出了适于用来实现本申请实施例的电子设备的计算机系统结构框图。
具体实施方式
现在将参考附图更全面地描述示例实施方式。然而，示例实施方式能够以多种形式实施，且不应被理解为限于在此阐述的范例；相反，提供这些实施方式使得本申请将更加全面和完整，并将示例实施方式的构思全面地传达给本领域的技术人员。
此外，所描述的特征、结构或特性可以以任何合适的方式结合在一个或更多实施例中。在下面的描述中，提供许多具体细节从而给出对本申请的实施例的充分理解。然而，本领域技术人员将意识到，可以实践本申请的技术方案而没有特定细节中的一个或更多，或者可以采用其它的方法、组元、装置、步骤等。在其它情况下，不详细示出或描述公知方法、装置、实现或者操作以避免模糊本申请的各方面。
附图中所示的方框图仅仅是功能实体，不一定必须与物理上独立的实体相对应。即，可以采用软件形式来实现这些功能实体，或在一个或多个硬件模块或集成电路中实现这些功能实体，或在不同网络和/或处理器装置和/或微控制器装置中实现这些功能实体。
附图中所示的流程图仅是示例性说明，不是必须包括所有的内容和操作/步骤，也不是必须按所描述的顺序执行。例如，有的操作/步骤还可以分解，而有的操作/步骤可以合并或部分合并，因此实际执行的顺序有可能根据实际情况改变。
需要说明的是：在本文中提及的“多个”是指两个或两个以上。“和/或”描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。字符“/”一般表示前后关联对象是一种“或”的关系。
在本申请的具体实施方式中，涉及到流媒体资源等与用户相关的数据，当本申请的各个实施例运用到具体产品或技术中时，需要获得用户许可或者同意，且相关数据的收集、使用和处理需要遵守相关国家和地区的相关法律法规和标准。
本申请实施例中涉及的相关术语或者缩略语解释如下。
DASH：dynamic adaptive streaming over HTTP(基于HTTP(Hyper Text Transfer Protocol，超文本传输协议)的动态自适应流)，基于HTTP的动态自适应流是一种自适应比特率流技术，使高质量流媒体可以通过传统的HTTP网络服务器以互联网传递。
MPD：media presentation description，DASH中的媒体演示描述信令，用于描述媒体片段信息。
Representation：DASH中，一个或多个媒体成分的组合，比如某种分辨率的视频文件可以看做一个Representation。
Adaptation Sets：DASH中，一个或多个视频流的集合，一个Adaptation Sets中可以包含多个Representation。
多视角/多视点视频：指采用多组摄像机阵列，从多个角度拍摄的带有深度信息的视频。多视角/多视点视频也叫自由视角/自由视点视频，是一种提供六自由度体验的沉浸式媒体。
点云：点云是空间中一组无规则分布的、表达三维物体或场景的空间结构及表面属性的离散点集。点云中的每个点至少具有三维位置信息，根据应用场景的不同，还可能具有色彩、材质或其他信息。通常，点云中的每个点都具有相同数量的附加属性。
V3C容积媒体：visual volumetric video-based coding media，指捕获自三维空间视觉内容并提供3DoF+、6DoF观看体验的，以传统视频编码的，在文件封装中包含容积视频类型轨道的沉浸式媒体，包括多视角视频、视频编码点云等。
DoF：Degree of Freedom，自由度。本申请中是指用户在观看沉浸式媒体时支持的运动并产生内容交互的自由度。
3DoF：即三自由度，指用户头部围绕x,y,z轴旋转的三种自由度。
3DoF+：即在三自由度的基础上，用户还拥有沿x,y,z轴有限运动的自由度。
6DoF：即在三自由度的基础上，用户还拥有沿x,y,z轴自由运动的自由度。
图1示出了可以应用本申请实施例的技术方案的示例性系统架构的示意图。
如图1所示，系统架构100包括多个终端装置，所述终端装置可通过例如网络150彼此通信。举例来说，系统架构100可以包括通过网络150互连的第一终端装置110和第二终端装置120。在图1的实施例中，第一终端装置110和第二终端装置120执行单向数据传输。
举例来说，第一终端装置110可对视频数据(例如由第一终端装置110采集的视频图片流)进行编码以通过网络150传输到第二终端装置120，已编码的视频数据以一个或多个已编码视频码流形式传输，第二终端装置120可从网络150接收已编码视频数据，对已编码视频数据进行解码以恢复视频数据，并根据恢复的视频数据显示视频图片。
在本申请的一些实施例中，系统架构100可以包括执行已编码视频数据的双向传输的第三终端装置130和第四终端装置140，所述双向传输比如可以发生在视频会议期间。对 于双向数据传输，第三终端装置130可对视频数据(例如由第三终端装置采集的视频图片流)进行编码，以通过网络150传输到第四终端装置140。第三终端装置130还可以接收由第四终端装置140传输的已编码视频数据，对已编码视频数据进行解码以恢复视频数据，根据恢复的视频数据在可访问的显示装置上显示视频图片。
在图1的实施例中，第一终端装置110、第二终端装置120、第三终端装置130和第四终端装置140可为服务器、个人计算机和智能电话，但本申请公开的原理可不限于此。本申请公开的实施例适用于膝上型计算机、平板电脑、媒体播放器和/或专用视频会议设备。网络150表示在第一终端装置110、第二终端装置120、第三终端装置130和第四终端装置140之间传送已编码视频数据的任何数目的网络，包括例如有线和/或无线通信网络。通信网络150可在电路交换和/或分组交换信道中交换数据。该网络可包括电信网络、局域网、广域网和/或互联网。出于本申请的目的，除非在下文中有所解释，否则网络150的架构和拓扑对于本申请公开的操作来说可能是无关紧要的。
本申请实施例中的服务器可以是独立的物理服务器，也可以是多个物理服务器构成的服务器集群或者分布式系统，还可以是提供云计算服务的云服务器。终端可以是智能手机、平板电脑、笔记本电脑、台式计算机、智能音箱、智能手表、车载终端、智能电视等，但并不局限于此。终端以及服务器可以通过有线或无线通信方式进行直接或间接地连接，本申请在此不做限制。
在本申请的一些实施例中，图2示出视频编码装置和视频解码装置在流式传输环境中的放置方式。本申请所公开主题可同等地适用于其它支持视频的应用，包括例如视频会议、数字TV(television，电视机)、在包括CD、DVD、存储棒等的数字介质上存储压缩视频等等。
流式传输系统包括采集子系统213，采集子系统213包括数码相机等视频源201，视频源201创建未压缩的视频图片流202。在实施例中，视频图片流202包括由数码相机拍摄的样本。相较于已编码的视频数据204(或已编码的视频码流204)，视频图片流202被描绘为粗线以强调高数据量的视频图片流，视频图片流202可由电子装置220处理，电子装置220包括耦接到视频源201的视频编码装置203。视频编码装置203包括硬件、软件或软硬件组合以实现或实施如下文更详细地描述的所公开主题的各方面。相较于视频图片流202，已编码的视频数据204(或已编码的视频码流204)被描绘为细线以强调较低数据量的已编码的视频数据204(或已编码的视频码流204)，其可存储在流式传输服务器205上以供将来使用。一个或多个流式传输客户端子系统，例如图2中的客户端子系统206和客户端子系统208，可访问流式传输服务器205以检索已编码的视频数据204的副本207和副本209。客户端子系统206包括电子装置230，电子装置230包括视频解码装置210。视频解码装置210对已编码的视频数据的传入副本207进行解码，产生可在显示器212(例如显示屏)或另一呈现装置上呈现的输出视频图片流211。在一些流式传输系统中，可根据视频编码/压缩标准对已编码的视频数据204、视频数据207和视频数据209(例如视频码流)进行编码。其中，可以利用的标准的实施例包括ITU-TH.265等。
应注意，电子装置220和电子装置230可包括图中未示出的其它组件。举例来说，电子装置220可包括视频解码装置，电子装置230还可包括视频编码装置。
在本申请的一些实施例中，以国际视频编码标准HEVC(High Efficiency Video Coding，高效率视频编码)、VVC(Versatile Video Coding，多功能视频编码)，以及中国国家视频编码标准AVS为例，当输入一个视频帧图像之后，会根据一个块大小，将视频帧图像划分成若干个不重叠的处理单元，每个处理单元将进行类似的压缩操作。这个处理单元被称作CTU(Coding Tree Unit，编码树单元)，或者称之为LCU(Largest Coding Unit，最大编码单元)。CTU再往下可以继续进行更加精细的划分，得到一个或多个基本的编码单元CU，CU是一个编码环节中最基本的元素。以下介绍对CU进行编码时的一些概念：
预测编码(Predictive Coding)：预测编码包括了帧内预测和帧间预测等方式，原始视 频信号经过选定的已重建视频信号的预测后，得到残差视频信号。编码端需要为当前CU决定选择哪一种预测编码模式，并告知解码端。其中，帧内预测是指预测的信号来自于同一图像内已经编码重建过的区域；帧间预测是指预测的信号来自已经编码过的、不同于当前图像的其它图像(称之为参考图像)。
变换及量化(Transform&Quantization)：残差视频信号经过DFT(Discrete Fourier Transform，离散傅里叶变换)、DCT(Discrete Cosine Transform，离散余弦变换)等变换操作后，将信号转换到变换域中，称之为变换系数。变换系数进一步进行有损的量化操作，丢失掉一定的信息，使得量化后的信号有利于压缩表达。在一些视频编码标准中，可能有多于一种变换方式可以选择，因此编码端也需要为当前CU选择其中的一种变换方式，并告知解码端。
熵编码(Entropy Coding)或统计编码：量化后的变换域信号将根据各个值出现的频率进行统计压缩编码，最后输出二值化(0或者1)的压缩码流。同时，编码产生其他信息，例如选择的编码模式、运动矢量数据等，也需要进行熵编码以降低码率。统计编码是一种无损的编码方式，可以有效的降低表达同样信号所需要的码率，常见的统计编码方式有变长编码(Variable Length Coding，简称VLC)或者基于上下文的二值化算术编码(Content Adaptive Binary Arithmetic Coding，简称CABAC)。
环路滤波(Loop Filtering)：经过变化及量化的信号会通过反量化、反变换及预测补偿的操作获得重建图像。重建图像与原始图像相比由于存在量化的影响，部分信息与原始图像有所不同，即重建图像会产生失真(Distortion)。因此，可以对重建图像进行滤波操作，例如去块效应滤波(Deblocking filter，简称DB)、SAO(Sample Adaptive Offset，自适应像素补偿)或者ALF(Adaptive Loop Filter，自适应环路滤波)等滤波器，可以有效降低量化所产生的失真程度。由于这些经过滤波后的重建图像将作为后续编码图像的参考来对将来的图像信号进行预测，因此上述的滤波操作也被称为环路滤波，即在编码环路内的滤波操作。
在本申请的一些实施例中，图3示出了一个视频编码器的基本流程图，在该流程中以帧内预测为例进行说明。首先，会将原始图像信号与预测图像信号做差值运算，得到残差信号，再对残差信号进行变换以及量化处理得到量化系数，通过熵编码得到量化系数相应的编码后比特流，通过反量化及反变换处理得到量化系数相应的重构残差信号，叠加预测图像信号和重构残差信号生成图像信号，将图像信号输入至帧内模式决策模块和帧内预测模块进行帧内预测处理，获得帧内预测结果，并对图像信号进行环路滤波，获得重建图像信号，该重建图像信号可以作为下一帧的参考图像进行运动估计以及运动补偿预测，基于该重建图像信号进行运动补偿预测，获得运动补偿预测的结果，基于运动补偿预测的结果和帧内预测结果得到下一帧的预测图像信号，并继续重复上述过程，直至编码完成。
基于上述的编码过程，在解码端针对每一个编码单元，在获取到所针对的编码单元的压缩码流(即比特流)之后，会先对压缩码流进行熵解码获得各种模式信息及量化系数，然后对量化系数进行反量化及反变换处理得到残差信号，同时，根据已知的编码模式信息，获得所针对的编码单元对应的预测信号，最后将残差信号与预测信号相加即可得到重建信号，重建信号再经过环路滤波等操作，即可产生最终的输出信号。
在视频传输技术中，通常采用流化传输技术来处理服务器和客户端之间的媒体资源传输。常见的媒体流化传输技术包括DASH(Dynamic Adaptive Streaming over HTTP)，HLS(HTTP Live Streaming，基于HTTP的流媒体网络传输协议)，SMT(Smart Media Transport，智能媒体传输)等技术。
以DASH为例，DASH是一种自适应比特率流技术，使高质量流媒体可以通过传统的HTTP网络服务器以互联网传递。DASH会将内容分解成一系列小型的基于HTTP的文件片段，每个片段包含很短长度的可播放内容，而内容总长度可能长达数小时(例如电影或体育赛事直播)。内容将被制成多种比特率的备选片段，以提供多种比特率的版本供选用。 当媒体内容被DASH客户端播放时，客户端将根据当前网络条件自动选择下载和播放哪一个备选方案。客户端将选择可及时下载的最高比特率片段进行播放，从而避免播放卡顿或重新缓冲事件。也因如此，DASH客户端可以无缝适应不断变化的网络条件并提供高质量的播放体验，拥有更少的卡顿与重新缓冲发生率。
DASH使用现有的HTTP网络服务器基础设施。它允许如互联网电视、电视机顶盒、台式电脑、智能手机、平板电脑等设备消费通过互联网传送的多媒体内容(如视频、电视、广播等)，并可应对变动的互联网接收条件。
新兴的媒体资源播放场景对于流媒体数据的网络传输提出了更高的要求，本申请实施例提供了一种针对流媒体数据的质量、优先级等信息进行指示的方法。通过本申请实施例提出的方法，可以灵活定义和指示流媒体数据在特定范围内的质量、优先级等相关信息，从而使客户端能够根据自身需求请求相应的流媒体数据。
可读存储介质本申请实施例的各项技术方案可以应用于流媒体传输系统的服务器端、播放器端以及中间节点等环节。
图4示出了本申请一些实施例中由解码侧执行的流媒体传输方法的步骤流程图，解码侧可以理解为接收流媒体数据的数据接收方，该方法可以应用于用户的终端设备或者可以应用于在服务器与终端设备之间进行数据通信的中间节点。
如图4所示，该流媒体传输方法可以包括如下的步骤S410至步骤S440。
在步骤S410中，解析流媒体传输信令，得到各流媒体对象各自具有的排序关系描述符，排序关系描述符包括排序标识字段以及排序信息字段，排序标识字段用于表示排序范围，排序信息字段用于作为排序依据。
流媒体传输信令是在数据发送方和数据接收方之间进行传输的用于协调通信过程的消息，例如，流媒体传输信令可以是基于DASH协议的DASH信令，还可以是SMT信令。
排序关系描述符是用于定义不同的流媒体对象之间的质量排序信息、优先级排序或者其他维度的排序信息，排序关系描述符在不用类型的信令中可以用不同元素定义。例如，在DASH信令中可以将排序关系描述符定义为元素RankingRelationship，在SMT信令中可以将排序关系描述符定义为元素Ranking_set_descriptor。
本申请实施例中，数据接收方通过解析流媒体传输信令，可以得到各流媒体对象各自具有的排序关系描述符，该排序关系描述符包括排序标识字段以及排序信息字段，基于排序标识字段可以将各流媒体对象划分到至少一个排序范围，针对于至少一个排序范围中每一个排序范围，可以基于排序信息字段在所针对的排序范围内对流媒体对象进行排序。
在步骤S420中，将具有相同排序标识字段的至少两个流媒体对象组成排序关系集合。
在流媒体传输信令中携带有多个流媒体对象的排序关系描述符，数据接收方通过比较排序关系描述符可以确定各个流媒体对象之间的排序关系，该排序关系包括了排序范围和排序位置两个层面，其中排序范围是根据排序关系描述符中的排序标识字段来确定的。当任意两个流媒体对象具有相同的排序标识字段时，表示该两个流媒体对象归属于相同的排序范围，可以将相同的排序范围内的流媒体对象组成排序关系集合。例如，流媒体对象A和流媒体对象B的排序标识字段均取值为1，流媒体对象C和流媒体对象D的排序标识字段均取值为2，则可以将流媒体对象A和流媒体对象B组成一个排序关系集合，同时可以将流媒体对象C和流媒体对象D组成另一个排序关系集合。
在本申请的一些实施例中，一个流媒体对象可以具有一个排序关系描述符，由此，该流媒体对象可以具有一个排序标识字段，基于该流媒体对象的排序标识字段可以将该流媒体对象组合至与排序标识字段相应的排序关系集合中。
在本申请的一些实施例中，一个流媒体对象也可以同时具有多个不同的排序关系描述符，每个排序关系描述符所包括的排序标识字段的取值可以相同，也可以不同。在每个排序关系描述符所包含的排序标识字段的取值不完全相同的情况下，该流媒体对象具有不同取值的至少两个排序标识字段，据此，可以将该流媒体对象组合至分别与不同取值的至少 两个排序标识字段各自相应的排序关系集合中。组合对象
举例说明，流媒体对象A可以同时具有两个排序关系描述符(假设为排序关系描述符1和排序关系描述2)，且排序关系描述符1所包括的排序标识字段的取值为1，排序关系描述符2所包括的排序标识字段的取值为2，则可以将流媒体对象A组合至与排序标识字段的取值为1相应的排序关系集合1中，并将流媒体对象A组合至与排序标识字段的取值为2相应的排序关系集合2中。
在步骤S430中，根据排序信息字段，对排序关系集合中至少两个流媒体对象进行排序，得到流媒体对象序列。
在同一个排序关系集合中分布有至少两个流媒体对象，数据接收方根据排序信息字段的取值可以对排序关系集合中所分布的至少两个流媒体对象进行排序，得到按照指定顺序排列的流媒体对象序列。在本申请的一些实施例中，数据接收方可以按照排序信息字段取值从小到大的顺序对排序关系集合中至少两个流媒体对象进行排序，或者也可以按照排序信息字段取值从大到小的顺序对排序关系集合中至少两个流媒体对象进行排序。
在本申请的一些实施例中，排序信息字段可以包括质量排序信息、优先级排序信息或者未定义排序信息中的至少一种，质量排序信息用于表示按照流媒体质量对流媒体对象进行排序的排序等级，优先级排序信息用于表示按照选取优先级对流媒体对象进行排序的排序等级，未定义排序信息用于表示按照指定规则对流媒体对象进行排序的排序等级。其中，指定规则可以按照实际应用场景进行配置。
本实施例中，通过提供多种排序信息字段供选择，能够提高流媒体对象排序的灵活多样性，从而可以为流媒体对象的网络传输提供多样化的选择方案，可以保证流媒体数据传输的稳定性和可靠性，提高流媒体数据的传输效率。
在本申请的一些实施例中，排序信息字段可以包括质量排序信息、优先级排序信息或者未定义排序信息中的一种指定信息，也可以包括两种或者两种以上的指定信息。
在本申请的一些实施例中，数据接收方可以按照同一类型的指定信息对排序关系集合中至少两个流媒体对象进行排序，也可以按照多种类型的指定信息对排序关系集合中至少两个流媒体对象进行综合排序。
在步骤S440中，从流媒体对象序列中选取待接收的目标流媒体对象，发送针对目标流媒体对象的流媒体传输请求，流媒体传输请求用于请求获取所述目标流媒体对象。
在本申请的一些实施例中，数据接收方可以从流媒体对象序列中选取符合自身数据需求的对象作为待接收的目标流媒体对象，该目标流媒体对象可以是一个单独的流媒体对象，也可以是一个组合对象中所有的流媒体对象。例如，当数据接收方的设备性能较好且网络状态良好时，可以优先选择质量排序较高或者优先级排序较高的流媒体对象作为待接收的目标流媒体对象，当数据接收方的设备性能较差或者网络状态较差时，可以选择质量排序较低又或者优先级排序较低的流媒体对象作为待接收的目标流媒体对象。
在选定目标流媒体对象后，数据接收方可以向数据发送方发送针对目标流媒体对象的流媒体传输请求，当数据发送方接收到流媒体传输请求后，可以根据数据接收方的需求向数据接收方传输对应的目标流媒体对象。
在本申请实施例提供的流媒体传输方法中，通过将流媒体对象划分成多个不同的排序范围，并且在特定的排序范围内按照不同的排序依据对流媒体对象进行排序，可以提高流媒体对象排序的灵活多样性，从而可以为流媒体对象的网络传输提供多样化的选择方案，进而可以使得数据接收方在解析流媒体传输信令，得到各流媒体对象各自具有的排序关系描述符，将具有相同排序标识字段的至少两个流媒体对象组成排序关系集合，根据排序信息字段，对排序关系集合中至少两个流媒体对象进行排序，得到流媒体对象序列的情况下，依据自身的设备性能和网络情况从流媒体对象序列中选取待接收的目标流媒体对象，发送针对目标流媒体对象的流媒体传输请求以获取目标流媒体对象，可以保证流媒体数据传输的稳定性和可靠性，提高流媒体数据的传输效率。
在本申请的一些实施例中，排序关系描述符还包括用于表示是否对流媒体对象进行组合排序的组合判定字段，根据排序信息字段，对排序关系集合中至少两个流媒体对象进行排序，得到流媒体对象序列包括：基于排序关系集合中至少两个流媒体对象各自具有的组合判定字段，获得组合判定结果，基于组合判定结果和排序信息字段，对排序关系集合中至少两个流媒体对象进行排序，得到流媒体对象序列。
在本申请的一些实施例中，组合判定字段可以用于判断是否需要对流媒体对象进行组合排序，针对于至少两个流媒体对象中每一个流媒体对象，在所针对的流媒体对象具有的组合判定字段的取值为预设的目标字段值的情况下，表示所针对的流媒体对象需要与排序关系集合中至少一个其他流媒体对象组成排序绑定的组合对象，该组合对象将作为一个整体参与排序。在所针对的流媒体对象具有的组合判定字段的取值不为目标字段值的情况下，表示所针对的流媒体对象不需要与排序关系集合中其他流媒体对象组合参与排序，而是单独地参与到排序中。其中，目标字段值可按照实际应用场景进行配置。举例说明，目标字段值可以为1。
本实施例中，能够基于组合判定字段，实现对是否需要对流媒体对象进行组合排序进行判定，获得组合判定结果，进而可以通过结合组合判定结果和排序信息字段，对排序关系集合中至少两个流媒体对象进行准确排序，得到流媒体对象序列。
在本申请的一些实施例中，在排序关系集合中任一流媒体对象具有的组合判定字段的取值为目标字段值的情况下，表示需要进行组合判定，数据接收方会获得表征需要进行组合判定的组合判定结果，在排序关系集合中至少两个流媒体对象具有的组合判定字段的取值均不为目标字段值的情况下，表示不需要进行组合判定，数据接收方会获得表征不需要进行组合判定的组合判定结果。
在本申请的一些实施例中，在流媒体对象具有的组合判定字段的取值不是目标字段值的情况下，可以将其对应的组合标识字段省去，即不在排序关系描述符中示出组合标识字段。在流媒体对象具有的组合判定字段的取值是目标字段值的情况下，则必须在排序关系描述符中示出相应的组合标识字段。
在本申请的一些实施例中，在组合判定结果表征为需要进行组合判定的情况下，数据接收方会基于至少两个流媒体对象各自具有的组合标识字段，对至少两个流媒体对象进行组合，获得组合结果，再基于组合结果和排序信息字段，对至少两个流媒体对象进行排序，获得流媒体对象序列。
在本申请的一些实施例中，基于至少两个流媒体对象各自具有的组合标识字段，对至少两个流媒体对象进行组合，获得组合结果包括：将具有相同组合标识字段的流媒体对象组合为组合对象，获得组合结果，组合结果包括至少一个组合对象。
在本申请的一些实施例中，针对组合判定字段的取值为目标字段值的多个流媒体对象，数据接收方可以根据多个流媒体对象各自具有的组合标识字段，对多个流媒体对象进行组合匹配，具有相同组合标识字段的流媒体对象可以组成具有排序绑定关系的组合对象。例如，流媒体对象E和流媒体对象F的组合标识字段均取值为100，流媒体对象G和流媒体对象H的组合标识字段均取值为101，此时可以将流媒体对象E和流媒体对象F组成一个具有排序绑定关系的组合对象，同时可以将流媒体对象G和流媒体对象H组成另一个具有排序绑定关系的组合对象。
本实施例中，能够基于组合标识字段，实现对至少两个流媒体对象进行组合，获得组合结果，进而可以通过组合结果和排序信息字段，对至少两个流媒体对象进行准确排序，得到流媒体对象序列。
在本申请的一些实施例中，基于组合结果和排序信息字段，对至少两个流媒体对象进行排序，获得流媒体对象序列包括：
在至少两个流媒体对象均被组合的情况下，基于至少一个组合对象各自具有的排序信息字段，对至少一个组合对象进行排序，获得第一对象序列，将第一对象序列作为流媒体 对象序列；
在至少两个流媒体对象中存在至少一个未组合流媒体对象的情况下，基于排序信息字段，对至少一个组合对象和至少一个未组合流媒体对象进行排序，获得第二对象序列，将第二对象序列作为流媒体对象序列。
在本申请的一些实施例中，在至少两个流媒体对象均被组合的情况下，表示排序关系集合中的所有元素均为由流媒体对象组成的组合对象，数据接收方会根据至少一个组合对象各自具有的排序信息字段的取值，对至少一个组合对象进行排序，获得按照顺序排列的第一对象序列，将第一对象序列作为流媒体对象序列。在至少两个流媒体对象中存在至少一个未组合流媒体对象的情况下，表示排序关系集合中的元素既包括相互独立的未组合流媒体对象，也包括由流媒体对象组成的组合对象，数据接收方会基于至少一个组合对象以及至少一个未组合流媒体对象各自具有的排序信息字段的取值，对至少一个组合对象和至少一个未组合流媒体对象进行排序，获得按照顺序排列的第二对象序列，将第二对象序列作为流媒体对象序列。
在本申请的一些实施例中，在根据排序信息字段的取值进行排序的情况下，可以按照排序信息字段的取值从小到大的顺序进行排序，也可以按照排序信息字段的取值从大到小的顺序进行排序。
在本申请的一些实施例中，排序信息字段可以包括质量排序信息、优先级排序信息或者未定义排序信息中的一种指定信息，也可以包括两种或者两种以上的指定信息。数据接收方可以按照同一类型的指定信息对至少一个组合对象进行排序，也可以按照多种类型的指定信息对至少一个组合对象进行综合排序。数据接收方可以按照同一类型的指定信息对至少一个组合对象以及至少一个未组合流媒体对象进行排序，也可以按照多种类型的指定信息对至少一个组合对象以及至少一个未组合流媒体对象进行综合排序。
在本申请的一些实施例中，组合对象具有的排序信息字段的取值是根据组成组合对象的流媒体对象具有的排序信息字段的取值确定的。本实施例中，不限定根据组成组合对象的流媒体对象具有的排序信息字段的取值确定组合对象具有的排序信息字段的取值的方式，只要能够实现确定即可。比如，可以直接将组成组合对象的任一流媒体对象具有的排序信息字段的取值，作为组合对象具有的排序信息字段的取值。又比如，可以将组成组合对象的流媒体对象具有的排序信息字段的取值中的最小值，作为组合对象具有的排序信息字段的取值。再比如，组成组合对象的流媒体对象可以共享一个相同的排序信息字段，该相同的排序信息字段即为组合对象具有的排序信息字段。
本实施例中，针对于流媒体对象被组合的不同情况，分别采用不同的方式对至少两个流媒体对象进行排序，能够实现对至少两个流媒体对象的准确排序，获得流媒体对象序列。
在本申请的一些实施例中，在组合判定结果表征为不需要进行组合判定的情况下，数据接收方会基于至少两个流媒体对象各自具有的排序信息字段，对至少两个流媒体对象进行排序，得到流媒体对象序列。
在本申请的一些实施例中，在组合判定结果表征为不需要进行组合判定的情况下，表示排序关系集合中至少两个流媒体对象均为相互独立的流媒体对象且单独地参与排序，数据接收方会直接基于至少两个流媒体对象各自具有的排序信息字段的取值，对至少两个流媒体对象进行排序，得到流媒体对象序列。
在本申请的一些实施例中，在根据排序信息字段的取值进行排序的情况下，可以按照排序信息字段的取值从小到大的顺序进行排序，也可以按照排序信息字段的取值从大到小的顺序进行排序。
在本申请的一些实施例中，排序信息字段可以包括质量排序信息、优先级排序信息或者未定义排序信息中的一种指定信息，也可以包括两种或者两种以上的指定信息。数据接收方可以按照同一类型的指定信息对至少两个流媒体对象进行排序，也可以按照多种类型的指定信息对至少两个流媒体对象进行综合排序。
举例说明，在组合判定结果表征为不需要进行组合判定的情况下，假设排序关系集合中包括流媒体对象A、流媒体对象B以及流媒体对象C，且流媒体对象A具有的排序字段信息的取值为1，流媒体对象B具有的排序字段信息的取值为2，流媒体对象C具有的排序字段信息的取值为3，排序方式为按照排序信息字段取值从大到小的顺序对排序关系集合中至少两个流媒体对象进行排序，则数据接收方得到的流媒体对象序列为流媒体对象C-流媒体对象B-流媒体对象A。本实施例中，在组合判定结果表征为不需要进行组合判定的情况下，直接基于至少两个流媒体对象各自具有的排序信息字段，对至少两个流媒体对象进行排序，能够实现快速准确排序，得到流媒体对象序列。
在本申请的一些实施例中，排序关系描述符还包括范围数量字段，范围数量字段用于表示流媒体对象所属的排序范围的数量，例如在SMT信令中，可以将范围数量字段定义为元素ranking_set_num，表示一个流媒体对象所属的排序范围的数量。通过在排序关系描述符中定义出范围数量字段，可以方便解码侧对流媒体对象所属的排序范围进行遍历，避免出现排序错漏，也避免进行全量检测，因此可以进一步提高流媒体对象的排序可靠性和排序效率。
在本申请的一些实施例中，流媒体对象包括码流文件Representation、自适应子集Adaptation Set或者预选集Preselection中的至少一种。码流文件Representation是包括至少一个流媒体资源的文件对象，自适应子集Adaptation Set是包括至少一个码流文件Representation的文件对象，预选集Preselection是按照预设规则对码流文件Representation或者自适应子集Adaptation Set进行组合得到的文件对象。本申请实施例可以适用于各种不同维度的流媒体对象传输，具有极强的普遍适用性，能够适应各种不同版本的硬件设备，也可以适用于不同类型的流媒体传输协议。
在本申请的一些实施例中，流媒体资源为点云媒体资源；流媒体对象包括几何组件和属性组件，几何组件是用于表示点云几何成分的自适应子集，属性组件是用于表示点云属性成分的自适应子集。
点云是空间中一组无规则分布的、表达三维物体或场景的空间结构及表面属性的离散点集。点云中的每个点至少具有三维位置信息，根据应用场景的不同，还可能具有色彩、材质或其他信息。通常，点云中的每个点都具有相同数量的附加属性。点云媒体从编码方式上又可以分为基于传统视频编码方式进行压缩的点云媒体(Video-based Point Cloud Compression，VPCC)以及基于几何特征进行压缩的点云媒体(Geometry-based Point Cloud Compression，GPCC)。在点云媒体的文件封装中，三维位置信息通常称为点云文件的几何分量(Geometry Component)，属性信息称为点云文件的属性分量(Attribute Component)。一个点云文件仅有一个几何分量，但可以存在一个或多个属性分量。
在对点云媒体资源进行编码后，需要对编码后的数据流进行封装并传输给用户。相对应的，在点云媒体播放器端，需要先对点云文件进行解封装，然后再进行解码，最后将解码后的数据流呈现。
在本申请的一些实施例中，流媒体资源可以是沉浸式媒体资源，例如多视角视频、视频编码点云等。相应地流媒体对象可以包括具有不同自由度的容积媒体，例如自由度为3DoF、3DoF+或者6DoF的包含容积视频类型轨道的沉浸式媒体。
本实施例中，通过提供多种组件，可以为流媒体对象的网络传输提供多样化的选择方案，可以保证流媒体数据传输的稳定性和可靠性，提高流媒体数据的传输效率。
图5示出了本申请一些实施例中由编码侧执行的流媒体传输方法的步骤流程图，编码侧可以理解为发送流媒体数据的数据发送方，该方法可以应用于服务器或者可以应用于在服务器与终端设备之间进行数据通信的中间节点。如图5所示，该流媒体传输方法可以包括如下的步骤S510至步骤S550。
在步骤S510中，获取至少一个流媒体对象各自相应的排序范围以及各自相应的排序依据。
在本申请的一些实施例中，对至少一个流媒体对象进行分类，获得至少一个流媒体对象各自相应的排序范围，对归属于相同排序范围的流媒体对象进行排序，获得至少一个流媒体对象各自相应的排序依据。
各个流媒体对象之间的排序关系包括了排序范围和排序位置两个层面，归属于相同排序范围内的流媒体对象可以按照排序依据进行对象排序，归属于不同排序范围内的流媒体对象则不能进行相互排序。数据发送方会获取至少一个流媒体对象各自相应的质量评价结果，根据质量评价结果和预配置的分类规则，对至少一个流媒体对象进行分类，获得至少一个流媒体对象各自相应的排序范围。
质量评价结果是指对流媒体对象进行质量评价所得到的结果，基于质量评价结果可以确定流媒体对象的质量。预配置的分类规则可按照实际应用场景进行配置。比如，预配置的分类规则可以为将高质量流媒体对象划分为一类，并将低质量流媒体对象划分为一类。需要说明的是，所谓的高质量和低质量是一个相对的概念，在一些实施例中，流媒体对象的质量的高低可以基于质量评价结果和预配置的质量评价阈值来确定，质量评价结果具体可以为质量评价分数，在质量评价分数大于质量评价阈值的情况下，流媒体对象的质量为高质量，在质量评价分数小于或者等于质量评价阈值的情况下，流媒体对象的质量为低质量。在本申请的一些实施例中，一个流媒体对象可以归属于至少一个排序范围。
在本申请的一些实施例中，至少一个流媒体对象各自相应的排序依据可以在获取至少一个流媒体对象各自相应的排序范围之后，基于预配置的排序规则获取，该预配置的排序规则可按照实际应用场景进行配置，该预配置的排序规则定义了对排序范围内的流媒体对象进行排序的方式。需要说明的是，归属于同一个排序范围的多个流媒体对象可以具有单一类型的排序依据，或者也可以具有多种类型的排序依据。
本实施例中，通过将流媒体对象划分成多个不同的排序范围，并且在特定的排序范围内按照不同的排序依据对流媒体对象进行排序，可以提高流媒体对象排序的灵活多样性，从而可以为流媒体对象的网络传输提供多样化的选择方案，可以保证流媒体数据传输的稳定性和可靠性，提高流媒体数据的传输效率。
在步骤S520中，根据排序范围，分别对至少一个流媒体对象各自具有的排序关系描述符的排序标识字段进行配置。
在本申请的一些实施例中，当两个流媒体对象归属于相同的排序范围时，可以为其配置取值相同的排序标识字段，当两个流媒体对象归属于两个不同的排序范围时，则可以分别为其配置取值不同的排序标识字段。当一个流媒体对象同时归属于两个或者两个以上的排序范围时，可以为该流媒体对象配置分别与每个排序范围相应的排序标识字段值。例如，流媒体对象A和流媒体对象B归属于一个排序范围，流媒体对象C和流媒体对象D归属于另一个排序范围，则可以将流媒体对象A和流媒体对象B的排序标识字段均配置为1，同时可以将流媒体对象C和流媒体对象D的排序标识字段均配置为2。又例如，流媒体对象A同时归属于第一排序范围和第二排序范围，则可以为流媒体对象A配置与第一排序范围相应的第一排序标识字段，并为流媒体对象A配置与第二排序范围相应的第二排序标识字段。
在步骤S530中，根据排序依据，分别对至少一个流媒体对象各自具有的排序关系描述符的排序信息字段进行配置。
在本申请的一个实施例中，数据发送方在获取到排序依据后，可以直接根据排序依据，分别对至少一个流媒体对象各自具有的排序关系描述符的排序信息字段进行配置。
在本申请的一些实施例中，在将流媒体传输信令发送至数据接收方之前，可以统计至少一个流媒体对象各自相应的排序范围的范围数量，然后根据范围数量对排序关系描述符的范围数量字段进行配置。范围数量字段用于表示流媒体对象所属的排序范围的数量，例如在SMT信令中，可以将范围数量字段定义为元素ranking_set_num，表示一个流媒体对象所属的排序范围的数量。通过在排序关系描述符中定义出范围数量字段，可以方便解码 侧对流媒体对象所属的排序范围进行遍历，避免出现排序错漏，也避免进行全量检测，因此可以进一步提高流媒体对象的排序可靠性和排序效率。
在本申请的一些实施例中，在将流媒体传输信令发送至数据接收方之前，还包括：
获取至少一个流媒体对象各自相应的组件类型，基于组件类型，对至少一个流媒体对象进行排序绑定判定，获得排序绑定判定结果；
在排序绑定判定结果表征为存在具有排序绑定关系的待绑定流媒体对象的情况下，根据排序绑定关系，对待绑定流媒体对象各自具有的排序关系描述符的组合标识字段进行配置，并按照目标字段值对待绑定流媒体对象各自具有的排序关系描述符的组合判定字段进行配置。
在本申请的一些实施例中，组件类型用于表示流媒体对象的类型，比如，组件类型具体可以是几何型，则流媒体对象为几何组件，又比如，组件类型具体可以是属性型，则流媒体对象为属性组件。基于组件类型，可以确定至少一个流媒体对象中哪些流媒体对象可以进行绑定，在确定可以进行绑定的流媒体对象后，可以基于预配置的绑定规则，从可以进行绑定的流媒体对象中确定出具有排序绑定关系的待绑定流媒体对象，获得排序绑定判定结果。
在本申请的一些实施例中，只有归属于不同组件类型的流媒体对象可以进行绑定，即只有不同类型的流媒体对象可以进行绑定。比如，几何组件可以与属性组件进行绑定，但是几何组件之间不能进行绑定，属性组件之间与不能进行绑定。预配置的绑定规则可按照实际应用场景进行配置，该预配置的绑定规则用于确定将何种质量的流媒体对象进行绑定。比如，该预配置的绑定规则具体可以为将归属于不同组件类型的相同质量的流媒体对象进行绑定。举例说明，可以将高质量的几何组件和属性组件进行绑定，并将低质量的几何组件和属性组件进行绑定。又比如，该预配置的绑定规则具体可以为将归属于不同组件类型的不同质量的流媒体对象进行绑定。举例说明，可以将高质量的几何组件和低质量的属性组件进行绑定，也可以将低质量的几何组件和高质量的属性组件进行绑定。
在本申请的一些实施例中，针对于具有排序绑定关系的待绑定流媒体对象，可以为该待绑定流媒体对象具有的组合标识字段赋予相同的字段值，同时可以按照目标字段值对该待绑定流媒体对象具有的排序关系描述符的组合判定字段进行配置。通过对组合标识字段和组合判定字段进行配置，可以方便解码侧高效地判定是否能够对流媒体对象进行组合。如果可以将具有排序绑定关系的流媒体对象组成组合对象，则可以在进行对象排序时，以组合对象作为一个整体进行排序，因此可以将降低对象排序的计算量，提高对象排序的效率。在针对目标流媒体对象进行数据传输时，也可以将组合对象作为一个整体进行传输，因此可以进一步提高流媒体数据的传输效率。
在步骤S540中，基于至少一个流媒体对象各自具有的排序关系描述符，生成流媒体传输指令，将流媒体传输信令发送至数据接收方。
在步骤S550中，响应于数据接收方发送的对应于流媒体传输信令的流媒体传输请求，向数据接收方返回与流媒体传输请求相匹配的目标流媒体对象。
在本申请的一些实施例中，数据接收方可以从流媒体对象序列中选取符合自身数据需求的对象作为待接收的目标流媒体对象，目标流媒体对象可以是一个单独的流媒体对象，也可以是一个组合对象中的所有流媒体对象。例如，当数据接收方的设备性能较好且网络状态良好时，可以优先选择质量排序较高或者优先级排序较高的流媒体对象作为待接收的目标流媒体对象；当数据接收方的设备性能较差或者网络状态较差时，可以选择质量排序较低又或者优先级排序较低的流媒体对象作为待接收的目标流媒体对象。
在选定目标流媒体对象后，数据接收方可以向数据发送方发送针对目标流媒体对象的流媒体传输请求，数据发送方接收到流媒体传输请求后，可以根据数据接收方的需求向数据接收方传输与流媒体传输请求相匹配的目标流媒体对象。
在本申请实施例提供的流媒体传输方法中，通过将流媒体对象划分成多个不同的排序 范围，并且在特定的排序范围内按照不同的排序依据对流媒体对象进行排序，可以提高流媒体对象排序的灵活多样性，从而可以为流媒体对象的网络传输提供多样化的选择方案，进而可以在基于至少一个流媒体对象各自具有的排序关系描述符，生成流媒体传输指令，将流媒体传输指令发送至数据接收方的基础上，使得数据接收方在解析流媒体传输信令得到流媒体对象序列的情况下，依据自身的设备性能和网络情况从流媒体对象序列中选取待接收的目标流媒体对象，发送针对目标流媒体对象的流媒体传输请求以获取目标流媒体对象，可以保证流媒体数据传输的稳定性和可靠性，提高流媒体数据的传输效率。
在本申请实施例的一个应用场景中，服务器端根据媒体文件的结构以及包含的质量和优先级信息，将媒体文件以视频流的形式进行组织并生成相应的信令文件。根据文件中的质量和优先级信息，指示视频流在特定范围内的质量和优先级信息。服务器将信令文件发送给客户端。客户端根据自身的需求、网络条件或解码能力，结合信令文件中的质量和优先级信息，请求合适的视频流并解码消费。
本申请实施例可以在系统层添加若干描述性字段，包括信令消息层面的字段扩展，以支持本申请实施例的实施步骤。下文中，以扩展现有DASH信令的形式举例，定义了视频流的质量、优先级等相关排序信息的指示方法。
在本申请的一些实施例中，用一个@schemeIdUri属性值为"urn:avs:ims:2020:rk-relation"的SupplementalProperty元素表示一个排序关系描述符。该描述符用于定义特定Representation之间、特定Adaptation Set之间的质量排序或优先级排序信息。该描述符可描述Representation、Adaptation Set或者Preselection级别的媒体资源。一个MPD信令中可存在一个或多个排序关系描述符。该描述符的语法和语义如下方表1所示。
表1排序关系描述符的语法及语义
Figure PCTCN2022137754-appb-000001
Figure PCTCN2022137754-appb-000002
在本申请的一些实施例中，上述描述符中的RankingRelationship@qualityRanking字段可以使用@qualityRanking字段替代。上述描述符中的RankingRelationship@priorityRanking字段可以使用@selectionPriority字段替代。
在本申请的一些实施例中，排序关系描述符也可以在流媒体传输信令的已有字段基础上进行字段扩展，达到在特定范围内对Representation、Adaptation Set、Preselection等媒体资源进行灵活的质量和优先级排序的目的。例如，下方的表2示出了本申请一些实施例中对质量排序等价描述符进行扩展的语法及语义，表3示出了本申请一些实施例中对优先级选择信息描述符进行扩展的语法及语义，表4示出了本申请一些实施例中对可替换信息描述符进行扩展的语法及语义。
表2质量排序等价描述符扩展的语法及语义
Figure PCTCN2022137754-appb-000003
表3优先级选择信息描述符扩展的语法及语义
Figure PCTCN2022137754-appb-000004
Figure PCTCN2022137754-appb-000005
表4可替换信息描述符扩展的语法及语义
Figure PCTCN2022137754-appb-000006
以下以SMT信令扩展作为示例对本申请实施例中的排序关联描述符的信令扩展进行说明。在SMT信令中，排序关联描述符可以定义为排序集合描述符Ranking_set_descriptor。排序集合描述符Ranking_set_descriptor用于指示当前媒体资源所属的排序集合的信息，一个媒体资源可以属于一个或多个排序集合。表5示出了排序集合描述符Ranking_set_descriptor的语法信息。
表5排序集合描述符的语法信息
Figure PCTCN2022137754-appb-000007
其中各个元素定义如下。
descriptor_tag：标识符，用于表示descriptor的类型。
descriptor_length：标识符的长度，单位为字节。
ranking_set_num：指示当前媒体资源所属的排序集合的数量。
ranking_set_id：排序集合标识符，包含相同ranking_set_id的媒体资源共同构成了质量排序、优先级排序、未定义排序的范围。包含不同ranking_set_id的媒体资源之间，不能依据相应的quality_ranking、priority_ranking、undefined_ranking进行比较。
combine_ranking_flag：该字段取值为1表示携带相同combine_ranking_id的所有资源集合作为整体参与质量排序、优先级排序或未定义排序。
combine_ranking_id：携带相同combine_ranking_id的所有资源集合作为整体参与质量排序、优先级排序或未定义排序。
quality_ranking：指定相应资源或资源集合的质量排序信息，该字段取值越小代表质量等级越高。
priority_ranking：指定相应资源或资源集合的优先级排序信息，该字段取值越小代表优先级越高。
undefined_ranking：当不同资源或资源集合之间按照特殊的规则进行分类(比如算法类型)，无法以常规的质量等级或优先级进行排序时，该字段指定特殊规则下的分类方式，该字段取值相同的资源或资源集合为该分类方式下同等级别的资源或资源集合。
下面以点云媒体资源作为示例，对本申请实施例在一个应用场景中对点云媒体资源进行流媒体传输的过程进行说明。
首先，服务器端根据媒体文件的结构以及包含的质量信息，将媒体文件以视频流的形式进行组织并生成相应的信令文件。根据文件中的质量信息，指示视频流在特定范围内的质量信息。假设点云文件F1包含几何组件、属性组件-颜色。几何组件、属性组件分别具备2个不同的质量版本，且不同的质量版本组合起来构成了点云消费的整体质量信息。据此生成DASH信令如下：
【Adaptation Set1(几何组件)：
{Representation1：
{RankingRelationship@id＝1；
RankingRelationship@combineRankingFlag＝1；
RankingRelationship@combineRankingId＝100；
RankingRelationship@qualityRanking＝0}
{RankingRelationship@id＝1；
RankingRelationship@combineRankingFlag＝1；
RankingRelationship@combineRankingId＝101；
RankingRelationship@qualityRanking＝1}
}
{Representation2：
{RankingRelationship@id＝1；
RankingRelationship@combineRankingFlag＝1；
RankingRelationship@combineRankingId＝102；
RankingRelationship@qualityRanking＝2}
{RankingRelationship@id＝1；
RankingRelationship@combineRankingFlag＝1；
RankingRelationship@combineRankingId＝103；
RankingRelationship@qualityRanking＝3}
}】
【Adaptation Set2(属性组件-颜色)：
{Representation3：{RankingRelationship@id＝1；
RankingRelationship@combineRankingFlag＝1；
RankingRelationship@combineRankingId＝100}
{RankingRelationship@id＝1；
RankingRelationship@combineRankingFlag＝1；
RankingRelationship@combineRankingId＝101}}
{Representation4：
{RankingRelationship@id＝1；
RankingRelationship@combineRankingFlag＝1；
RankingRelationship@combineRankingId＝102}
{RankingRelationship@id＝1；
RankingRelationship@combineRankingFlag＝1；
RankingRelationship@combineRankingId＝103}
}】
其次，服务器将信令文件发送给客户端。
客户端结合信令消息，即可知道对于点云文件F1，服务器上包含2个不同质量的几何组件(Representation1、Representation2)和2个不同质量的属性组件(Representation3、Representation4)。且Representation1+Representation3和Representation2+Representation4两种媒体流组合在一起进行质量排序。
最后，客户端根据自身的需求、网络条件或解码能力，结合信令文件中的质量信息，请求合适的视频流并解码消费。
为了满足应用需求，本申请实施例针对视频流媒体，特别是沉浸式媒体(点云、多视角视频、全景视频等)，提出了一种针对视频流的质量与优先级信息指示方法。通过本申请实施例提出的方法，可以灵活定义和指示视频流在特定范围内的质量和优先级信息，从而使客户端能够根据自身需求请求相应的视频流。
应当注意，尽管在附图中以特定顺序描述了本申请中方法的各个步骤，但是，这并非要求或者暗示必须按照该特定顺序来执行这些步骤，或是必须执行全部所示的步骤才能实现期望的结果。附加的或备选的，可以省略某些步骤，将多个步骤合并为一个步骤执行，以及/或者将一个步骤分解为多个步骤执行等。
以下介绍本申请的装置实施例，可以用于执行本申请上述实施例中的流媒体传输方法。
图6示出了本申请实施例提供的解码侧的流媒体传输装置的结构框图。如图6所示，解码侧的流媒体传输装置600主要可以包括：
信令解析模块610，被配置为解析流媒体传输信令，得到各流媒体对象各自具有的排序关系描述符，排序关系描述符包括排序标识字段以及排序信息字段；排序标识字段用于表示排序范围；排序信息字段用于作为排序依据；
对象集合模块620，被配置为将具有相同排序标识字段的至少两个流媒体对象组成排序关系集合；
对象排序模块630，被配置为根据排序信息字段，对排序关系集合中至少两个流媒体对象进行排序，得到流媒体对象序列；
请求发送模块640，被配置为从流媒体对象序列中选取待接收的目标流媒体对象，发送针对目标流媒体对象的流媒体传输请求，流媒体传输请求用于请求获取目标流媒体对象。
在本申请实施例提供的流媒体传输装置中，通过将流媒体对象划分成多个不同的排序范围，并且在特定的排序范围内按照不同的排序依据对流媒体对象进行排序，可以提高流媒体对象排序的灵活多样性，从而可以为流媒体对象的网络传输提供多样化的选择方案，进而可以使得数据接收方在解析流媒体传输信令，得到各流媒体对象各自具有的排序关系 描述符，将具有相同排序标识字段的至少两个流媒体对象组成排序关系集合，根据排序信息字段，对排序关系集合中至少两个流媒体对象进行排序，得到流媒体对象序列的情况下，依据自身的设备性能和网络情况从流媒体对象序列中选取待接收的目标流媒体对象，发送针对目标流媒体对象的流媒体传输请求以获取目标流媒体对象，可以保证流媒体数据传输的稳定性和可靠性，提高流媒体数据的传输效率。
在本申请的一些实施例中，对象排序模块还用于基于排序关系集合中至少两个流媒体对象各自具有的组合判定字段，获得组合判定结果，基于组合判定结果和排序信息字段，对排序关系集合中至少两个流媒体对象进行排序，得到流媒体对象序列。
在本申请的一些实施例中，排序关系描述符还包括用于表示组合排序范围的组合标识字段；对象排序模块还用于在组合判定结果表征为需要进行组合判定的情况下，基于至少两个流媒体对象各自具有的组合标识字段，对至少两个流媒体对象进行组合，获得组合结果，基于组合结果和所述排序信息字段，对至少两个流媒体对象进行排序，获得流媒体对象序列。
在本申请的一些实施例中，对象排序模块还用于将具有相同组合标识字段的流媒体对象组合为组合对象，获得组合结果，组合结果包括至少一个组合对象，在至少两个流媒体对象均被组合的情况下，基于至少一个组合对象各自具有的排序信息字段，对至少一个组合对象进行排序，获得第一对象序列，将第一对象序列作为流媒体对象序列，在至少两个流媒体对象中存在至少一个未组合流媒体对象的情况下，基于排序信息字段，对至少一个组合对象和至少一个未组合流媒体对象进行排序，获得第二对象序列，将第二对象序列作为流媒体对象序列。
在本申请的一些实施例中，对象排序模块还用于在组合判定结果表征为不需要进行组合判定的情况下，基于至少两个流媒体对象各自具有的排序信息字段，对至少两个流媒体对象进行排序，得到流媒体对象序列。
图7示出了本申请一些实施例中的编码侧的流媒体传输装置的结构框图。如图7所示，编码侧的流媒体传输装置700主要可以包括：
获取模块710，被配置为获取至少一个流媒体对象各自相应的排序范围以及各自相应的排序依据；
范围配置模块720，被配置为根据排序范围，分别对至少一个流媒体对象各自具有的排序关系描述符的排序标识字段进行配置；
依据配置模块730，被配置为根据排序依据，分别对至少一个流媒体对象各自具有的排序关系描述符的排序信息字段进行配置；
信令发送模块740，被配置为基于至少一个流媒体对象各自具有的排序关系描述符，生成流媒体传输指令，将流媒体传输信令发送至数据接收方；
对象返回模块750，被配置为响应于数据接收方发送的对应于流媒体传输信令的流媒体传输请求，向数据接收方返回与流媒体传输请求相匹配的目标流媒体对象。
在本申请实施例提供的流媒体传输装置中，通过将流媒体对象划分成多个不同的排序范围，并且在特定的排序范围内按照不同的排序依据对流媒体对象进行排序，可以提高流媒体对象排序的灵活多样性，从而可以为流媒体对象的网络传输提供多样化的选择方案，进而可以在基于至少一个流媒体对象各自具有的排序关系描述符，生成流媒体传输指令，将流媒体传输指令发送至数据接收方的基础上，使得数据接收方在解析流媒体传输信令得到流媒体对象序列的情况下，依据自身的设备性能和网络情况从流媒体对象序列中选取待接收的目标流媒体对象，发送针对目标流媒体对象的流媒体传输请求以获取目标流媒体对象，可以保证流媒体数据传输的稳定性和可靠性，提高流媒体数据的传输效率。
在本申请的一些实施例中，获取模块710可以包括：
分类子模块711，被配置为对至少一个流媒体对象进行分类，获得至少一个流媒体对象各自相应的排序范围；
获取子模块712，被配置为对归属于相同排序范围的流媒体对象进行排序，获得至少一个流媒体对象各自相应的排序依据。
在本申请的一些实施例中，编码侧的流媒体传输装置700还包括：
数量统计模块760，被配置为统计至少一个流媒体对象各自相应的排序范围的范围数量；
数量配置模块770，被配置为根据范围数量，分别对至少一个流媒体对象各自具有的排序关系描述符的范围数量字段进行配置。
在本申请的一些实施例中，编码侧的流媒体传输装置700还包括：
绑定比较模块780，被配置为获取至少一个流媒体对象各自相应的组件类型，基于组件类型，对至少一个流媒体对象进行排序绑定判定，获得排序绑定判定结果；
绑定配置模块790，被配置为在排序绑定判定结果表征为存在具有排序绑定关系的待绑定流媒体对象的情况下，根据排序绑定关系，对待绑定流媒体对象各自具有的排序关系描述符的组合标识字段进行配置，并按照目标字段值对待绑定流媒体对象各自具有的排序关系描述符的组合判定字段进行配置。
本申请各实施例中提供的流媒体传输装置的具体细节已经在对应的方法实施例中进行了详细的描述，此处不再赘述。
图8示意性地示出了用于实现本申请实施例的电子设备的计算机系统结构框图，该电子设备包括一个或多个处理器和存储器，存储器中存储有计算机程序，该处理器执行计算机程序时实现上述各方法实施例中的步骤。
需要说明的是，图8示出的电子设备的计算机系统800仅是一个示例，不应对本申请实施例的功能和使用范围带来任何限制。
如图8所示，计算机系统900包括中央处理器901(Central Processing Unit，CPU)，其可以根据存储在只读存储器802(Read-Only Memory，ROM)中的程序或者从存储部分808加载到随机访问存储器803(Random Access Memory，RAM)中的程序而执行各种适当的动作和处理。在随机访问存储器803中，还存储有系统操作所需的各种程序和数据。中央处理器801、在只读存储器802以及随机访问存储器803通过总线804彼此相连。输入/输出接口805(Input/Output接口，即I/O接口)也连接至总线804。
以下部件连接至输入/输出接口805：包括键盘、鼠标等的输入部分806；包括诸如阴极射线管(Cathode Ray Tube，CRT)、液晶显示器(Liquid Crystal Display，LCD)等以及扬声器等的输出部分807；包括硬盘等的存储部分808；以及包括诸如局域网卡、调制解调器等的网络接口卡的通信部分809。通信部分809经由诸如因特网的网络执行通信处理。驱动器810也根据需要连接至输入/输出接口805。可拆卸介质811，诸如磁盘、光盘、磁光盘、半导体存储器等等，根据需要安装在驱动器810上，以便于从其上读出的计算机程序根据需要被安装入存储部分808。
特别地，根据本申请的实施例，各个方法流程图中所描述的过程可以被实现为计算机软件程序。例如，本申请的实施例包括一种计算机程序产品，其包括承载在计算机可读存储介质上的计算机可读指令，该计算机可读指令包含用于执行流媒体传输方法的程序代码。在这样的实施例中，该计算机可读指令可以通过通信部分809从网络上被下载和安装，和/或从可拆卸介质811被安装。在该计算机可读指令被中央处理器801执行时，执行本申请的系统中限定的各种功能。
在一些实施例中，本申请还提供了一个或多个计算机存储介质，且计算机存储介质中存储有至少一条计算机可读指令，且该计算机可读指令包括程序指令，当处理器执行上述程序指令时，能够执行前文所对应实施例中的流媒体传输方法，因此，这里将不再进行赘述。
根据本申请的一个方面，提供了一种计算机程序产品或计算机可读指令，该计算机程序产品或计算机可读指令包括计算机可读指令代码，该计算机可读指令代码存储在计算机 可读存储介质中。计算机设备的处理器从计算机可读存储介质读取该计算机可读指令代码，处理器执行该计算机可读指令代码，使得该计算机设备可以执行前文所对应实施例中的流媒体传输方法，因此，这里将不再进行赘述。
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程，是可以通过计算机程序来指令相关的硬件来完成，所述的计算机程序可存储于一非易失性计算机可读取存储介质中，该计算机程序在执行时，可包括如上述各方法的实施例的流程。其中，本申请所提供的各实施例中所使用的对存储器、数据库或其它介质的任何引用，均可包括非易失性和易失性存储器中的至少一种。非易失性存储器可包括只读存储器(Read-Only Memory，ROM)、磁带、软盘、闪存、光存储器、高密度嵌入式非易失性存储器、阻变存储器(ReRAM)、磁变存储器(Magnetoresistive Random Access Memory，MRAM)、铁电存储器(Ferroelectric Random Access Memory，FRAM)、相变存储器(Phase Change Memory，PCM)、石墨烯存储器等。易失性存储器可包括随机存取存储器(Random Access Memory，RAM)或外部高速缓冲存储器等。作为说明而非局限，RAM可以是多种形式，比如静态随机存取存储器(Static Random Access Memory，SRAM)或动态随机存取存储器(Dynamic Random Access Memory，DRAM)等。以上所述实施例的各技术特征可以进行任意的组合，为使描述简洁，未对上述实施例中的各个技术特征所有可能的组合都进行描述，然而，只要这些技术特征的组合不存在矛盾，都应当认为是本说明书记载的范围。
以上所述实施例仅表达了本申请的几种实施方式，其描述较为具体和详细，但并不能因此而理解为对申请专利范围的限制。应当指出的是，对于本领域的普通技术人员来说，在不脱离本申请构思的前提下，还可以做出若干变形和改进，这些都属于本申请的保护范围。因此，本申请专利的保护范围应以所附权利要求为准。

Claims (20)

  1. 一种流媒体传输方法，由电子设备执行，其特征在于，包括：
    解析流媒体传输信令，得到各流媒体对象各自具有的排序关系描述符，所述排序关系描述符包括排序标识字段以及排序信息字段；所述排序标识字段用于表示排序范围；所述排序信息字段用于作为排序依据；
    将具有相同排序标识字段的至少两个流媒体对象组成排序关系集合；
    根据所述排序信息字段，对所述排序关系集合中至少两个流媒体对象进行排序，得到流媒体对象序列；及
    从所述流媒体对象序列中选取待接收的目标流媒体对象，发送针对所述目标流媒体对象的流媒体传输请求，所述流媒体传输请求用于请求获取所述目标流媒体对象。
  2. 根据权利要求1所述的流媒体传输方法，其特征在于，所述排序关系描述符还包括用于表示是否对流媒体对象进行组合排序的组合判定字段；所述根据所述排序信息字段，对所述排序关系集合中至少两个流媒体对象进行排序，得到流媒体对象序列包括：
    基于所述排序关系集合中至少两个流媒体对象各自具有的组合判定字段，获得组合判定结果；及
    基于所述组合判定结果和所述排序信息字段，对所述排序关系集合中至少两个流媒体对象进行排序，得到流媒体对象序列。
  3. 根据权利要求2所述的流媒体传输方法，其特征在于，所述排序关系描述符还包括用于表示组合排序范围的组合标识字段；所述基于所述组合判定结果和所述排序信息字段，对所述排序关系集合中至少两个流媒体对象进行排序，得到流媒体对象序列包括：
    在所述组合判定结果表征为需要进行组合判定的情况下，基于所述至少两个流媒体对象各自具有的组合标识字段，对所述至少两个流媒体对象进行组合，获得组合结果；及
    基于所述组合结果和所述排序信息字段，对所述至少两个流媒体对象进行排序，获得流媒体对象序列。
  4. 根据权利要求3所述的流媒体传输方法，其特征在于，所述基于所述至少两个流媒体对象各自具有的组合标识字段，对所述至少两个流媒体对象进行组合，获得组合结果包括：
    将具有相同组合标识字段的流媒体对象组合为组合对象，获得组合结果，所述组合结果包括至少一个组合对象；
    所述基于所述组合结果和所述排序信息字段，对所述至少两个流媒体对象进行排序，获得流媒体对象序列包括：
    在所述至少两个流媒体对象均被组合的情况下，基于所述至少一个组合对象各自具有的排序信息字段，对所述至少一个组合对象进行排序，获得第一对象序列，将所述第一对象序列作为流媒体对象序列；及
    在所述至少两个流媒体对象中存在至少一个未组合流媒体对象的情况下，基于所述排序信息字段，对所述至少一个组合对象和所述至少一个未组合流媒体对象进行排序，获得第二对象序列，将所述第二对象序列作为流媒体对象序列。
  5. 根据权利要求2所述的流媒体传输方法，其特征在于，所述基于所述组合判定结果和所述排序信息字段，对所述排序关系集合中至少两个流媒体对象进行排序，得到流媒体对象序列包括：
    在所述组合判定结果表征为不需要进行组合判定的情况下，基于所述至少两个流媒体对象各自具有的排序信息字段，对所述至少两个流媒体对象进行排序，得到流媒体对象序列。
  6. 根据权利要求1至5中任意一项所述的流媒体传输方法，其特征在于，所述排序关系描述符还包括范围数量字段，所述范围数量字段用于表示流媒体对象所属的排序范围的 数量。
  7. 根据权利要求1至5中任意一项所述的流媒体传输方法，其特征在于，所述排序信息字段包括质量排序信息、优先级排序信息或者未定义排序信息中的至少一种，所述质量排序信息用于表示按照流媒体质量对流媒体对象进行排序的排序等级，所述优先级排序信息用于表示按照选取优先级对流媒体对象进行排序的排序等级，所述未定义排序信息用于表示按照指定规则对流媒体对象进行排序的排序等级。
  8. 根据权利要求1至5中任意一项所述的流媒体传输方法，其特征在于，所述流媒体对象包括码流文件、自适应子集或者预选集中的至少一种，所述码流文件是包括至少一个流媒体资源的文件对象，所述自适应子集是包括至少一个码流文件的文件对象，所述预选集是按照预设规则对所述码流文件或者所述自适应子集进行组合得到的文件对象。
  9. 根据权利要求8所述的流媒体传输方法，其特征在于，所述流媒体资源为点云媒体资源；所述流媒体对象包括几何组件和属性组件，所述几何组件是用于表示点云几何成分的自适应子集，所述属性组件是用于表示点云属性成分的自适应子集。
  10. 一种流媒体传输方法，由电子设备执行，其特征在于，包括：
    获取至少一个流媒体对象各自相应的排序范围以及各自相应的排序依据；
    根据所述排序范围，分别对所述至少一个流媒体对象各自具有的排序关系描述符的排序标识字段进行配置；
    根据所述排序依据，分别对所述至少一个流媒体对象各自具有的排序关系描述符的排序信息字段进行配置；
    基于所述至少一个流媒体对象各自具有的排序关系描述符，生成流媒体传输指令，将所述流媒体传输信令发送至数据接收方；及
    响应于所述数据接收方发送的对应于所述流媒体传输信令的流媒体传输请求，向所述数据接收方返回与所述流媒体传输请求相匹配的目标流媒体对象。
  11. 根据权利要求10所述的流媒体传输方法，其特征在于，获取至少一个流媒体对象各自相应的排序范围以及各自相应的排序依据，包括：
    对至少一个流媒体对象进行分类，获得所述至少一个流媒体对象各自相应的排序范围；及
    对归属于相同排序范围的流媒体对象进行排序，获得所述至少一个流媒体对象各自相应的排序依据。
  12. 根据权利要求10所述的流媒体传输方法，其特征在于，在所述将所述流媒体传输信令发送至数据接收方之前，还包括：
    统计所述至少一个流媒体对象各自相应的排序范围的范围数量；及
    根据所述范围数量，分别对所述至少一个流媒体对象各自具有的排序关系描述符的范围数量字段进行配置。
  13. 根据权利要求10至12中任意一项所述的流媒体传输方法，其特征在于，在所述将所述流媒体传输信令发送至数据接收方之前，还包括：
    获取所述至少一个流媒体对象各自相应的组件类型，基于所述组件类型，对所述至少一个流媒体对象进行排序绑定判定，获得排序绑定判定结果；及
    在所述排序绑定判定结果表征为存在具有排序绑定关系的待绑定流媒体对象的情况下，根据所述排序绑定关系，对所述待绑定流媒体对象各自具有的排序关系描述符的组合标识字段进行配置，并按照目标字段值对所述待绑定流媒体对象各自具有的排序关系描述符的组合判定字段进行配置。
  14. 一种流媒体传输装置，其特征在于，包括：
    信令解析模块，被配置为解析流媒体传输信令，得到各流媒体对象各自具有的排序关系描述符，所述排序关系描述符包括排序标识字段以及排序信息字段；所述排序标识字段用于表示排序范围；所述排序信息字段用于作为排序依据；
    对象集合模块，被配置为将具有相同排序标识字段的至少两个流媒体对象组成排序关系集合；
    对象排序模块，被配置为根据所述排序信息字段，对所述排序关系集合中至少两个流媒体对象进行排序，得到流媒体对象序列；
    请求发送模块，被配置为从所述流媒体对象序列中选取待接收的目标流媒体对象，发送针对所述目标流媒体对象的流媒体传输请求，所述流媒体传输请求用于请求获取所述目标流媒体对象。
  15. 根据权利要求14所述的流媒体传输装置，其特征在于，所述对象排序模块还用于基于所述排序关系集合中至少两个流媒体对象各自具有的组合判定字段，获得组合判定结果，基于所述组合判定结果和所述排序信息字段，对所述排序关系集合中至少两个流媒体对象进行排序，得到流媒体对象序列。
  16. 根据权利要求15所述的流媒体传输装置，其特征在于，所述排序关系描述符还包括用于表示组合排序范围的组合标识字段；所述对象排序模块还用于在所述组合判定结果表征为需要进行组合判定的情况下，基于所述至少两个流媒体对象各自具有的组合标识字段，对所述至少两个流媒体对象进行组合，获得组合结果，基于所述组合结果和所述排序信息字段，对所述至少两个流媒体对象进行排序，获得流媒体对象序列。
  17. 一种流媒体传输装置，其特征在于，包括：
    获取模块，被配置为获取至少一个流媒体对象各自相应的排序范围以及各自相应的排序依据；
    范围配置模块，被配置为根据所述排序范围，分别对所述至少一个流媒体对象各自具有的排序关系描述符的排序标识字段进行配置；
    依据配置模块，被配置为根据所述排序依据，分别对所述至少一个流媒体对象各自具有的排序关系描述符的排序信息字段进行配置；
    信令发送模块，被配置为基于所述至少一个流媒体对象各自具有的排序关系描述符，生成流媒体传输指令，将所述流媒体传输信令发送至数据接收方；
    对象返回模块，被配置为响应于所述数据接收方发送的对应于所述流媒体传输信令的流媒体传输请求，向所述数据接收方返回与所述流媒体传输请求相匹配的目标流媒体对象。
  18. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质上存储有计算机程序，所述计算机程序被处理器执行时实现权利要求1至13中任意一项所述的流媒体传输方法。
  19. 一种电子设备，其特征在于，包括：
    处理器；以及
    存储器，用于存储所述处理器的可执行指令；
    其中，所述处理器配置为经由执行所述可执行指令使得所述电子设备执行权利要求1至13中任意一项所述的流媒体传输方法。
  20. 一种计算机程序产品，包括计算机程序，其特征在于，所述计算机程序被处理器执行时实现权利要求1至13中任意一项所述的流媒体传输方法。
PCT/CN2022/137754 2022-01-10 2022-12-09 流媒体传输方法、装置、电子设备及计算机可读存储介质 WO2023130893A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210022663.8 2022-01-10
CN202210022663.8A CN116455880A (zh) 2022-01-10 2022-01-10 流媒体传输方法及相关产品

Publications (1)

Publication Number Publication Date
WO2023130893A1 true WO2023130893A1 (zh) 2023-07-13

Family

ID=87073053

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/137754 WO2023130893A1 (zh) 2022-01-10 2022-12-09 流媒体传输方法、装置、电子设备及计算机可读存储介质

Country Status (2)

Country Link
CN (1) CN116455880A (zh)
WO (1) WO2023130893A1 (zh)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5933567A (en) * 1993-01-13 1999-08-03 Hitachi America, Ltd. Method and apparatus for controlling the position of the heads of a digital video tape recorder during trick play operation and for recording digital data on a tape
CN102130886A (zh) * 2010-01-18 2011-07-20 中国电信股份有限公司 网络视频流媒体系统及传输处理方法、发送端和接收端
US8565228B1 (en) * 2007-03-28 2013-10-22 Control4 Corporation Systems and methods for selecting and ranking video streams
US20140143385A1 (en) * 2012-11-20 2014-05-22 Pelco, Inc. Method and apparatus for efficiently prioritizing elements in a video stream for low-bandwidth transmission
CN108900804A (zh) * 2018-07-09 2018-11-27 南通世盾信息技术有限公司 一种基于视频熵的自适应视频流处理方法
US20210409998A1 (en) * 2020-06-26 2021-12-30 T-Mobile Usa, Inc. Quality of experience flow control for dual connectivity

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5933567A (en) * 1993-01-13 1999-08-03 Hitachi America, Ltd. Method and apparatus for controlling the position of the heads of a digital video tape recorder during trick play operation and for recording digital data on a tape
US8565228B1 (en) * 2007-03-28 2013-10-22 Control4 Corporation Systems and methods for selecting and ranking video streams
CN102130886A (zh) * 2010-01-18 2011-07-20 中国电信股份有限公司 网络视频流媒体系统及传输处理方法、发送端和接收端
US20140143385A1 (en) * 2012-11-20 2014-05-22 Pelco, Inc. Method and apparatus for efficiently prioritizing elements in a video stream for low-bandwidth transmission
CN108900804A (zh) * 2018-07-09 2018-11-27 南通世盾信息技术有限公司 一种基于视频熵的自适应视频流处理方法
US20210409998A1 (en) * 2020-06-26 2021-12-30 T-Mobile Usa, Inc. Quality of experience flow control for dual connectivity

Also Published As

Publication number Publication date
CN116455880A (zh) 2023-07-18

Similar Documents

Publication Publication Date Title
AU2017213593B2 (en) Transmission of reconstruction data in a tiered signal quality hierarchy
EP3493547B1 (en) Video streaming delivery
CN112804256B (zh) 多媒体文件中轨道数据的处理方法、装置、介质及设备
WO2018014523A1 (zh) 一种媒体数据的获取方法和装置
CN113329267B (zh) 一种视频播放方法、装置、终端设备及存储介质
WO2019128668A1 (zh) 视频码流处理方法、装置、网络设备和可读存储介质
WO2021057697A1 (zh) 视频编解码方法和装置、存储介质及电子装置
CN113348666A (zh) 用于图块组识别的方法
Laghari et al. The state of art and review on video streaming
US20240080487A1 (en) Method, apparatus for processing media data, computer device and storage medium
WO2023130893A1 (zh) 流媒体传输方法、装置、电子设备及计算机可读存储介质
WO2022042325A1 (zh) 视频处理方法、装置、设备及存储介质
CN114600083A (zh) 信令通知云工作流中的输入数据的超时和完整
WO2022247452A1 (zh) 多媒体资源中轨道数据的处理方法、装置、介质及设备
WO2024114519A1 (zh) 点云封装与解封装方法、装置、介质及电子设备
US20240121397A1 (en) Temporal sublayer information for video coding
WO2023130899A1 (zh) 环路滤波方法、视频编解码方法、装置、介质及电子设备
US20130287100A1 (en) Mechanism for facilitating cost-efficient and low-latency encoding of video streams
WO2023059689A1 (en) Systems and methods for predictive coding
Goyal et al. Comparative analysis of performance of WMV & MPEG formats video streaming in a cloud
JP2024516366A (ja) ブロック重要性マッピング

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22918343

Country of ref document: EP

Kind code of ref document: A1