WO2024017135A1

WO2024017135A1 - Method for processing image, and apparatus

Info

Publication number: WO2024017135A1
Application number: PCT/CN2023/107255
Authority: WO
Inventors: 胡少领; 窦凤辉
Original assignee: 华为技术有限公司
Priority date: 2022-07-21
Filing date: 2023-07-13
Publication date: 2024-01-25

Abstract

The present application relates to a method for processing an image, and an apparatus. In the method, by using a backward reference range, an encoding side of an image indicates a reference relationship between information units of the image during an encoding process. A first network element receives first information from the encoding side, wherein the first information indicates a backward reference range of a first information unit of the image or time parameters determined by the backward reference range, and since the backward reference range reflects the situation that an information unit generated after the first information unit performs encoding with reference to the first information unit, the first information can provide a basis for the first network element to improve the transmission guarantee and cache of the information unit; and after acquiring the first information, the first network element performs caching or transmission on the first information unit in a cache region according to the backward reference range indicated by the first information or the time parameters indicated by the first information. In this way, the image transmission performance can be improved, for example, the transmission efficiency is improved, the frame loss rate is reduced and the like.

Description

Image processing methods and devices

This application requests the priority of the Chinese patent application submitted to the State Intellectual Property Office of China on July 21, 2022, with the application number "202210864123.4" and the application title "A dependency indication method, terminal equipment, and network equipment". As well as the priority of the Chinese patent application submitted to the State Intellectual Property Office of China on August 9, 2022, with the application number "202210952281.5" and the application title "Method and Device for Processing Images", the entire content of which is incorporated into this application by reference. middle.

Technical field

The embodiments of the present application relate to the field of image processing, and more specifically, to a method and device for processing images.

Background technique

In video coding and decoding technology, video frames can be encoded at frame or slice granularity. According to the existing video coding standards, a frame is a complete picture, and a slice is a partial area in the picture, which is coded separately from other areas in the picture. For frame, there are I frame and P frame. I frame can be decoded independently, and the decoding of P frame needs to refer to other frames. The slice is similar. I slice can be decoded independently, while the decoding of P slice needs to refer to other slices. In extended reality (XR) media services, a PDU set consists of one or more PDUs, containing information of a unit (such as a frame or a slice) generated by the application layer. The reference relationship between PDU sets plays an important role in improving the video transmission efficiency of the communication system.

Contents of the invention

Some technologies carry the sequence number of the PDU set (PDU set sequence number, PDU-SN) and the sequence number of the reference PDU set (reference PDU set sequence number, RPS-SN) in a PDU set. Among them, RPS-SN is used to indicate the sequence number of the PDU set referenced by the current PDU set. In this technology, the reference relationship between PDU sets is based on forward reference. It is meaningful to send the current PDU set only if the referenced PDU set is successfully transmitted. When the referenced PDU set is not successfully transmitted, the RPS-SN carried in the current PDU set is used by the fifth generation core network (the fifth generation core, 5GC) or the radio access network (radio access network, RAN). Used to decide to discard all PDU sets referencing this RPS.

However, providing the sequence number of the PDU set referenced by the PDU set (that is, providing RPS-SN) can only be used by 5GC or RAN to decide whether to discard the PDU set. For 5GC or RAN, it can improve the transmission performance of PDU set (for example, improve transmission Efficiency, reducing frame loss rate, etc.) does not help much.

In order to solve the above technical problems, this application provides an image processing method and device, which can improve image transmission performance.

The first aspect provides a method for processing images, which method includes:

The first network element obtains the first information from the application function network element, where,

The first information indicates the backward reference range of the first information unit of the image. The backward reference range is used to indicate the distance between the index of the second information unit and the index of the first information unit. The information unit generated after the second information unit No further reference is made to the first information unit; or,

The first information indicates the temporal parameter of the first information unit of the image. The temporal parameter is determined according to the backward reference range of the first information unit. The backward reference range is used to indicate the index of the second information unit and the index of the first information unit. The spacing between them means that information units generated after the second information unit no longer refer to the first information unit;

The first network element processes the first information unit in the buffer area of the first network element according to the first information.

In this technical solution, the encoding side of the image (such as the application function network element) uses the backward reference range to indicate the reference relationship between the information units (such as frames or slices) of the image during the encoding process. The encoding side sends first information to the first network element, where the first information is used to indicate the backward reference range of the first information unit or the time parameter determined by the backward reference range. After the first network element obtains the first information, it can obtain the time parameter of the first information unit. Since the time parameter is determined based on the backward reference range of the first information unit, and the backward reference range reflects the situation in which information units generated after the first information unit are encoded with reference to the first information unit, the time parameter may be the first information unit. Network elements improve the transmission guarantee of information units and provide a basis for caching. The first network element caches the buffer area according to the time parameter. Compared with blindly deciding the cache duration or transmission duration of the first information unit, and randomly discarding the information units in the cache area to affect other information units generated after the first information unit. In terms of decoding, the related processing such as caching and transmission performed by the first network element on the first information unit is more rational, thereby improving image transmission performance. For example, it can reduce the frame loss rate, reduce resource waste caused by random packet loss, and improve transmission efficiency.

In a possible implementation manner, if the first information indicates the backward reference range of the first information unit, the method further includes: the first network element determines the time parameter according to the backward reference range.

In this implementation, if the encoding side indicates to the first network element the backward reference range of the first information unit instead of the time parameter of the first information unit, the first network element can convert the backward reference range into a time parameter, The flexibility of the encoding side indicating the backward reference relationship of the information unit to the first network element can be increased.

Optionally, compared to the first information indicating the time parameter, if the first information indicates the backward reference range, the impact caused by data packet jitter can be reduced and the first network element can be provided with more accurate information for caching or transmitting. basis of the unit.

In a possible implementation, the time parameter includes at least one of a maximum storage duration and a maximum allowed transmission duration of the first information unit, where the maximum storage duration is the first information unit received from the first network element. The duration from the beginning of a moment to the second moment when the first network element receives M information units generated after the first information unit, M is the value of the spacing indicated by the backward reference range, and M is an integer; The maximum allowed transmission duration is the duration from the first moment when the first network element receives the first information unit to the third moment when it is determined that the first network element will no longer send the first information unit to the terminal device.

In this implementation, the time parameter of the first information unit mainly includes the maximum storage time of the first information unit in the cache area of the first network element, and the maximum allowed transmission time of the first information unit. The first network element will not delete the first information unit from the cache area within the maximum storage time, so as to prevent other information units generated after the first information unit from referring to the first information unit code, and the first information unit is deleted prematurely, resulting in reference These information units encoded by the first information unit cannot be successfully decoded on the decoding side, thereby reducing image transmission efficiency and wasting resource scheduling. In addition, if the first information unit is not successfully received by the decoding side (for example, the UE), the first network element can repeatedly transmit the first information unit within the maximum allowed transmission duration to improve the reception success rate of the decoding side. Moreover, within the maximum allowed transmission duration, if there are other information units generated after the first information unit that are encoded with reference to the first information unit, these information units that are generated and encoded with reference to the first information unit can also be promoted through retransmission. Decoding success rate on the decoding side until the first information unit loses transmission value (or exceeds the maximum allowed transmission duration). With the assistance of the maximum storage duration and the maximum allowed transmission duration, the decoding success rate on the decoding side can be improved, the waste of transmission resources and the scheduling of invalid information units can be reduced, and the image transmission performance can be improved.

For example, the loss of transmission value of the first information unit may mean that subsequent information units no longer directly refer to the first information unit for encoding (or that the first information unit no longer has a reference role in decoding subsequent information units), Or the maximum allowed transmission duration of the first information unit has been exceeded (or the first information unit is an invalid information unit).

In a possible implementation, each information unit in the cache area of the first network element corresponds to a reference mark, and the reference mark corresponding to each information unit is one of the following three types: for long-term reference, for user use. For short-term reference and not for reference, the three types respectively correspond to different backward reference ranges.

In this implementation, different backward reference range values are set for different types of reference marks. The backward reference range values of different information units can be set according to the objective conditions of the reference relationship between information units. value to improve transmission performance.

In a possible implementation, the first information unit corresponds to the first reference mark; the method further includes:

The first network element obtains first change signaling and second information from the application function network element. The first change signaling is used to indicate that the reference mark of the first information unit is changed from the first reference mark to the second reference mark. The second information Used to indicate the backward reference range corresponding to the second reference mark, wherein the first change signaling carries the identity of the first information unit, and the type of the first reference mark is different from the type of the second reference mark;

The first network element modifies the reference mark of the first information unit to the second reference mark according to the first change signaling and the second information, and modifies the backward reference range of the first information unit to the backward reference range corresponding to the second reference mark. towards the reference range.

In this implementation, after the encoding side (for example, AF network element) updates the reference mark of an information unit, the backward reference range of the information unit changes accordingly. Therefore, the encoding side needs to change the backward reference range of the information unit. Notify the first network element of the update, so that the first network element can synchronously update the backward reference range of the information unit. The first network element determines the time parameters of the information unit (such as the maximum storage duration and the maximum allowed transmission duration) based on the updated backward reference range, and can dynamically adapt to objective changes in the reference relationship between information units to maintain a stable Image transmission efficiency and transmission performance.

In a possible implementation, the first network element obtains the first information from the application function network element, including:

The first network element obtains the first information from the application function network element through control plane signaling; or,

The first network element receives a first protocol data unit PDU set from the application function network element, and the first PDU set contains first information of the first information unit.

In the implementation, the encoding side can send the first information to the first network element through control plane signaling or "on-the-road transmission". The former method is suitable when the backward reference range is relatively stable or the change period is relatively long. For long-term application scenarios, the latter approach is suitable for application scenarios where the backward reference range changes more frequently. The encoding side uses different ways to transmit the first information according to different application scenarios, which can improve the transmission performance of the first information, such as reducing transmission overhead.

In a possible implementation, the first network element processes the first information unit in the buffer area of the first network element according to the first information, including:

The first network element retains the first information unit in the cache area within the maximum storage time period according to the first information.

In this implementation, the first network element retains the first information unit in the cache area within the maximum storage time, so as to avoid premature deletion of the first information unit and affect later generated information units that are encoded with reference to the first information unit. Decoding can reduce the probability of losing valid information units.

In a possible implementation, the method also includes:

M information units generated after the first network element receives the first information unit;

Within the maximum storage time, if the first network element determines that none of the M information units are encoded with reference to the first information unit, the first network element deletes the first information unit.

In this implementation, if the first network element determines that no information unit within the backward reference range of the first information unit refers to the first information unit within the maximum storage time, the first information unit can be deleted in time to avoid invalid The information unit occupies the buffer area.

In a possible implementation, the first network element is a RAN network element, and the first network element determines that none of the M information units are encoded with reference to the first information unit, including:

The first network element obtains M third information from the user plane function UPF network element. The M third information corresponds to the M information units one-to-one. Each of the M third information is used to determine the corresponding Whether the information unit refers to the first information unit for encoding;

The first network element determines, based on the M pieces of third information, that none of the M information units are encoded with reference to the first information unit.

In this implementation, if the first network element is a RAN network element, the RAN network element needs the assistance of the UPF network element to determine whether the information units within the backward reference range of the first information unit refer to the first information unit. Encoding, and then when no information unit within the backward reference range refers to the first information unit, delete the first information unit from the cache area in time to prevent the invalid first information unit from occupying the cache area.

For example, the third information in this implementation may be RPS-SN. The M pieces of third information are the RPS-SNs indicated by each of the M pieces of information units generated after the first information unit.

In a possible implementation, the method also includes:

The first network element receives the first information unit from the application function network element and saves the first information unit in the buffer area of the first network element;

The first network element sends the first information unit to the terminal device;

And, the first network element processes the first information unit in the buffer area of the first network element according to the first information, including:

If the terminal device fails to receive the first information unit successfully, the first network element repeatedly sends the first information unit to the terminal device within the maximum transmission duration according to the first information.

In this implementation, within the maximum allowed transmission duration, the decoding success rate on the decoding side of the information units generated and encoded with reference to the first information unit can also be improved through retransmission.

In a possible implementation, the first network element processes the first information unit in the buffer area of the first network element according to the first information, which further includes:

The first network element determines based on the first information that when the maximum transmission duration is exceeded and the terminal device still has not correctly received the first information unit, the first network element no longer sends the first information unit to the terminal device.

In this implementation, after the maximum allowed transmission duration is exceeded, the first information unit no longer has transmission value. For example, the first information unit is useless for subsequent decoding of other information units. At this time, the first network element The first unit is no longer sent repeatedly to avoid invalid transmission and waste of transmission resources as much as possible.

In a possible implementation manner, the first network element is a UPF network element, and the method further includes:

The first network element sends the first information unit to the RAN network element;

The first network element receives feedback information from the RAN network element, and the feedback information is used to indicate that the RAN network element has not successfully sent the first information unit to the terminal device;

If the first information unit is retained in the buffer area of the first network element, and one or more third information units generated after the first information unit is determined to be encoded with reference to the first information unit, the first network element sends to the RAN network element a first information unit and one or more third information units.

In this implementation, the first network element is a UPF network element, and the information unit on the encoding side is cached in the UPF network element. If the terminal device fails to receive a certain information unit (for example, the first information unit), the RAN network element provides feedback information to the UPF network element, so that several pieces of information are generated by the UPF network element after the first information unit is not deleted and exists. When the unit is encoded with reference to the first information unit, the first information unit and the information units encoded with reference to the first information unit may be repeatedly sent to the terminal device to improve the image reception success rate.

In a possible implementation, the method also includes:

If the terminal device fails to receive the first information unit successfully, and the first network element determines that the first information unit has been deleted from the cache area of the first network element, the first network element deletes the first information unit from the cache and performs encoding with reference to the first information unit. one or more third information units.

In this implementation, since the reference relationship between information units provided by this application is a backward reference relationship, if the information unit generated first is not successfully received and has been deleted from the buffer area of the first network element, then the information unit generated later will Information units that are encoded with reference to this information unit cannot be successfully decoded. In this case, the first network element deletes these information units that cannot be successfully decoded from the buffer area, which can avoid invalid scheduling and reduce the waste of transmission resources. .

In a possible implementation manner, the first network element is a RAN network element or a UPF network element.

A second aspect provides a device for processing images, the device having the function of implementing the method in the first aspect or any possible implementation of the first aspect. The functions described can be implemented by hardware, or can be implemented by hardware executing corresponding software. The hardware or software includes one or more units corresponding to the above functions.

In a third aspect, an image processing device is provided, including a processor and a memory. Optionally, a transceiver may also be included. Wherein, the memory is used to store computer programs, and the processor is used to call and run the computer programs stored in the memory, and control the transceiver to send and receive signals, so that the communication device performs the method of the first aspect, or any possible method of the first aspect. Methods in the implementation.

A fourth aspect provides a device for processing images, including a processor and a communication interface. The communication interface is used to receive data and/or information and transmit the received data and/or information to the processor. The processor processes the data and/or information, and the communication interface is also used to output the data and/or information processed by the processor, so as to achieve the method of the first aspect, or any possible method of the first aspect. The methods in the implementation are executed.

A fifth aspect provides an apparatus for processing images, including at least one processor coupled to at least one memory, and the at least one processor is configured to execute a computer program or instructions stored in the at least one memory. , so that the communication device performs the method of the first aspect, or the method in any possible implementation of the first aspect.

In a sixth aspect, a computer-readable storage medium is provided. Computer instructions are stored in the computer-readable storage medium. When the computer instructions are run on a computer, the first aspect or the second aspect, or the method in these aspects is achieved. Methods in any possible implementation of either aspect are executed.

In a seventh aspect, a computer program product is provided, the computer program product comprising computer program code, when the computer program code is run on a computer, it causes the first aspect or the second aspect, or any of these aspects. Methods in any possible implementation of the aspect are executed.

An eighth aspect provides a system for processing images, including the device for processing images as described in the second aspect.

Description of drawings

Figure 1 is a schematic diagram of using sliding window mode to mark reference pictures.

Figure 2 is a schematic diagram of adaptive mode marking reference pictures.

Figure 3 is a schematic diagram of a system architecture suitable for embodiments of the present application.

Figure 4 is a schematic flow chart of the image processing method provided by this application.

Figure 5 is a schematic diagram of the backward reference range proposed in this application.

Figure 6 is a schematic diagram of a method of obtaining the maximum allowed transmission duration based on the backward reference range.

Figure 7 is a schematic diagram of the AF network element transmitting the backward reference range through the control plane transmission path.

Figure 8 is a schematic diagram of the backward reference range of each information unit transmitted along the path.

Figure 9 is an example of an image processing method provided by this application.

Figure 10 is an example of an image processing method provided by this application.

Figure 11 is a schematic block diagram of an image processing device provided by this application.

Figure 12 is a schematic structural diagram of an image processing device provided by this application.

Detailed ways

The technical solutions in this application will be described below with reference to the accompanying drawings.

In order to facilitate understanding of the technical solutions of this application, some concepts that may be involved in the embodiments of this application are first briefly introduced.

Video coding generally refers to the processing of sequences of pictures that form a video or video sequence. In the field of video coding, the terms "picture", "frame" or "image" can be used as synonyms. Video encoding as used in this article means video encoding or video decoding. Video encoding is performed on the source side and typically involves processing the raw video pictures. Video decoding is performed on the destination side and typically involves inverse processing relative to the encoder to reconstruct the video picture.

A video sequence includes a series of video frames, hereinafter referred to as "frames", and the frames are further divided into slices. A frame is a complete picture. According to the H.264 standard, or moving pictures experts group version 4 (MPEG-4) advanced video coding (AVC) standard, a slice is a partial area in a picture.

In XR media services, a protocol data unit (PDU) set (PDU set) consists of one or more PDUs and contains information of a unit (such as a frame or a slice) generated by the application layer. A frame or a slice corresponds to one or more PDU sets, and a PDU set contains one or more PDUs.

When performing video decoding, pictures in the decoded picture buffer (DPB) can be marked as the following three types:

unused for reference(not used for reference);

used for long term reference(used for long term reference);

used for short term reference(used for short term reference).

The conversion between these three types is controlled through the labeling process of decoded pictures.

Among them, pictures marked as "used for long term reference" can be stored in the decoding buffer area for a long time and affect the decoding of subsequent pictures until the application layer on the decoding end actively deletes the picture. Compared with the pictures marked "used for long term reference", the pictures marked "used for short term reference" are cached in the decoding buffer for a relatively shorter time. Pictures marked as "used for short term reference" have two marking modes: one is the "first in, first out" sliding window mode. Figure 1 is a schematic diagram of using the sliding window mode to mark reference pictures. As shown in Figure 1, the picture that enters the decoding buffer first slides out of the decoding buffer. When a picture comes out of the decoding buffer, the mark of the picture changes from "used for short term reference" to "unused for reference". The marking mode of the sliding window mode ensures that the pictures cached in the decoding buffer area are all recently decoded pictures; the other is the adaptive marking mode. Figure 2 is a schematic diagram of adaptive marking reference pictures. As shown in Figure 2, in this mode, the application layer on the decoder side uses syntax elements to change the mark of a picture from "used for short term reference" to "used for long term reference" or "unused for reference".

The following describes the image processing method provided by this application.

The technical solutions involved in the embodiments of this application can not only be applied to some video coding standards, such as H.264, high efficiency video coding (HEVC) and other standards, but can also be used in future video coding standards, such as H .265, H.266 standards, or other video coding standards, without limitation.

As an example of an application scenario, the technical solution provided in this application can be applied to the video frame transmission scenario of the XR service. In this scenario, the encoding of the later generated video frame on the encoding side will refer to the previously generated video frame to achieve video compression. Under such a reference relationship, if the online video frame referenced by a certain video frame is not successfully transmitted, the video frame cannot be successfully decoded on the decoding side.

Referring to Figure 3, Figure 3 is a schematic diagram of a system architecture suitable for embodiments of the present application. As shown in Figure 3, the functions of the terminal equipment and each network entity are as described below.

Terminal equipment: It can also be called user equipment (UE), access terminal, terminal equipment unit (subscriber unit), terminal equipment station, mobile station, mobile station (MS), remote station, remote terminal, Mobile device, user terminal, terminal, wireless communication device, terminal equipment agent or terminal equipment device.

Radio access network (RAN): used to implement wireless physical layer functions, resource scheduling and wireless resource management, Wireless access control and mobility management functions. The RAN in the fifth generation (5G) system is connected to the user plane function (UPF) through the user plane interface N3 and is used to transmit data of terminal equipment; 5G-RAN uses the control plane interface N2 and AMF Establish a control plane signaling connection to implement functions such as wireless access bearer control.

Access and mobility management function (AMF): used to provide transmission for session management messages between UE and SMF.

UPF: External PDU session point interconnected with the data network.

Application function (AF): Interacts with the core network to provide some services, for example, interacts with the policy and control function (PCF) to provide business policy control.

Policy control function (PCF): Provides configuration policy information for the UE, and provides policy information for controlling the UE for the control plane elements of the network (for example, AMF, SMF).

Data network (DN): A network used to provide services or content.

The interfaces between each network element in Figure 3 are shown in Figure 3 .

It should be understood that FIG. 3 is only an exemplary network architecture, and the network architecture applicable to the embodiment of the present application is not limited thereto. Any network architecture that can realize the functions of each of the above network elements is applicable to the embodiment of the present application. Moreover, the system architecture in Figure 3 only shows the network elements that are mainly related to the embodiments of the present application and the interfaces between some network elements, and is not the complete architecture of the communication system.

The above network elements or functions may be network elements in hardware devices, software functions running on dedicated hardware, or virtualization functions instantiated on a platform (for example, a cloud platform). Optionally, the above network element or function can be implemented by one device, or can be implemented by multiple devices together, or can be a functional module in one device, which is not specifically limited in the embodiments of this application.

In addition, the above-mentioned "network element" can also be called an entity, equipment, device or module, etc., which is not specifically limited in this application. Moreover, in order to facilitate understanding and explanation, the description of "network element" is omitted in some of the following descriptions. For example, the UPF network element is referred to as UPF. In this case, "UPF" should be understood as UPF network element or UPF entity, which is omitted below. Description of the same or similar situation.

Next, we will take the network elements in the 5G system as an example to introduce the specific solution details. It can be understood that when this solution is used in an LTE system or a future communication system, each network element in the solution can be replaced by other network elements with corresponding functions, and this application does not limit this.

Referring to Figure 4, Figure 4 is a schematic flow chart of an image processing method provided by this application.

410. The first network element obtains the first information from the application function network element.

In this application, the first network element may be a UPF network element or a RAN network element. It should be noted that the first network element is specifically a UPF network element or a RAN network element, depending on who performs caching of the information unit (for example, frame or slice) received from the AF network element. In a possible implementation, if the UPF network element is configured to cache the received information unit, the first network element is the UPF network element. At this time, the RAN network element no longer caches the information unit. In another possible implementation, if the RAN network element is configured to cache the received information unit, the first network element is the RAN network element. At this time, the UPF network element does not cache the information unit. For specific implementation, please refer to the detailed description below.

In a possible implementation, the first information indicates the backward reference range of the first information unit of the image, and the backward reference range is used to indicate the distance between the index of the second information unit and the index of the first information unit, Information units generated after the second information unit no longer refer to the first information unit. or,

In another possible implementation, the first information indicates the time parameter of the first information unit of the image, the time parameter is determined according to the backward reference range of the first information unit, and the backward reference range is used to indicate the second information The distance between the index of the unit and the index of the first information unit. Information units generated after the second information unit no longer refer to the first information unit.

It should be noted that in the embodiment of the present application, if an information unit (for example, information unit #2) generated after information unit #1 is encoded with reference to information unit #1, it means that information unit #2 directly refers to information unit #1 Encoding, or the reference relationship between information units in the embodiment of this application, refers to "direct reference" rather than "indirect reference". For example, if information unit #2 refers to information unit #1 for encoding, and information unit #5 generated after information unit #2 refers to information unit #2 for encoding, information unit #2 directly refers to information unit #2. Unit #1, and information unit #5 can actually be considered to be an indirect reference to information unit #1. Therefore, in the embodiments of the present application, if it is said that several information units generated after the first information unit are encoded with reference to the first information unit, it means that the several information units are directly encoded with reference to the first information unit. , the description will not be repeated below.

Optionally, the first information unit may be of frame granularity or slice granularity, without limitation.

In this application, the backward reference range of the first information unit refers to the distance between the index of the second information unit and the index of the first information unit. Wherein, the information units generated after the second information unit no longer refer to the first information unit.

It should be understood that the first information unit may be an information unit previously generated during the video encoding process at the encoding end. Information units generated after the unit may be encoded with reference to the first information unit. According to the backward reference range proposed in this application, if the backward reference range of the first information unit is M, it represents the distance between the index of the second information unit generated after the first information unit and the index of the first information unit. is M, and the information unit generated after the second information unit is no longer possible to be encoded with reference to the first information unit, and M is the value of the spacing indicated by the backward reference range. In other words, if there is an information unit generated after the first information unit and is encoded with reference to the first information unit, the distance between the index of the information unit that is encoded with reference to the first information unit and the index of the first information unit is not greater than M. In addition, it should be understood that the information units generated after the first information unit and encoded with reference to the first information unit may be one or more. In the case where multiple information units are encoded with reference to the first information unit, the multiple information units The distance between the index of any one information unit in the unit and the index of the first information unit is less than or equal to M.

Refer to Figure 5, which is a schematic diagram of the backward reference range proposed in this application. As shown in Figure 5, assume that frame #0 shown in Figure 5 is used as an example of the first information unit. The index of frame #0 is 0. If the backward reference range of frame #0 is M, it means that in frame #0 The distance between the index of a frame generated later and the index of frame #0 is M, and the frame generated after this frame no longer refers to frame #0 for encoding. It can be seen that the frame is frame #M shown in Figure 5. In other words, the index of the frame generated after frame #0 and encoded with reference to frame #0 is less than or equal to M. The frame in the range shown in the dotted box in Figure 5 may directly refer to frame #0, and is located in It is no longer possible for subsequent frames outside the dotted box to be encoded with reference to frame #0.

From another perspective, the backward reference range can also be understood this way: the backward reference range of an information unit is the maximum number of information units that the encoding end can encode after the information unit is generated by the encoding end from entering the encoding end's cache to exiting the cache. . When an information unit is in the cache, the encoding end may refer to the information unit when encoding the subsequently generated information unit. However, when the information unit is not in the cache, the encoding end may no longer directly encode the subsequently generated information unit. Refer to this information unit.

In a possible implementation, the backward reference range of the first information unit may be derived from the number of information units stored in the decoded picture buffer area.

In order to clearly describe the derivation process, several parameters involved are first explained. For a frame, its backward reference range depends on the maximum number of macroblocks in the decoded picture buffer (decoded picture buffer) of the application layer video codec, denoted as MaxDpbMbs. MaxDpbMbs is related to the level of video encoding and decoding (lever). Different levels correspond to different parameters such as resolution and frame rate. MaxDpbMbs determines the maximum number of reference frames, MaxDpbFrames, where MaxDpbFrames can be determined by the following formula:

Among them, PicWidthInMbs is the number of horizontal macroblocks, hrameGeightInMbs is the number of vertical macroblocks, Indicates the maximum number of frames that can be stored in the decoded picture buffer, Indicates rounding down. In addition, the maximum number of reference frames is 16.

As mentioned above, each picture in the decoded picture buffer can be marked as one of three types: "used for long term reference" or "used for short term reference" or "unused for reference". Based on the backward reference range proposed in this application, the corresponding backward reference ranges for the above three types of pictures may be different.

For example, for a picture marked "used for long term reference", its backward reference range is M, and M can be greater than or equal to MaxDpbFrames. For example, it can be A times MaxDpbFrames, and A is an integer greater than 0. For the picture marked "used for short term reference", its backward reference range is M, and the value of M can be smaller than the value of the backward reference range M of the picture marked "used for long term reference". For example, a picture marked "used for long term reference" has its backward reference range M set to 60; a picture marked "used for short term reference" has its backward reference range M set to 40. For pictures marked as "unused for reference", their backward reference range M=0.

In addition, the time parameter of the first information unit may refer to at least one of the maximum storage duration and the maximum allowed transmission duration of the first information unit.

The maximum storage duration is the duration from the first moment when the first network element receives the first information unit to the second moment when the first network element receives M information units generated after the first information unit. Taking frame #0 in Figure 5 as an example, the maximum storage time of frame #0 is from the moment the first network element receives frame #0 to the M information units generated after the first network element receives frame #0. (Specifically, the duration from frame #1 to frame #M).

The maximum allowed transmission duration is the duration from the first moment when the first network element receives the first information unit to the third moment when it is determined that the first network element will no longer send the first information unit to the terminal device.

It should be understood that in this embodiment of the present application, the source side of the information unit is the AF network element, the destination side is the UE, and the first network element may be a UPF network element or a RAN network element. Taking the first network element as a UPF network element as an example, the UPF network element receives the information unit from the AF network element, sends the information unit to the RAN network element, and then the RAN network element sends the information unit to the UE. The maximum allowed transmission duration of the first information unit may refer to the moment when the UPF network element receives the first information unit from the AF network element until the UPF network element determines that it will no longer send the first information unit to the terminal device (via the RAN network element). The length of time between moments of information units. Taking the first network element as a RAN network element as an example, the maximum allowable transmission duration of the first information unit may refer to the time when the RAN network element receives the first information unit from the UPF network element, and when the RAN network element determines that it will no longer transmit the information. The length of time until the time when the first information unit is sent to the terminal device.

In other words, when the maximum allowed transmission duration is exceeded, even if the terminal device fails to receive the first information unit successfully, the first network element will not resend the first information unit to the terminal device. Within the maximum allowed transmission duration, if the terminal device fails to receive the first information unit successfully, the first network element may repeatedly send the first information unit to the terminal device. The repeated sending may be one or more times without limitation. Therefore, if the terminal device fails to successfully receive the first information unit initially sent by the first network element, within the maximum allowed transmission duration of the first information unit, the terminal device may successfully receive the second information unit by receiving a retransmission of the first information unit. An information unit. However, once the maximum allowed transmission duration is exceeded, even if the terminal device still fails to receive the first information unit successfully, the first network element will no longer transmit the first information unit.

Refer to Figure 6, which is a schematic diagram of a method of obtaining the maximum allowed transmission duration according to the backward reference range.

Optionally, if in step 410, the first information indicates the backward reference range of the first information unit, the first network element may determine the time parameter of the first information unit according to the backward reference range.

420. The first network element processes the first information unit in the buffer area of the first network element according to the first information.

In this application, the first network element processes the first information unit in the buffer area of the first network element according to the first information, which may specifically include:

The first network element retains the first information unit in the cache area within the maximum storage time of the first information unit according to the first information. In other words, within the maximum storage time period, the first network element will not delete the first information unit from the cache area.

In addition, within the maximum storage time, the first network element receives M information units generated after the first information unit. If it is determined that none of the M information units are encoded with reference to the first information unit, the first network element The first information unit can be deleted. After the maximum storage time is exceeded, since the subsequently generated information unit can no longer refer to the first information unit for encoding, the first network element can delete the first information unit from the buffer area.

Optionally, if the first network element is a UPF network element, the UPF network element can determine whether the M information units are encoded with reference to the first information unit by reading the header of the PDU set corresponding to each of the M information units. Specifically, the UPF network element can determine whether the M information units generated after the first information unit refer to the first information unit for encoding by detecting the header of each received information unit. Since the header of each information unit carries the sequence number (that is, RPS-SN) of the PDU set referenced by the information unit, UPF detects the header of each information unit received and can learn about each received The sequence number of the PDU set that each of the information units refers to. From this, it is possible to know which of the M information units after the first information unit each refers to the previously generated information unit for encoding, or does not refer to any information unit. Encoding is performed, so it can be determined whether each of the M information units refers to the first information unit for encoding. However, if the first network element is a RAN network element, since the RAN network element cannot directly read the header of the PDU set, the UPF network element reads the header of the PDU set corresponding to the M information units and learns the After each of the M information units refers to which previously generated information unit is encoded, the information that each information unit refers to which previously generated information unit is encoded is encapsulated into the GTP-U header so that it can be read by the RAN network element. Each of the M information units refers to information encoded by a previously generated information unit, and then it can be determined whether the M information units refer to the first information unit for encoding.

Therefore, if the first network element is a RAN network element, the RAN network element obtains M pieces of third information from the UPF network element, and the M pieces of third information correspond to the M pieces of information units generated after the first information unit. Each third information among the three pieces of information is used to determine whether the corresponding information unit refers to the first information unit for encoding. Based on the M pieces of third information, the RAN network element can determine whether the M information units generated after the first information unit refer to the first information unit for encoding.

In addition, during the process of sending the information unit to the terminal device, if the terminal device fails to receive the first information unit successfully, the first network element determines based on the first information that the maximum allowed transmission duration of the first information unit has not been exceeded, or in other words, Within the maximum allowed transmission duration of the first information unit, the first network element repeatedly sends the first information unit to the terminal device. If, based on the first information, the first network element determines that the maximum allowed transmission duration of the first information unit has been exceeded, and the terminal device still fails to receive the first information unit successfully, the first network element will no longer send data to the terminal. The device sends the first information unit.

If the first network element is a UPF network element, that is, the UPF network element caches the information unit from the encoding end. If the terminal device fails to receive the first information unit successfully, the RAN network element can send feedback information to the UPF network element. The feedback The information is used to indicate that the RAN network element has not successfully sent the first information unit to the terminal device. At this time, if it is still within the maximum storage time of the first information unit, the first information unit is still retained in the buffer area of the UPF network element, and the UPF network element determines one or more (for example, P (P is less than or equal to M, P is a positive integer) the third information unit is encoded with reference to the first information unit, then the UPF network element sends the first information unit and the one or more third information units to the RAN network element. Further, the RAN network element sends the first information unit and the one or more third information units to the terminal device again.

It should be understood that P third information units do not refer to P identical information units, but are a collective name for P information units generated after the first information unit and encoded with reference to the first information unit, which will not be discussed further below. Repeat instructions.

In another possible situation, if the terminal device successfully receives the first information unit, and the first network element (UPF network element or RAN network element) determines that the first information unit has been retrieved from the buffer area of the first network element, Delete, the first network element deletes one or more third information units that are encoded with reference to the first information unit from the cache area. It should be noted that since the one or more third information units are encoded with reference to the first information unit, and the first information unit is not successfully received at the UE side, the one or more third information units are also not received at the UE side. Unable to successfully decode, therefore, when the first information unit is not successfully received and the first network element determines that the first information unit has been deleted from the buffer area, the information unit that refers to the first information unit for encoding will no longer be sent to the UE. , directly delete these third information units that are encoded with reference to the first information unit from the buffer area.

It can be seen that the first network element can determine at least one parameter of the maximum storage duration and the maximum allowed transmission duration of the first information unit based on the first information. The maximum storage duration can be used by the first network element to determine how long to cache the first information unit, or when the first information unit can be deleted from the cache area without affecting the decoding of information units generated after the first information unit. The maximum allowed transmission duration can assist the first network element in transmitting the first information unit.

In addition, in the above step 410, the first network element can obtain the first information from the AF network element in two ways.

In a possible application scenario, if the backward reference range is relatively constant or the period of change is long, the AF network element can send the first information to the first network element through the control plane signaling transmission path.

Refer to Figure 7, which is a schematic diagram of an AF network element transmitting a backward reference range through a control plane transmission path. It should be understood that, based on the system architecture shown in Figure 3 and the interfaces between network elements, combined with Figure 7, those skilled in the art can clearly understand how the AF network element communicates to the first network element (UPF) through control plane signaling. network element or RAN network element) to send the first information. Moreover, each message shown in Figure 7 is known to those skilled in the art. For example, you can refer to the protocol TS 23.502v17.2.1, which will not be described in detail here.

In another possible application scenario, the backward reference range changes frequently, and the AF network element can send the first information to the first network element through "transmission along the road".

In this transmission method, the application layer of the AF network element carries the first information of the PDU set (backward reference range or time parameter) in each PDU set. Specifically, a fixed number of bits can be used to specify the backward reference range or time parameter of the PDU set at a specified position in the header of the first data packet of the PDU set. Alternatively, a fixed number of bits can be used to specify the backward reference range or time parameter of the PDU set at a designated position in the header of each data packet in the PDU set. When the PDU set reaches the UPF network element, the UPF network element detects the packet header and can learn the backward reference range or time parameter of the PDU set. Or, as shown above, if the caching operation of the information unit is performed by the RAN network element, after the UPF network element obtains the backward reference range or time parameter carried by the PDU set, the UPF network element will The range or time parameters are taken out and placed in the designated position of the tunnel transmission protocol (GPRS tunnel protocol for user plane, GTP-U) header of the user plane. Among them, GPRS stands for general packet radio services (general packet radio services) to facilitate the RAN network. Yuan obtains the backward reference range or time parameter of the PDU set.

Referring to Figure 8, Figure 8 is a schematic diagram of the backward reference range of each information unit transmitted along the path. Figure 8 shows an example of the backward reference range of three PDU sets transmitted along the road. The three PDU sets are PDU set#0, PDU set#1 and PDU set#2 as shown in Figure 8. The 3 The values of the backward reference ranges carried by each PDU set are M ₀ =1, M ₁ =2, and M ₂ =4.

In the above embodiment, the information unit generated by the AF network element is encoded, passes through the UPF network element and the RAN network element, and is finally sent to the UE. Each information unit in the buffer area of the AF network element corresponds to one of the following three types of reference marks: "used for long term reference", "used for short term reference" or "unused for reference". The first network element receives an information unit from the AF network element, and also includes a reference mark for receiving the information unit. The AF network element also needs to notify the first network element of the update of the reference mark of the information unit on the encoding side. The first network element also modifies the reference mark of the information unit accordingly and modifies the backward reference range of the information unit to The backward reference range corresponding to the updated reference mark. For example, an information unit whose reference mark is "used for long term reference" may have its reference mark modified to "used for short term reference" or "unused for reference", and the backward reference range of this information unit is correspondingly changed to "used for short term reference" or "unused for reference". The value of the backward reference range corresponding to "used for long term reference" is modified to the value of the backward reference range corresponding to "used for short term reference" or the value of the backward reference range corresponding to "unused for reference". If the application layer of the AF network element uniformly marks the pictures in the buffer area as "unused for reference", it is necessary to modify the values of the backward reference range of all pictures before changing the reference mark to 0, and notify the change The first network element. For another example, in the sliding window mode shown in Figure 2, the backward reference range of the information unit marked "used for short term reference" is the range of the information unit "used for short term reference" currently stored in the buffer area. The number is the number of information units that remove "used for long term reference" from MaxDpbFrames. In the adaptive reference picture marking mode shown in Figure 3, the initial backward reference range of an information unit whose reference mark is "used for short term reference" is the same as the backward reference range in the sliding window mode in Figure 2. When the application layer of the AF network element updates the reference tag of an information unit "used for short term reference" to "used for long term reference" or "unused for reference", the backward reference range corresponding to the information unit also changes accordingly. At this time, the AF network element needs to provide the updated reference mark and the updated backward reference range of the information unit to the first network element.

Specifically, the first network element receives the first change signaling and the second information from the AF network element.

The first change signaling is used to indicate that the reference mark of the first information unit is changed from the first reference mark to the second reference mark, and the first reference mark and the second reference mark are different. The first change signaling carries the identity of the first information unit. The second information is used to indicate the backward reference range corresponding to the second reference mark.

Optionally, the first change signaling can be transmitted from the AF network element to the first network element through control plane signaling, such as the transmission path in Figure 7.

Optionally, as another alternative implementation, the AF network element can also use "channel associated signaling" to provide the first network element with the updated reference mark of the first information unit and the updated reference mark. The corresponding backward reference range, where the channel associated signaling also needs to include the identifier of the first information unit. The information carried in the first change signaling (specifically, the second reference mark and the identification of the first information unit) may also be transmitted from the AF network element to the first network element through the "transmission along the way" transmission path.

The following is an example of the specific application of the backward reference range provided in this application with reference to Figures 9 and 10.

Example 1

The first network element is a UPF network element, and the UPF network element caches the received information unit.

Referring to Figure 9, Figure 9 is an example of an image processing method provided by this application.

501. The UPF network element receives the information unit sequence from the AF network element.

It should be understood that the information unit sequence refers to a series of information units generated by the AF network element (or considered as the encoding end). The following uses any information unit (for example, the first information unit) in the information unit sequence as an example to illustrate the application of the backward reference range proposed in this application.

The UPF network element sends the received information unit sequence to the RAN network element, and then the RAN network element sends the received information unit sequence to the UE.

In addition, the UPF network element caches the received information units according to certain caching rules. Each information unit in the buffer corresponds to a reference tag.

502. The UPF network element receives the first information from the AF network element.

503. The UPF network element obtains the backward reference range or time parameter of the first information unit according to the first information.

504. The UPF network element receives the first change signaling and the second information from the AF network element.

505. The UPF network element updates the reference mark and backward reference range of the first information unit according to the first change signaling and the second information.

Optionally, if the UE fails to receive the first information unit successfully, the RAN network element sends feedback information to the UPF network element, and the feedback information is used to indicate that the RAN network element fails to send the first information unit to the UE.

506. The UPF network element receives feedback information from the RAN network element.

One possible situation is step 507.

507. The UPF network element determines that the first information unit is still stored in the cache area, and the P third information units generated are encoded with reference to the first information unit.

At this time, the UPF network element executes step 508.

508. The UPF network element repeatedly sends the first information unit and P to the RAN network element within the maximum allowed transmission duration of the first information unit. a third information unit.

The RAN network element sends the first information unit and P third information units to the UE. It can be understood that here, the RAN network element actually retransmits the first information unit to the UE. In addition, several later generated information units encoded with reference to the first information unit are also sent to the UE.

Another possible situation after the UPF network element receives the feedback information from the RAN network element is step 509.

509. The UPF network element determines that the first information unit has been deleted from the cache area. If the P third information units generated after the determination refer to the first information unit for encoding, the UPF network element deletes the P third information units from the cache area. The third information unit.

For each of the above steps, please refer to the detailed description of the relevant content above. In order to avoid redundancy, details will not be described in Figure 9. In addition, the steps shown by dotted lines in Figure 9 are represented as optional steps.

It can be seen that in this example, after obtaining the first information of the first information unit, the UPF network element can obtain at least one of the maximum storage duration and the maximum allowed transmission duration of the first information unit based on the first information. Caching or transmitting the first information unit according to the maximum storage time or the maximum allowed transmission time of the first information unit can avoid inefficient video transmission caused by the UPF network element randomly discarding information units (such as frames or slices), and can improve video transmission. performance.

Example 2

The first network element is a RAN network element, and the RAN network element caches the received information units.

Referring to Figure 10, Figure 10 is an example of an image processing method provided by this application.

601. The RAN network element receives the information unit sequence from the UPF network element. The information unit sequence is generated by the AF network element and sent to the UPF network element.

The RAN network element sends the received information unit sequence to the UE.

In addition, the RAN network element caches the received information units according to certain caching rules. Each information unit in the buffer corresponds to a reference tag.

602. The RAN network element receives the first information from the UPF network element. The first information is sent by the AF network element to the UPF network element.

603. The RAN network element obtains the backward reference range or time parameter of the first information unit according to the first information.

604. The RAN network element receives the first change signaling and the second information from the UPF network element. The first change signaling and the second information are sent by the UPF network element to the UPF network element.

605. The RAN network element updates the reference mark and backward reference range of the first information unit according to the first change signaling and the second information.

Optionally, if the UE fails to receive the first information unit successfully, the UE sends feedback information to the RAN network element, and the feedback information is used to indicate that the UE fails to receive the first information unit successfully.

606. The RAN network element receives the information that the first information unit was not successfully received as fed back by the UE.

Optionally, the UE can pass the hybrid automatic repeat request (HARQ) information through the media access control (MAC) layer, or through the packet data convergence protocol (PDCP) layer or The status report of the radio resource control (RRC) layer feeds back to the RAN network element the failure to receive the first information unit, which is not limited.

A possible situation is as shown in step 607.

607. The RAN network element determines that the first information unit is still stored in the cache area, and the P third information units generated later are encoded with reference to the first information unit.

As mentioned above, the RAN network element can determine whether the M information units generated after the first information unit refer to the first information unit for encoding based on the M pieces of third information provided by the UPF.

At this time, the UPF network element executes step 608.

608. The RAN network element sends the first information unit and P third information units to the UE within the maximum allowed transmission duration of the first information unit.

Another possible situation after the RAN network element receives the feedback information from the UE is step 609.

609. The RAN network element determines that the first information unit has been deleted from the cache area. If the P third information units generated after the determination refer to the first information unit for encoding, the RAN network element deletes the P third information units from the cache area. The third information unit.

The steps shown as dotted lines in Figure 10 represent optional steps.

It can be seen that in this example, after obtaining the first information of the first information unit, the RAN network element can obtain at least one of the maximum storage duration and the maximum allowed transmission duration of the first information unit based on the first information. According to the maximum storage time of the first information unit Or cache or transmit the first information unit according to the maximum allowed transmission time, which can avoid inefficient video transmission caused by random discarding of information units (such as frames or slices) by RAN network elements, and can improve video transmission performance.

The method for processing images provided by this application has been described in detail above. The device for processing images provided by this application will be introduced below.

Referring to Figure 11, Figure 11 is a schematic block diagram of an image processing device provided by this application. As shown in Figure 11, the device 1000 includes a processing unit 1100, a storage unit 1200 and a transceiver unit 1300.

Among them, the storage unit 1200 is used to store the received information unit;

The transceiver unit 1300 is used to obtain the first information from the application function network element, where,

The first information indicates a backward reference range of the first information unit of the image, the backward reference range is used to indicate a distance between an index of the second information unit and an index of the first information unit, and the third The information unit generated after the second information unit no longer refers to the first information unit; or,

The first information indicates the temporal parameter of the first information unit of the image, the temporal parameter is determined according to the backward reference range of the first information unit, and the backward reference range is used to indicate the second information unit. The distance between the index and the index of the first information unit, the information unit generated after the second information unit no longer refers to the first information unit;

The processing unit 1100 is configured to process the first information unit in the storage unit 1200 according to the first information.

Optionally, in one embodiment, if the first information indicates the backward reference range of the first information unit, the processing unit 1100 is further configured to:

The time parameter is determined based on the backward reference range.

Optionally, in one embodiment, the time parameter includes at least one of the maximum storage duration and the maximum allowed transmission duration of the first information unit, wherein,

The maximum storage duration is from the first time when the device receives the first information unit to the second time when the device receives M information units generated after the first information unit. The duration of the interval, M is the value of the interval indicated by the backward reference range, and M is an integer;

The maximum allowed transmission duration is from the first moment when the device receives the first information unit to the third moment when it is determined that the device will no longer send the first information unit to the terminal device. of duration.

Optionally, in one embodiment, each information unit stored in the storage unit 1200 corresponds to a reference mark, and the reference mark corresponding to each information unit is one of the following three types: for long-term reference , used for short-term reference and not used for reference, wherein the three types respectively correspond to different backward reference ranges.

Optionally, in one embodiment, the first information unit corresponds to a first reference mark;

The transceiver unit 1300 is further configured to obtain first change signaling and second information from the application function network element, where the first change signaling is used to indicate that the reference mark of the first information unit is determined by the The first reference mark is changed to a second reference mark, and the second information is used to indicate the backward reference range corresponding to the second reference mark, wherein the first change signaling carries the identity of the first information unit , the type of the first reference mark and the type of the second reference mark are different;

The processing unit 1100 is further configured to modify the reference mark of the first information unit to the second reference mark according to the first change signaling and the second information, and change the first information unit to the second reference mark. The backward reference range of the unit is modified to the backward reference range corresponding to the second reference mark.

Optionally, in one embodiment, the transceiver unit 1300 is used for:

Obtain the first information from the application function network element through control plane signaling; or,

A first set of PDUs is received from the application function network element, the first set of PDUs containing the first information of the first information unit.

Optionally, in one embodiment, the storage unit 1200 is configured to retain the first information unit in the cache within the maximum storage duration according to the first information.

Optionally, in one embodiment, the transceiver unit 1300 is configured to receive M information units generated after the first information unit;

The processing unit 1100 is configured to delete the first information unit if it is determined that none of the M information units is encoded with reference to the first information unit within the maximum storage time period.

Optionally, in one embodiment, the device is a RAN network element, and the transceiver unit 1300 is used for:

M pieces of third information are obtained from the UPF network element. The M pieces of third information correspond to the M pieces of information units one-to-one. Each of the M pieces of third information is used to determine the corresponding information. Whether the unit refers to the first information unit for encoding;

The processing unit 1100 is configured to determine, according to the M pieces of third information, that none of the M information units are encoded with reference to the first information unit.

Optionally, in one embodiment, the transceiver unit 1300 is also configured to receive the first information unit from the application function network element;

The storage unit 1200 is used to save the first information unit into the cache area of the first network element;

The transceiver unit 1300 is also used to send the first information unit to the terminal device;

The processing unit 1100 is also used to determine that the terminal device has not successfully received the first information unit;

And, the transceiver unit 1300 is also configured to repeatedly send the first information unit to the terminal device within the maximum transmission delay.

Optionally, in one embodiment, the processing unit 1100 is also used to:

According to the first information, it is determined that the terminal device still has not correctly received the first information unit when the maximum transmission duration is exceeded, and it is determined that the first information unit is no longer sent to the terminal device.

Optionally, in one embodiment, the device is a UPF network element;

The transceiver unit 1300 is configured to receive the first information unit from the application function network element;

The cache unit 1200 is used to save the first information unit into the cache area of the first network element;

The transceiver unit 1300 is also configured to receive feedback information from the RAN network element, where the feedback information is used to indicate that the RAN network element has not successfully sent the first information unit to the terminal device;

The processing unit 1100 is configured to determine that the first information unit is retained in the buffer area of the first network element, and that one or more third information units generated after the first information unit refer to the first information unit. An information unit is encoded;

And, the transceiver unit 1300 is configured to send the first information unit and the one or more third information units to the RAN network element.

Optionally, in one embodiment, the processing unit 1100 is configured to determine that the terminal device has not successfully received the first information unit, and the first information unit has been retrieved from the buffer area of the first network element. been deleted;

The cache unit 1200 is configured to delete one or more third information units encoded with reference to the first information unit from the cache.

Optionally, in one embodiment, the device is a RAN network element or a UPF network element.

In various embodiments in which the device 1000 corresponds to a UPF network element, the processing unit 1100 is configured to perform processing and/or operations implemented internally by the UPF network element in addition to actions of sending and receiving. The transceiver unit 1300 is used to perform the receiving or sending action of the UPF network element.

For example, in FIG. 4 , the transceiver unit 1300 is configured to perform the action of receiving the first information in step 410 ; the processing unit 1100 is configured to perform step 420 . Optionally, the storage unit 1200 is used to store the received information unit sequence from the application function network element (not shown in Figure 4).

For example, in Figure 9, the transceiver unit 1300 is used to perform the actions received in step 501, step 502, step 504, and step 506; it is also used to perform the action of sending the information unit sequence to the RAN network element, and the action sent in step 508. Action: The processing unit 1100 is used to execute step 503, step 505, step 507 and step 509. In addition, the storage unit 1300 is used to cache information units received from the AF network element.

In various embodiments in which the device 1000 corresponds to a RAN network element, the processing unit 1100 is configured to perform processing and/or operations implemented internally by the RAN network element in addition to actions of sending and receiving. The transceiver unit 1300 is used to perform receiving or transmitting actions of the RAN network element.

For example, in Figure 10, the transceiver unit 1300 is used to perform the actions received in step 601, step 602, step 604 and step 606; it is also used to perform the action of sending the information unit sequence to the UE, and the action of sending in step 608; The processing unit 1100 is used to execute step 603, step 605, step 607 and step 609. In addition, the storage unit 1300 is used to cache information units received from the UPF network element.

Referring to Figure 12, Figure 12 is a schematic structural diagram of an image processing device provided by this application. As shown in FIG. 12 , the device 10 includes: one or more processors 11 , one or more memories 12 and one or more communication interfaces 13 . The processor 11 is used to control the communication interface 13 to send and receive signals, the memory 12 is used to store a computer program, and the processor 11 is used to call and run the computer program from the memory 12, This causes the device 10 to perform the processing performed by the first network element in each method embodiment of the present application.

For example, the processor 11 may have the function of the processing unit 1100 shown in FIG. 11 , the memory 12 may have the function of the storage unit 1200 , and the communication interface 13 may have the function of the transceiver unit 1300 . Specifically, the processor 11 may be used to perform processing or operations performed internally by the device, the memory may be used to perform operations of caching and deleting information units from the cache, and the communication interface 13 is used to perform operations of sending and/or receiving by the device. .

In one implementation, the device 10 may be the first network element in the method embodiment. In this implementation, the communication interface 13 may be a transceiver of the first network element. A transceiver may include a receiver and/or a transmitter. Alternatively, the processor 11 may be a baseband device of the first network element, and the communication interface 13 may be a radio frequency device.

In another implementation, the device 10 may be a chip (or chip system) installed in the first network element. In this implementation, communication interface 13 may be an interface circuit or an input/output interface.

Among them, the dotted box behind the device (for example, processor, memory or communication interface) in Figure 12 indicates that there can be more than one device.

Optionally, the memory and the processor in the above device embodiments may be physically independent units, or the memory may be integrated with the processor, which is not limited herein.

In addition, this application also provides a computer-readable storage medium. Computer instructions are stored in the computer-readable storage medium. When the computer instructions are run on the computer, each method embodiment of the application is executed by the first network element. operations and/or processing are performed.

In addition, this application also provides a computer program product. The computer program product includes computer program code or instructions. When the computer program code or instructions are run on the computer, the operations performed by the first network element in each method embodiment of the application are performed. and/or processing is performed.

In addition, the present application also provides a chip. The chip includes a processor. A memory used to store a computer program is provided independently of the chip. The processor is used to execute the computer program stored in the memory, so that a device equipped with the chip executes Operations and/or processes performed by the first network element in any method embodiment.

Further, the chip may also include a communication interface. The communication interface may be an input/output interface, or an interface circuit, etc. Further, the chip may further include the memory.

In addition, this application also provides a communication device (for example, it can be a chip or a chip system), including a memory, a processor and a communication interface. The communication interface is used to receive the first information and send the first information to the processing processor; the processor is configured to process the first information unit cached in the memory according to the first information.

In addition, the present application also provides a communication device, including at least one processor, the at least one processor is coupled to at least one memory, and the at least one processor is used to execute a computer program or instructions stored in the at least one memory, The communication device is caused to perform the operation and/or processing performed by the first network element in any method embodiment.

In addition, this application also provides a wireless communication system, including UPF network elements and RAN network elements in various embodiments of this application. Optionally, it also includes AF network elements. Optionally, terminal equipment may also be included.

It should be noted that "and/or" describes the relationship between associated objects, indicating that there can be three relationships. For example, A and/or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. three conditions. Among them, the number of A is not limited, it can be one or more than one, and the number of B is not limited, it can be one or more than one.

In the above embodiments, terms such as first, second, third, etc. are only for convenience of distinguishing different objects and should not constitute any limitation on this application. For example, distinguish different information, different information units, etc.

The processor mentioned in the embodiments of this application can be a central processing unit (CPU), or other general-purpose processor, digital signal processor (DSP), application specific integrated circuit (application specific integrated circuit) circuit, ASIC), off-the-shelf programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

The memory in the embodiment of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, non-volatile memory can be read-only memory (ROM), programmable ROM (PROM), erasable programmable read-only memory (erasable PROM, EPROM), electrically removable memory. Erase programmable read-only memory (electrically EPROM, EEPROM) or flash memory. Volatile memory may be random access memory (RAM), which is used as an external cache. By way of illustration, but not limitation, many forms of RAM are available, For example, static random access memory (static RAM, SRAM), dynamic random access memory (dynamic RAM, DRAM), synchronous dynamic random access memory (synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (double data rate SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (enhanced SDRAM, ESDRAM), synchronous link dynamic random access memory (synchlink DRAM, SLDRAM) and direct memory bus random access memory (direct rambus RAM, DRRAM) ). It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each specific application, but such implementations should not be considered beyond the scope of this application.

Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, devices and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be described again here.

In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or can be integrated into another system, or some features can be ignored, or not implemented. On the other hand, the coupling or direct coupling or communication connection between each other shown or discussed may be through some interfaces, and the indirect coupling or communication connection of the devices or units may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or they may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application can be integrated into one processing unit, each unit can exist physically alone, or two or more units can be integrated into one unit.

If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the existing technology or the part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program code. .

The above are only specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application. should be covered by the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims

A method for processing images, characterized by including:

The first network element obtains the first information from the application function network element, where,

The first information indicates a backward reference range of the first information unit of the image, and the backward reference range is used to indicate a distance between an index of the second information unit and an index of the first information unit. , the information unit generated after the second information unit no longer refers to the first information unit; or,

The first information indicates the time parameter of the first information unit of the image, the time parameter is determined according to the backward reference range of the first information unit, and the backward reference range is used to indicate the second information The distance between the index of the unit and the index of the first information unit, the information unit generated after the second information unit no longer refers to the first information unit;

The first network element processes the first information unit in the buffer area of the first network element according to the first information.
The method of claim 1, wherein if the first information indicates the backward reference range of the first information unit, the method further includes:

The first network element determines the time parameter according to the backward reference range.
The method according to claim 1 or 2, characterized in that the time parameter includes at least one of the maximum storage duration and the maximum allowed transmission duration of the first information unit, wherein,

The maximum storage duration is from the first moment when the first network element receives the first information unit to when the first network element receives M information units generated after the first information unit. The duration until the second moment, where M is the value of the spacing indicated by the backward reference range, and M is an integer;

The maximum allowed transmission duration is from the first moment when the first network element receives the first information unit to when it is determined that the first network element will no longer send the first information unit to the terminal device. The duration until the third moment.
The method according to any one of claims 1 to 3, characterized in that each information unit in the cache area of the first network element corresponds to a reference mark, and the reference mark corresponding to each information unit It is one of the following three types: used for long-term reference, used for short-term reference and not used for reference, wherein the three types respectively correspond to different backward reference ranges.
The method of claim 4, wherein the first information unit corresponds to a first reference mark; the method further includes:

The first network element obtains first change signaling and second information from the application function network element. The first change signaling is used to indicate that the reference mark of the first information unit is changed from the first reference mark. Changed to a second reference mark, the second information is used to indicate the backward reference range corresponding to the second reference mark, wherein the first change signaling carries the identity of the first information unit, and the third The type of one reference mark is different from the type of the second reference mark;

The first network element modifies the reference mark of the first information unit to the second reference mark according to the first change signaling and the second information, and changes the suffix of the first information unit to the second reference mark. The forward reference range is modified to the backward reference range corresponding to the second reference mark.
The method according to any one of claims 1 to 5, characterized in that the first network element obtains the first information from the application function network element, including:

The first network element obtains the first information from the application function network element through control plane signaling; or,

The first network element receives a first protocol data unit PDU set from the application function network element, and the first PDU set includes the first information of the first information unit.
The method according to any one of claims 3 to 6, characterized in that the first network element caches the first information unit in the cache area of the first network element based on the first information. Processing includes:

The first network element retains the first information unit in the cache within the maximum storage duration according to the first information.
The method of claim 7, further comprising:

M information units generated after the first network element receives the first information unit;

Within the maximum storage time period, if the first network element determines that none of the M information units are encoded with reference to the first information unit, the first network element deletes the first information unit.
The method according to claim 8, characterized in that:

The first network element is a RAN network element;

The first network element determines that none of the M information units are encoded with reference to the first information unit, including:

The first network element obtains M pieces of third information from the user plane function UPF network element. The M pieces of third information correspond to the M pieces of information units one-to-one. Each of the M pieces of third information The third information is used to determine whether the corresponding information unit refers to the first information unit for encoding;

The first network element determines, based on the M pieces of third information, that none of the M information units are encoded with reference to the first information unit.
The method of claim 9, further comprising:

The first network element receives the first information unit from the application function network element and saves the first information unit in a cache area of the first network element;

The first network element sends the first information unit to the terminal device;

Wherein, the first network element processes the first information unit in the buffer area of the first network element according to the first information, including:

If the terminal device fails to receive the first information unit successfully, the first network element repeatedly sends the first information unit to the terminal device within the maximum transmission delay according to the first information. .
The method according to claim 10, characterized in that the first network element processes the first information unit in the buffer area of the first network element according to the first information, and further includes:

The first network element determines, based on the first information, that when the maximum transmission duration is exceeded, the terminal device still does not receive the first information unit correctly, and the first network element no longer transmits data to the terminal device. Send the first information unit.
The method according to any one of claims 3 to 8, characterized in that the first network element is a UPF network element, and the method further includes:

The first network element receives the first information unit from the application function network element and saves the first information unit in a cache area of the first network element;

The first network element sends the first information unit to the RAN network element;

The first network element receives feedback information from the RAN network element, where the feedback information is used to indicate that the RAN network element has not successfully sent the first information unit to the terminal device;

If the first information unit is retained in the buffer area of the first network element, and one or more third information units generated after the first information unit is determined to be encoded with reference to the first information unit, then The first network element sends the first information unit and the one or more third information units to the RAN network element.
The method according to any one of claims 3 to 12, characterized in that the method further includes:

If the terminal device does not successfully receive the first information unit and the first network element determines that the first information unit has been deleted from the cache area of the first network element, the first network element removes the first information unit from the first network element. One or more third information units encoded with reference to the first information unit are deleted from the cache.
The method according to any one of claims 1 to 8, 13, characterized in that the first network element is a RAN network element or a user plane function UPF network element.
A device for processing images, characterized in that it includes at least one processor, the at least one processor is coupled to at least one memory, and the at least one processor is used to execute computer programs or instructions stored in the at least one memory, So that the device performs the method according to any one of claims 1-14.
A chip, characterized by comprising a memory, a processor and a communication interface, the communication interface being used to receive first information and transmit the received first information to the processor, the processor according to the First information: process the first information unit stored in the memory to perform the method according to any one of claims 1-14.
A computer-readable storage medium, characterized in that computer instructions are stored in the computer-readable storage medium. When the computer instructions are run on a computer, the method as described in any one of claims 1-14 is performed. accomplish.
A computer program product, characterized in that the computer program product includes computer program code, and when the computer program code is run on a computer, the method according to any one of claims 1-14 is implemented.
A wireless communication system, characterized by comprising the device for processing images as claimed in claim 15.