CN116367210A

CN116367210A - Quality of experience reporting method, terminal and network side equipment for augmented reality XR service

Info

Publication number: CN116367210A
Application number: CN202111630409.8A
Authority: CN
Inventors: 姜炜
Original assignee: Vivo Mobile Communication Co Ltd
Current assignee: Vivo Mobile Communication Co Ltd
Priority date: 2021-12-28
Filing date: 2021-12-28
Publication date: 2023-06-30

Abstract

The application discloses a method, a terminal and network side equipment for reporting the experience quality of an extended reality XR service, which belong to the technical field of communication. The terminal collects and measures the experience quality information of XR service; the quality of experience information includes at least one of: application layer related quality of experience QoE, wireless related QoE and XR user quality of experience; the XR user experience quality is obtained based on the QoE related to the application layer and the QoE related to wireless; and the terminal reports the experience quality information to the first equipment.

Description

Quality of experience reporting method, terminal and network side equipment for augmented reality XR service

Technical Field

The application belongs to the technical field of communication, and particularly relates to a quality of experience reporting method, a terminal and network side equipment for an extended reality XR service.

Background

Extended reality (XR) refers to all real and virtual combined environments and human-machine interactions generated by computer technology and wearable devices. It includes representative forms of augmented Reality (Augmented Reality, AR), mixed Reality (MR), virtual Reality (VR), and the like, as well as interpolation regions therebetween. The level of the virtual world is from a partially sensory input to a fully immersive virtual reality. One key aspect of XR is the expansion of human experience, especially experience related to sense of presence (represented by VR) and cognition learning (represented by AR).

Due to the low latency, high reliability, high rate of XR traffic, and the presence of multi-stream, jitter (jitter) characteristics, and the diversity of XR devices. The quality of experience (Quality of Experience, qoE) indicators currently applied to other business fields do not reflect the quality of user experience of XR users well.

Disclosure of Invention

The embodiment of the application provides a quality of experience reporting method, a terminal and network side equipment for an extended reality XR service, which can solve the problem that the existing QoE index applied to other service fields can not well reflect the user quality of experience of XR users.

In a first aspect, a method for reporting quality of experience of an augmented reality XR service is provided, and the method is applied to a terminal, and includes:

the terminal collects and measures the experience quality information of XR service; the quality of experience information includes at least one of: application layer related quality of experience QoE, wireless related QoE and XR user quality of experience; the XR user experience quality is obtained based on the QoE related to the application layer and the QoE related to wireless;

and the terminal reports the experience quality information to the first equipment.

In a second aspect, a quality of experience reporting method for an augmented reality XR service is provided, and the method is applied to an access network device, and includes:

The access network equipment collects and measures the experience quality information of the XR service; the quality of experience information includes at least one of: application layer related quality of experience QoE, wireless related QoE and XR user quality of experience; the XR user experience quality is obtained based on the QoE related to the application layer and the QoE related to wireless; and/or, the access network equipment receives the experience quality information reported by the terminal;

the access network equipment performs target processing on the experience quality information; and/or the access network equipment reports the experience quality information to the second equipment.

In a third aspect, a method for reporting quality of experience of an augmented reality XR service is provided, and the method is applied to a core network device, and includes:

the core network equipment receives the reported experience quality information of the XR service; the quality of experience information includes at least one of: application layer related quality of experience QoE, wireless related QoE and XR user quality of experience; the XR user experience quality is obtained based on the QoE related to the application layer and the QoE related to wireless;

and the core network equipment performs target processing on the experience quality information of the XR service.

In a fourth aspect, a quality of experience reporting apparatus for an augmented reality XR service is provided, the apparatus being applied to a terminal, and the apparatus comprising:

The processing module is used for collecting and measuring the experience quality information of the XR service; the quality of experience information includes at least one of: application layer related quality of experience QoE, wireless related QoE and XR user quality of experience; the XR user experience quality is obtained based on the QoE related to the application layer and the QoE related to wireless;

and the sending module is used for reporting the experience quality information to the first equipment.

In a fifth aspect, a quality of experience reporting apparatus for an augmented reality XR service is provided, where the apparatus is applied to an access network device, and the apparatus includes:

the processing module is used for the access network equipment to collect and measure the experience quality information of the XR service; the quality of experience information includes at least one of: application layer related quality of experience QoE, wireless related QoE and XR user quality of experience; the XR user experience quality is obtained based on the QoE related to the application layer and the QoE related to wireless; and/or the number of the groups of groups,

the receiving module is used for receiving the experience quality information reported by the terminal;

the processing module is further used for performing target processing on the quality of experience information; and/or the number of the groups of groups,

and the sending module is used for reporting the experienced quality information to the core network equipment.

In a sixth aspect, a quality of experience reporting apparatus for an augmented reality XR service is provided, where the apparatus is applied to a core network device, and the apparatus includes:

the receiving module is used for receiving the reported experience quality information of the XR service; the quality of experience information includes at least one of: application layer related quality of experience QoE, wireless related QoE and XR user quality of experience; the XR user experience quality is obtained based on the QoE related to the application layer and the QoE related to wireless;

and the processing module is used for carrying out target processing on the experience quality information of the XR service.

In a seventh aspect, there is provided a terminal comprising a processor and a memory storing a program or instructions executable on the processor, which when executed by the processor, implement the steps of the method as described in the first aspect.

An eighth aspect provides a terminal, including a processor and a communication interface, where the processor is configured to collect and measure quality of experience information of XR service; the quality of experience information includes at least one of: application layer related quality of experience QoE, wireless related QoE and XR user quality of experience; the XR user experience quality is obtained based on the QoE related to the application layer and the QoE related to wireless; the communication interface is used for reporting the experience quality information to the first equipment.

In a ninth aspect, there is provided an access network device comprising a processor and a memory storing a program or instructions executable on the processor, which when executed by the processor, implement the steps of the method as described in the second aspect.

In a tenth aspect, an access network device is provided, including a processor and a communication interface, where the processor is configured to collect and measure quality of experience information of XR service; the quality of experience information includes at least one of: application layer related quality of experience QoE, wireless related QoE and XR user quality of experience; the XR user experience quality is obtained based on the QoE related to the application layer and the QoE related to wireless; and/or the communication interface is used for receiving the experience quality information reported by the terminal;

the processor is further configured to perform target processing on the quality of experience information, and/or the communication interface is further configured to report the quality of experience information to a second device.

In an eleventh aspect, there is provided a core network device comprising a processor and a memory storing a program or instructions executable on the processor, which when executed by the processor, implement the steps of the method according to the third aspect.

A twelfth aspect provides a core network device, including a processor and a communication interface, where the communication interface is configured to receive quality of experience information of XR service; the quality of experience information includes at least one of: application layer related quality of experience QoE, wireless related QoE and XR user quality of experience; the XR user experience quality is obtained based on the QoE related to the application layer and the QoE related to the wireless, and the processor is used for carrying out target processing on the experience quality information of the XR service.

In a thirteenth aspect, there is provided a communication system comprising: a terminal, an access network device and a core network device, where the terminal is configured to perform the steps of the quality of experience reporting method of the XR service according to the first aspect, the access network device is configured to perform the steps of the quality of experience reporting method of the XR service according to the second aspect, and the core network device is configured to perform the steps of the quality of experience reporting method of the XR service according to the third aspect.

In a fourteenth aspect, there is provided a readable storage medium having stored thereon a program or instructions which when executed by a processor, perform the steps of the method according to the first aspect, or perform the steps of the method according to the second aspect, or perform the steps of the method according to the third aspect.

In a fifteenth aspect, there is provided a chip comprising a processor and a communication interface, the communication interface and the processor being coupled, the processor being for running a program or instructions to implement the method as described in the first aspect, or to implement the method as described in the second aspect, or to implement the method as described in the third aspect.

In a sixteenth aspect, there is provided a computer program/program product stored in a storage medium, the computer program/program product being executable by at least one processor to perform the steps of the method as described in the first aspect, or to perform the steps of the method as described in the second aspect, or to perform the steps of the method as described in the third aspect.

In the embodiment of the application, the terminal collects and measures the experience quality information of XR service; the quality of experience information includes at least one of: application layer related quality of experience QoE, wireless related QoE and XR user quality of experience; the XR user experience quality is obtained based on QoE related to an application layer and QoE related to wireless; the experience quality information is strongly related to the XR service, so that the user experience quality of an XR user can be better reflected, and the user experience can be improved by reporting the experience quality information strongly related to the XR service and adaptively processing the XR service.

Drawings

Fig. 1 is a block diagram of a wireless communication system to which embodiments of the present application are applicable;

fig. 2 is one of flow diagrams of a quality of experience reporting method of an XR service provided in an embodiment of the present application;

fig. 3 is one of the interactive flow diagrams of the quality of experience reporting method of the XR service provided in the embodiment of the present application;

fig. 4 is a second schematic interaction flow diagram of a quality of experience reporting method of XR service provided in the embodiment of the present application;

fig. 5 is a third schematic interaction flow diagram of a quality of experience reporting method of XR service provided in the embodiment of the present application;

fig. 6 is a second flow chart of a quality of experience reporting method of XR service provided in the embodiment of the present application;

fig. 7 is a fourth schematic diagram of an interaction flow of the quality of experience reporting method of the XR service provided in the embodiment of the present application;

fig. 8 is a third flow chart of a quality of experience reporting method of XR service provided in the embodiment of the present application;

fig. 9 is one of structural schematic diagrams of a quality of experience reporting device for XR service provided in the embodiment of the present application;

fig. 10 is a second schematic structural diagram of an XR service quality of experience reporting device according to the embodiment of the present application;

fig. 11 is a third schematic structural diagram of an XR service quality of experience reporting device according to the embodiment of the present application;

Fig. 12 is a schematic structural diagram of a communication device provided in an embodiment of the present application;

fig. 13 is a schematic hardware structure of a terminal according to an embodiment of the present application;

fig. 14 is a schematic structural diagram of an access network device according to an embodiment of the present application;

fig. 15 is a schematic structural diagram of a core network device according to an embodiment of the present application.

Detailed Description

Technical solutions in the embodiments of the present application will be clearly described below with reference to the drawings in the embodiments of the present application, and it is apparent that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application are within the scope of the protection of the present application.

The terms first, second and the like in the description and in the claims, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the terms so used are interchangeable under appropriate circumstances such that the embodiments of the application are capable of operation in sequences other than those illustrated or otherwise described herein, and that the terms "first" and "second" are generally intended to be used in a generic sense and not to limit the number of objects, for example, the first object may be one or more. Furthermore, in the description and claims, "and/or" means at least one of the connected objects, and the character "/" generally means a relationship in which the associated object is an "or" before and after.

It is noted that the techniques described in embodiments of the present application are not limited to long term evolution (Long Term Evolution, LTE)/LTE implementationsThe LTE-Advanced, LTE-a system may also be used in other wireless communication systems, such as code division multiple access (Code Division Multiple Access, CDMA), time division multiple access (Time Division Multiple Access, TDMA), frequency division multiple access (Frequency Division Multiple Access, FDMA), orthogonal frequency division multiple access (Orthogonal Frequency Division Multiple Access, OFDMA), single-carrier frequency division multiple access (Single-carrier Frequency Division Multiple Access, SC-FDMA), and other systems. The terms "system" and "network" in embodiments of the present application are often used interchangeably, and the techniques described may be used for both the above-mentioned systems and radio technologies, as well as other systems and radio technologies. The following description describes a New air interface (NR) system for purposes of example and uses NR terminology in much of the description that follows, but these techniques are also applicable to applications other than NR system applications, such as generation 6 (6) ^th Generation, 6G) communication system.

Fig. 1 shows a block diagram of a wireless communication system to which embodiments of the present application are applicable. The wireless communication system includes a terminal 11 and a network device 12. The terminal 11 may be a mobile phone, a tablet (Tablet Personal Computer), a Laptop (Laptop Computer) or a terminal-side Device called a notebook, a personal digital assistant (Personal Digital Assistant, PDA), a palm top, a netbook, an ultra-mobile personal Computer (ultra-mobile personal Computer, UMPC), a mobile internet appliance (Mobile Internet Device, MID), an augmented reality (augmented reality, AR)/Virtual Reality (VR) Device, a robot, a Wearable Device (weather Device), a vehicle-mounted Device (VUE), a pedestrian terminal (PUE), a smart home (home Device with a wireless communication function, such as a refrigerator, a television, a washing machine, or a furniture), a game machine, a personal Computer (personal Computer, PC), a teller machine, or a self-service machine, and the Wearable Device includes: intelligent wrist-watch, intelligent bracelet, intelligent earphone, intelligent glasses, intelligent ornament (intelligent bracelet, intelligent ring, intelligent necklace, intelligent anklet, intelligent foot chain etc.), intelligent wrist strap, intelligent clothing etc.. Note that, the specific type of the terminal 11 is not limited in the embodiment of the present application. The network-side device 12 may comprise an access network device or a core network device, wherein the access network device 12 may also be referred to as a radio access network device, a radio access network (Radio Access Network, RAN), a radio access network function or a radio access network element. Access network device 12 may include a base station, a WLAN access point, a WiFi node, or the like, which may be referred to as a node B, an evolved node B (eNB), an access point, a base transceiver station (Base Transceiver Station, BTS), a radio base station, a radio transceiver, a basic service set (Basic Service Set, BSS), an extended service set (Extended Service Set, ESS), a home node B, a home evolved node B, a transmission and reception point (Transmitting Receiving Point, TRP), or some other suitable terminology in the art, and the base station is not limited to a particular technical vocabulary so long as the same technical effect is achieved, and it should be noted that in the embodiments of the present application, only a base station in an NR system is described as an example, and the specific type of the base station is not limited.

XR refers to all real and virtual combined environments and human-machine interactions produced by computer technology and wearable devices. It includes representative forms of AR, MR, VR, etc., as well as interpolation regions therebetween. The level of the virtual world is from a partially sensory input to a fully immersive virtual reality. One key aspect of XR is the expansion of human experience, especially experience related to sense of presence (represented by VR) and cognition learning (represented by AR).

For VR service, the uplink is mainly transmitted by denser small data packets, and the small data packets can bear information such as gestures and control and serve as input and reference of downlink presentation data. The downlink mainly transmits multimedia data such as video and audio, and provides immersive feeling for users through timely receiving and presenting the multimedia data. Taking downstream video data as an example, the data packets arrive periodically or quasi-periodically, the data rate can reach tens or even hundreds of Mbps, the typical value of the Frame rate (FPS) is 60 or 120, the interval between adjacent data packets is approximately 1/FPS seconds, these data typically need to be successfully transmitted within 10ms over the air interface, and the transmission success rate is required to be not less than 99% or even 99.9%.

For AR service, uplink may transmit multimedia data such as video and audio in addition to the above dense small data packet transmission, and its service characteristics are similar to those of downlink, and the data rate is usually relatively low, for example, at most several tens Mbps, and the time limit of air interface transmission may be widened, for example, it is generally required to perform successful transmission within 60 ms; the downlink data transmission characteristics are substantially consistent with VR traffic.

XR devices have different physical dimensions. These physical dimensions may vary in terms of processing power, communication type, and power consumption.

QoE refers to the overall subjective perception of quality and performance (including availability and availability, etc.) of a device, network and system, application or service by a user, i.e., as defined by the comfort of the service application. The QoE scoring operator can comprehensively evaluate the quality and performance of video service by users, and optimize the network. The ultimate goal of operators is to offer a wide variety of video services and convenience to end users, so the needs of the users are paramount. However, each user's assessment of service and video quality is based on his own specific experience and is subjective. In this case, the concept of "quality of user experience (QoE)" may be used to understand the corresponding quality of service and network performance.

QoE measurement collection (measurement collection) of streaming media services is standardized in UTRAN and E-UTRAN, and currently 3GPP is working on standardized more adaptive 5G QoE management schemes, so as to implement network intelligent optimization to meet user experience of various services.

The terminal reports QoE visible to the radio access network (Radio Access Network, RAN):

the RAN may not understand or utilize the traditional QoE metrics because they are assembled by operations administration maintenance (Operation Administrator Maintenance, OAM), sent within the container, intended to be handled by the measurement collection entity in the network. If the RAN needs to utilize the QoE concept, it may be required that QoE information should be visible to the RAN. The QoE information visible to the RAN is simplified QoE information abstracted from the QoE metrics of the terminals, which can be used by the RAN for various types of optimizations. The RAN-visible QoE metrics may be derived from single SA 4-defined QoE metrics deemed useful for the RAN, such as buffer levels. The RAN-visible QoE metrics may be, for example, reduced values derived from a single useful SA 4-defined QoE metric or a combination of these values.

In order for the network to further evaluate and improve QoE, the RAN may also trigger radio related measurements for a certain terminal according to the QoE measurement configuration received from the OAM. To trigger the measurement, existing mechanisms, such as the minimum drive test (Minimization of Drive-Tests) MDT procedure, may be used. The collection of radio related measurements should be done by existing methods, such as MDT, if needed.

The wireless related QoE measurements will be reported for all supported service types, including MDT like measurements and possibly additional measurements related to the radio interface. If NR QoE management requires new radio related measurements (relative to what is currently specified in MDT), these additional radio related QoE measurements will be specified as part of the MDT measurements. QoE measurements related to an application are only collected when an application session is in progress. If these wireless related measurements are used to assist in applying related QoE measurements, it would be beneficial and efficient if measurement collection and reporting could be started simultaneously. If configured together, e.g., using the same tracking reference and time alignment, e.g., based on a time stamp, correlation of the results may be accomplished through a background process. In addition to radio related measurements, radio related information may also be reported. The wireless-related information may be reported even if the wireless-related measurement is not triggered by radio.

Due to the low latency, high reliability, high rate of XR traffic, and the presence of multi-stream, jitter, etc. characteristics, as well as the diversity of XR equipment. The QoE index or the radio-side related index defined by the current 3GPP may not reflect the quality of user experience of the XR user well. In the embodiment of the application, the QoE measurement information related to the application layer and the wireless related measurement information are collected, measured and reported to the base station side, even the application layer of the core network or the terminal, so that adaptive scheduling control, transmission adjustment and control, error management or source coding and the like are performed for the XR service, the system performance is improved, and the user experience is improved.

The following describes in detail, by means of some embodiments and application scenarios thereof, a quality of experience reporting method for XR service provided in the embodiments of the present application with reference to the accompanying drawings.

Fig. 2 is a schematic flow chart of a quality of experience reporting method of an XR service according to an embodiment of the present application. As shown in fig. 2, the method provided in this embodiment includes:

step 101, a terminal collects and measures experience quality information of XR service; the quality of experience information includes at least one of: application layer related quality of experience QoE, wireless related QoE and XR user quality of experience; the XR user experience quality is obtained based on QoE related to an application layer and QoE related to wireless;

wherein the application layer related QoE is related to the application layer data, and the wireless related QoE is related to wireless measurement.

Optionally, the terminal side collects and measures quality of experience information of the XR service, and reports the quality of experience information to an application layer, an access network device, a core network device or an application server of the terminal.

Quality of experience information may include application layer related QoE, such as packet data convergence protocol (Packet Data Convergence Protocol, PDCP) packet information, source coding information, video acquisition information (e.g., resolution and frame rate of the acquisition device), and so forth.

The quality of experience information may also include wireless related QoE, such as packet error rate (Packet Error Ratio, PER), transmission duration, delay, jitter, etc.

The quality of experience information may also include XR user quality of experience, which may be derived based on application layer related QoE and radio related QoE, alternatively, the XR user quality of experience may be represented by an index or identifier, which may be a specific value, or may be a level (e.g., high, medium, low, or level 1, 2, 3, 4, or 5, etc.).

Step 102, the terminal reports the experience quality information to the first device.

Wherein the first device comprises at least one of: terminal, access network device, core network device and application server.

The terminal can report the experience quality information to an application layer, access network equipment, core network equipment or an application server of the terminal, and further can perform adaptive scheduling control, transmission adjustment and control, error management or source coding and the like aiming at XR service, so that the system performance can be improved, and the user experience can be improved.

According to the method, the terminal collects and measures the experience quality information of the XR service; the quality of experience information includes at least one of: application layer related quality of experience QoE, wireless related QoE and XR user quality of experience; the XR user experience quality is obtained based on QoE related to an application layer and QoE related to wireless; the experience quality information is strongly related to the XR service, so that the user experience quality of an XR user can be better reflected, and the user experience can be improved by reporting the experience quality information strongly related to the XR service and adaptively processing the XR service.

Optionally, the application layer related quality of experience QoE comprises at least one of:

PDCP layer packet information corresponding to at least one frame or slice;

mapping relation between PDCP layer data packet and frame;

mapping relation between PDCP layer data packet and slice;

source coding information;

uplink video acquisition information;

waiting for the buffer time of playing;

buffering times waiting for playing;

information whether the voice and the image are synchronous;

wherein the PDCP layer packet information includes at least one of: the number of the data packets, the size of the data packets and the total transmission delay of the data packets;

the source coded information includes at least one of: coding mode, coding rate and frame rate;

the uplink video acquisition information comprises at least one of the following: the resolution and frame rate of the acquisition device.

Specifically, the application layer related quality of experience QoE comprises at least one of:

the number, size, or total transmission delay of PDCP layer packets corresponding to one or more frames (or slices), etc.;

the mapping relation between the PDCP layer data packet and the frame (or slice), that is, the mapping relation between the data packet and the frame can be established according to a certain rule, for example, the PDCP data packet includes the number of the corresponding frame;

source coded information, such as: coding standard, frame rate, etc.;

Upstream video acquisition information, such as: camera resolution, frame rate;

the buffer time length or the buffer times of waiting for playing refer to the time length or the buffer times of waiting for the subsequent data to arrive when the arriving data does not support playing enough;

synchronous or asynchronous voice and image information.

Optionally, the wireless-related QoE includes at least one of:

packet error rate PER at the frame or application data unit ADU level;

the air interface transmission duration of the frame or ADU;

delay of the frame of the I frame, the P frame or the B frame reaching the target protocol layer of the terminal;

the data size of the I frame, P frame, or B frame;

information of lost slices in each frame;

the ratio of the number of frames of the base layer to the enhancement layer;

the number of enhancement layer lost frames;

the proportion of the lost frames of the enhancement layer to the total frames of the enhancement layer;

the mean or range of jitter;

the number of lost packets or the probability of lost packets caused by jitter reasons;

the time interval between the first data packet of the frame reaching the target protocol layer of the terminal and the last data packet reaching the target protocol layer of the terminal;

the time of reaching the target protocol layer of the terminal is later than the frame number of the target arrival time, and the frame number of the target protocol layer of the terminal is occupied in proportion;

The proportion of frames in which at least one data packet of an I frame, a P frame or a B frame is lost or late;

wherein the information of the lost slice in each frame includes at least one of: average number of slices lost per frame, duty cycle, index and location.

Specifically, the wireless-related QoE includes at least one of:

PER at frame or application data unit (Application Data Unit, ADU) level (e.g., transmission success rate of I/P/B frames);

the air interface transmits the duration of one frame or ADU;

the data size of the I frame, P frame, or B frame;

information of lost slices (slices) in each frame, such as average number of slices or duty cycle of lost slices, index or position of lost slices in one frame;

the ratio of the number of base layer frames to the number of enhancement layer frames;

the number of the lost frames of the enhancement layer or the proportion of the lost frames of the enhancement layer to the total frame number of the enhancement layer;

the mean or range of jitter;

the number of dropped packets or the probability of dropped packets caused by jitter is the number of dropped packets or the probability of dropped packets caused by direct packet dropping or indirect packet dropping;

the frame pulse duration, i.e. the time interval between the arrival of the first internet protocol (Internet Protocol, IP) packet at the target protocol layer of the terminal and the arrival of the last IP packet at the target protocol layer of the terminal;

Wherein the target protocol layer comprises at least one of the following: an application layer, an IP layer, a service data adaptation protocol (Service Data Adaptation Protocol, SDAP) layer, a packet data convergence protocol PDCP layer, a radio link control (Radio Link Conrtol, RLC) layer, a medium access control (Medium Access Control, MAC) layer, and a physical layer.

For example, the time interval between the arrival of the first IP packet at the application layer of the terminal and the arrival of the last IP packet at the application layer of the terminal, or the time interval between the arrival of the first IP packet at the RLC layer of the terminal and the arrival of the last IP packet at the RLC layer of the terminal.

The frame number of the target protocol layer reaching the terminal is later than the target reaching time, and the frame number of the target protocol layer reaching the terminal is in proportion; or, the time to reach the target protocol layer of the terminal is later than the number of frames of the target arrival time.

The target arrival time may be a preset value, and the network side is preconfigured, configured or predefined by a protocol, etc.

The proportion of frames in which at least one IP packet is lost or late in an I-frame, P-frame or B-frame.

In the above embodiment, by reporting the experience quality information strongly related to the XR service, adaptive scheduling control, transmission adjustment and control, error management, source coding and the like can be performed for the XR service, so that the system performance can be improved and the user experience can be improved.

In one embodiment, step 101 may be implemented as follows:

the terminal collects and measures the experience quality information of XR service in a target time range;

the target time range satisfies at least one of the following:

a multiple of group of pictures (Group of Pictures, GOP) length;

a multiple of the number of frames per second FPS;

a length less than the FPS;

the method comprises the steps of presetting absolute time length, wherein the unit of the absolute time length comprises any one of the following: symbols, slots, subframes, system frames, milliseconds, or seconds.

In particular, the terminal may collect and measure within a target time range, which may be a period of time or a point in time.

The target time range may be a multiple of the image group length;

the target time range may be a multiple of the FPS or may be less than the length of the FPS;

the target time range is an absolute duration, for example 1 second. The units of absolute time length include any one of the following: symbols, slots, subframes, system frames, milliseconds, or seconds.

The measurement result of the XR user experience information may be a historical value or a long-term statistical result, such as smoothing, averaging, etc.

Optionally, the measurement of quality of experience information of the XR service by the terminal satisfies at least one of:

The measurement of quality of experience information for XR services has multiple parallel measurement cycles;

the measurement interval of the quality of experience information of XR service is smaller than M GOPs;

the quality of experience information of the XR service is obtained after the first processing is performed based on the measurement result.

In particular, there may be multiple parallel measurement periods, e.g. one measurement period for the enhancement layer and one measurement period for the base layer;

at least one measurement is made every M GOPs, i.e., the measurement interval is less than M GOPs.

The quality of experience information of the XR service is obtained after processing based on the measurement result, and can be a historical value or a value counted for a long time, for example, the quality of experience information is subjected to operations such as smoothing, averaging and the like.

In one embodiment, as shown in fig. 3, the method further comprises:

step 110, the terminal receives measurement configuration information issued by the access network equipment; the measurement configuration information includes at least one of:

a switch for collecting, measuring or reporting the experience quality information;

the duration of collection and measurement of quality of experience information;

a starting point in time for the collection and measurement of quality of experience information;

the number of cycles of collection and measurement of quality of experience information;

the interval of the period of collection and measurement of quality of experience information;

The collection and measurement of quality of experience information is initiated after a first period of time.

Trigger conditions for collection and measurement of quality of experience information.

Optionally, the method further comprises: and 111, the terminal collects and measures the experience quality information of the XR service based on the measurement configuration information.

Specifically, a timer for collection and measurement may be configured, and measurement may be started after the timer expires, where the first duration of the timer may be associated with a specific service binding (e.g., related to a service type, a frame type, importance, etc.), or may be a preconfigured duration.

Optionally, the measurement configuration information is activated by downlink control information (Downlink Control Information, DCI) or media access layer control element (Medium Access Control Control Element, MACCE) sent by the access network device; or alternatively, the first and second heat exchangers may be,

the measurement configuration information is issued and activated by means of radio resource control (Radio Resource Control, RRC).

Specifically, the access network device may issue measurement configuration information through RRC and activate the measurement configuration information, and in addition, the measurement configuration information may not be activated first, and may be activated through corresponding DCI or MAC CE.

Alternatively, as shown in fig. 4, the measurement configuration information is issued to the access network device by the operation, maintenance and administration OAM, or as shown in fig. 5, the measurement configuration information configured by the OAM is activated by the core network device and issued to the access network device.

Optionally, the triggering conditions for the collection and measurement of quality of experience information include at least one of:

the target transmission unit exists in the measurement duration;

measuring the video frames with the enhancement layers in the duration;

dropping or continuously transmitting frames, slices, application data units ADU, internet protocol IP packets or Transport Blocks (TB);

higher layer or application layer triggers.

Wherein discarding includes at least one of: one, N out of consecutive X, M, or key frames (data slices, ADUs, IP packets, or TBs) are discarded.

Specifically, a target transmission unit, such as an I frame or a key frame, exists in the measured duration;

measuring the video frames with the enhancement layers in the duration;

frame loss, e.g., one frame drop, or, consecutive X frames drops; or N frames in M frames are discarded; or key frame dropping, e.g., the key frame may be an I-frame, etc.;

consecutive errors, such as multiple frames, data slices, ADUs, IP packets, or TBs.

In the embodiment, the terminal can collect and measure the experience quality information based on the measurement configuration information sent by the access network equipment, so that the implementation scheme is simple and the complexity is low.

In one embodiment, step 102 may be specifically implemented as follows:

Starting or restarting the process of reporting the quality of experience information to the first device when at least one of the following is satisfied:

ending the measurement;

the measurement result is smaller or larger than a first threshold; 1

The time length of the measurement result meeting the first target condition reaches the second time length;

the classified or quantized measurement results meet a second target condition;

the previous N times of measurement results are superior to the current measurement results, and N is a positive integer.

Specifically, the terminal can report after the measurement is completed; or, reporting according to the collected and measured content, including at least one of the following:

when the measurement result is smaller or larger than a first threshold, the reporting can be started or restarted; the first threshold may be associated with a specific service binding (e.g. related to service type, frame type, importance, etc.), or may be predefined by network side configuration/pre-configuration, terminal configuration or protocol;

when the measured time length of the measurement result meeting the first target condition reaches the second time length, the reporting can be started or restarted, wherein the first target condition is that the average proportion of the lost slices in each frame is greater than a preset value, for example, one percent; the second duration may be associated with a specific service binding (e.g., related to service type, frame type, importance, etc.), or may be predefined by a network side configuration/pre-configuration, terminal configuration, or protocol;

The measurement result obtained after grading or quantizing the measurement result meets a second target condition, and reporting can be started or restarted, for example, the measurement result after grading or quantizing can be represented by grades, for example, the grades of the measurement result have three grades of high, middle and low, and when the grade is low, direct reporting is triggered;

the previous N measurements are better than the current measurement, e.g., the current measurement is worse than the last measurement; or, the current measurement is worse than the historically averaged measurement, etc.

Optionally, when the first device is an access network device, the quality of experience information is reported through at least one of the following:

a radio resource control, RRC, message;

l1 signaling or medium access control MAC messages;

when the first device is the terminal, the experience quality information is reported through at least one of the following:

cross-layer interaction;

an AT instruction;

when the first device is a core network device or an application server, the quality of experience information is reported through at least one of the following:

a user plane link;

enhancement (Protocol Data Unit, PDU) session establishment procedure or enhancement PDU session modification procedure.

Specifically, after the terminal collects and measures the quality of experience information, the quality of experience information may be reported to at least one first device of an application layer, an access network device, a core network device, and an application server of the terminal.

If the first device is an access network device, the terminal may report to the access network device through RRC message or L1 signaling or MAC message.

If the first device is a terminal, the terminal may report the quality of experience information to an application layer of the terminal through cross-layer interaction, or the terminal may report the quality of experience information to the application layer from RRC through an AT command (Attention command).

If the first device is a core network device or an application server, the terminal may report to the core network device or the application server through a user plane link, where the user plane link is, for example, a terminal-RAN-user plane function (User Plane Function, UPF) -application server. Or, reporting through an enhanced PDU session establishment procedure or an enhanced PDU session modification procedure.

In the above embodiment, different modes may be adopted for reporting the quality of experience information to different devices, so that the flexibility is high.

In an embodiment, the method may further comprise:

the terminal receives a deactivation instruction sent by access network equipment; the deactivation indication is used for indicating the terminal to deactivate the measurement of the quality of experience information, and the deactivation indication is carried by at least one of the following:

an RRC message;

a MAC CE; or (b)

DCI。

Optionally, the deactivation indication is triggered if at least one of the following is satisfied:

Continuously reporting measurement results of the experience quality information of preset times to meet a third target condition;

the timer times out; the timer is started when the experience quality information is reported for the first time.

Specifically, the collection and measurement of the quality of experience information by the terminal may be started or stopped by an activation or deactivation indication sent by the access network device, where a triggering condition of the deactivation indication may be: the measurement results of the multiple experience quality information which are continuously reported meet a third target condition, such as being greater than or less than a certain threshold value; or alternatively, the first and second heat exchangers may be,

the timer times out, e.g. the timer may be configured for deactivation, the timer may be started when the quality of experience information is first reported. Alternatively, the duration of the timer may be the same as or different from the first duration.

In the embodiment, the terminal can activate or deactivate the terminal under the instruction of the access network equipment for collecting and measuring the experience quality information, so that the implementation scheme is simple, the complexity is low and the flexibility is high.

Illustratively, the terminal receives measurement configuration information of the access network device, and activates periodic collection and measurement of quality of experience information, wherein:

a) The starting time point of the collection and measurement is the receiving time of the next data packet;

b) The measurement duration is 3 GOP groups;

c) The measurement period is 1 second;

d) The number of the measuring periods is 10;

the terminal side collects and measures the transmission success rate of the I frame or the P frame in one measurement time length;

and if the transmission success rate of the I frame or the P frame in the measurement duration is smaller than a threshold value (for example, the threshold value is 99%), triggering reporting, and reporting the experience quality information to the access network equipment.

The access network device executes adjustment of the scheduling policy based on the transmission success rate of the received I frame or P frame.

Illustratively, the terminal receives measurement configuration information of the access network device and activates collection and measurement of XR user experience information, wherein:

a) The triggering condition of collection and measurement is that frame loss occurs;

b) The measurement duration is 3 GOP groups;

the terminal side collects and measures XR user experience quality in a measurement time period;

the XR user experience quality is a comprehensive index derived by comprehensively considering indexes such as PER, packet delay budget (Packet Delay Budget, PDB), frame and data packet mapping relation, and the derived formula is preconfigured.

And triggering and reporting to the access network equipment after the terminal side completes the collection and measurement of the XR user experience quality.

The reported XR user quality of experience may be a specific value, or may be a level (e.g., high, medium, low) or index/logo (e.g., level 1, 2, 3, 4, or 5, etc.).

The access network device performs adjustment of the scheduling policy based on the received XR user quality of experience.

Fig. 6 is a second flowchart of a quality of experience reporting method for XR service according to an embodiment of the present application. As shown in fig. 6 and 7, the method provided in this embodiment includes:

step 201, the access network device collects and measures the experience quality information of XR service; the quality of experience information includes at least one of: application layer related quality of experience QoE, wireless related QoE and XR user quality of experience; the XR user experience quality is obtained based on QoE related to an application layer and QoE related to wireless; and/or, the access network equipment receives the experience quality information reported by the terminal;

202, the access network equipment performs target processing on the experience quality information; and/or the access network equipment reports the experience quality information to the second equipment.

The second device may be a core network device or an application server.

Optionally, the application layer related quality of experience QoE includes at least one of:

PDCP layer packet information corresponding to at least one frame or slice;

the mapping relation between the PDCP layer data packet and the frame, that is, the mapping relation between the data packet and the frame may be established according to a certain rule, for example, the PDCP data packet includes the number of the corresponding frame;

mapping relation between PDCP layer data packet and slice;

source coding information;

uplink video acquisition information;

waiting for the buffer time of playing;

buffering times waiting for playing;

information whether the voice and the image are synchronous;

Optionally, the wireless-related QoE includes at least one of:

packet error rate PER at the frame or application data unit ADU level (e.g., transmission success rate of I/P/B frames);

the air interface transmission duration of the frame or ADU;

delay for the frame of the I frame, P frame or B frame to reach the target protocol layer of the access network device;

the data size of the I frame, P frame, or B frame;

Information of lost slices in each frame;

the ratio of the number of frames of the base layer to the enhancement layer;

the number of enhancement layer lost frames;

the mean or range of jitter;

the time interval between the arrival of the first data packet of a frame at the target protocol layer of the access network device and the arrival of the last data packet at the target protocol layer of the access network device, for example, the time interval between the arrival of the first IP packet at the application layer of the access network device and the arrival of the last IP packet at the application layer of the access network device, or the time interval between the arrival of the first IP packet at the RLC layer of the access network device and the arrival of the last IP packet at the RLC layer of the access network device;

the frame number of the target protocol layer reaching the access network equipment, the time of which is later than the target arrival time, accounts for the proportion of the frame number of the target protocol layer reaching the terminal;

Wherein the target arrival time may be the same as or different from the target arrival time at the terminal side.

Optionally, the "the access network device collects and measures quality of experience information of XR services" in step 201 may be implemented specifically by:

the access network equipment collects and measures the experience quality information of the XR service in a target time range;

the target time range satisfies at least one of the following:

a multiple of the group of pictures GOP length;

a multiple of the number of frames per second FPS;

a length less than the FPS;

In particular, the access network device may collect and measure within a target time range, which may be a period of time or a point in time.

Optionally, the measurement of quality of experience information of the XR service by the access network device meets at least one of:

The measurement of the quality of experience information of the XR service has a plurality of parallel measurement periods;

the measurement interval of the quality of experience information of the XR service is smaller than M GOPs;

the quality of experience information of the XR service is obtained after the first processing is carried out based on the measurement result.

Optionally, the method further comprises:

the access network equipment sends measurement configuration information to the terminal; the measurement configuration information includes at least one of:

the switch is used for collecting, measuring or reporting the experience quality information;

a time period for the collection and measurement of the quality of experience information;

a starting point in time for the collection and measurement of the quality of experience information;

the number of cycles of collection and measurement of the quality of experience information;

an interval of a period of collection and measurement of the quality of experience information;

And starting the collection and measurement of the quality of experience information after a first time period.

Trigger conditions for the collection and measurement of the quality of experience information.

Specifically, the access network device may configure a timer for collection and measurement, and after the timer expires, measurement is started, and the first duration of the timer may be associated with a specific service binding (e.g., related to a service type, a frame type, importance, etc.), or may be a preconfigured duration.

Optionally, the measurement configuration information is issued to the access network device by operation maintenance management OAM, or the core network device activates the measurement configuration information configured by OAM and issues to the access network device.

Optionally, the measurement configuration information is activated by downlink control information DCI or a media access layer control unit MAC CE sent by the access network device; or alternatively, the first and second heat exchangers may be,

the measurement configuration information is issued and activated by the radio resource control RRC.

Optionally, the triggering condition of the collection and measurement of the quality of experience information includes at least one of:

the target transmission unit exists in the measurement duration;

measuring the video frames with the enhancement layers in the duration;

discarding or continuously transmitting errors of frames, slices, application data units ADU, internet protocol IP packets or transmission blocks TB;

Higher layer or application layer triggers.

measuring the video frames with the enhancement layers in the duration;

Optionally, in step 202, "the access network device reports the quality of experience information to the second device", which may be specifically implemented by:

initiating or restarting the process of reporting the quality of experience information to the second device when at least one of:

ending the measurement;

the measurement result is smaller or larger than a first threshold;

the classified or quantized measurement results meet a second target condition;

Optionally, the first target process includes at least one of:

The uplink and downlink resource pre-scheduling, resource allocation, beam management or error management of the access network equipment;

time-frequency domain resource coordination among different terminals;

the access network device performs transmission or scheduling of dropping, delaying or merging of data packets.

The specific implementation process and technical effects of the method of the present embodiment are similar to those of the terminal side method embodiment, and specific reference may be made to the detailed description of the terminal side method embodiment, which is not repeated herein.

Fig. 8 is a third flow chart of a quality of experience reporting method of XR service according to the embodiment of the present application. As shown in fig. 8, the method provided in this embodiment includes:

step 301, receiving reported experience quality information of XR service by core network equipment; the quality of experience information includes at least one of: application layer related quality of experience QoE, wireless related QoE and XR user quality of experience; the XR user experience quality is obtained based on QoE related to an application layer and QoE related to wireless;

and 302, the core network equipment performs target processing on the experience quality information of the XR service.

PDCP layer packet information corresponding to at least one frame or slice;

mapping relation between PDCP layer data packet and slice;

source coding information;

uplink video acquisition information;

waiting for the buffer time of playing;

buffering times waiting for playing;

information whether the voice and the image are synchronous;

Optionally, the wireless-related QoE includes at least one of:

the air interface transmission duration of the frame or ADU;

delay of the frame of the I frame, the P frame or the B frame reaching the target protocol layer of the terminal or the access network equipment;

the data size of the I frame, P frame, or B frame;

information of lost slices in each frame;

the ratio of the number of frames of the base layer to the enhancement layer;

The number of enhancement layer lost frames;

the mean or range of jitter;

the time interval between the arrival of the first data packet of a frame at the target protocol layer of the terminal and the arrival of the last data packet at the target protocol layer of the terminal, for example, the time interval between the arrival of the first IP packet at the application layer of the terminal and the arrival of the last IP packet at the application layer of the terminal, or the time interval between the arrival of the first IP packet at the RLC layer of the terminal and the arrival of the last IP packet at the RLC layer of the terminal;

the time interval between the first data packet of the frame reaching the target protocol layer of the access network device and the last data packet reaching the target protocol layer of the access network device;

the frame number of the target protocol layer reaching the terminal or the access network equipment is later than the target reaching time, and the frame number of the target protocol layer reaching the terminal is in proportion;

the time of reaching the target protocol layer of the access network equipment is later than the frame number of the target arrival time, and the frame number of the target protocol layer of the access network equipment is taken up as the proportion of the frame number of the target protocol layer of the access network equipment;

Optionally, the second target process includes a source coding adjustment, the source coding adjustment including at least one of:

coding mode, coding rate, or video frame rate.

According to the method for reporting the experience quality of the XR service, which is provided by the embodiment of the application, an execution body can be the device for reporting the experience quality of the XR service. In the embodiment of the present application, an example is taken as an example of a method for reporting quality of experience of XR service executed by an XR service reporting device, which is provided by the embodiment of the present application.

Fig. 9 is a schematic structural diagram of an XR service quality of experience reporting device provided in the present application. As shown in fig. 9, the quality of experience reporting device for XR service provided in this embodiment includes:

a processing module 210, configured to collect and measure quality of experience information of XR service; the quality of experience information includes at least one of: application layer related quality of experience QoE, wireless related QoE and XR user quality of experience; the XR user experience quality is obtained based on the QoE related to the application layer and the QoE related to wireless;

and the sending module 220 is configured to report the quality of experience information to the first device.

packet data convergence protocol PDCP layer packet information corresponding to at least one Frame or Slice;

mapping relation between PDCP layer data packet and frame;

mapping relation between PDCP layer data packet and slice;

source coding information;

uplink video acquisition information;

waiting for the buffer time of playing;

buffering times waiting for playing;

information whether the voice and the image are synchronous;

Optionally, the wireless-related QoE includes at least one of:

packet error rate PER at the frame or application data unit ADU level;

the air interface transmission duration of the frame or ADU;

the data size of the I frame, P frame, or B frame;

information of lost slices in each frame;

the ratio of the number of frames of the base layer to the enhancement layer;

the number of enhancement layer lost frames;

the mean or range of jitter;

the time interval between the arrival of the first data packet of a frame at the target protocol layer of the terminal and the arrival of the last data packet at the target protocol layer of the terminal;

the frame number of the target protocol layer reaching the terminal is later than the target reaching time, and the frame number of the target protocol layer reaching the terminal is in proportion;

Optionally, the processing module 210 is specifically configured to:

collecting and measuring quality of experience information of the XR service within a target time range;

the target time range satisfies at least one of the following:

a multiple of the group of pictures GOP length;

a multiple of the number of frames per second of the FPS, or less than the length of the FPS;

Optionally, the measurement of quality of experience information of the XR service satisfies at least one of:

Optionally, the method further comprises:

the receiving module is used for receiving measurement configuration information issued by the access network equipment; the measurement configuration information includes at least one of:

Triggering conditions for the collection and measurement of the quality of experience information;

the processing module 210 is specifically configured to:

and collecting and measuring the experience quality information of the XR service based on the measurement configuration information.

The target transmission unit exists in the measurement duration;

measuring the video frames with the enhancement layers in the duration;

higher layer or application layer triggers.

Optionally, the sending module 220 is specifically configured to:

initiating or restarting the process of reporting the quality of experience information to the first device when at least one of:

ending the measurement;

the measurement result is smaller or larger than a first threshold;

the classified or quantized measurement results meet a second target condition;

Optionally, the first device comprises at least one of: access network equipment, core network equipment, the terminal or an application server.

a radio resource control, RRC, message;

l1 signaling or medium access control MAC messages;

Cross-layer interaction;

an AT instruction;

a user plane link;

an enhanced protocol data unit, PDU, session establishment procedure or an enhanced PDU, session modification procedure.

Optionally, the receiving module is further configured to:

receiving a deactivation instruction sent by the access network equipment; the deactivation indication is used for indicating the terminal to deactivate the measurement of the quality of experience information, and the deactivation indication is carried by at least one of the following:

an RRC message;

a MAC CE; or (b)

DCI。

The apparatus of this embodiment is configured to execute a terminal side method embodiment, and its specific implementation process and technical effects are similar to those of the terminal side method embodiment, and specific details of the terminal side method embodiment may be referred to in the description of the terminal side method embodiment, which is not repeated herein.

Fig. 10 is a second schematic structural diagram of an XR service quality of experience reporting device provided in the present application. As shown in fig. 10, the quality of experience reporting device for XR service provided in this embodiment includes:

A processing module 310, configured to collect and measure quality of experience information of XR service by the access network device; the quality of experience information includes at least one of: application layer related quality of experience QoE, wireless related QoE and XR user quality of experience; the XR user experience quality is obtained based on the QoE related to the application layer and the QoE related to wireless; and/or the number of the groups of groups,

a receiving module 320, configured to receive the quality of experience information reported by the terminal;

the processing module 310 is further configured to perform target processing on the quality of experience information; and/or the number of the groups of groups,

and the sending module 330 is configured to report the quality of experience information to the core network device.

PDCP layer packet information corresponding to at least one frame or slice;

mapping relation between PDCP layer data packet and frame;

mapping relation between PDCP layer data packet and slice;

source coding information;

uplink video acquisition information;

waiting for the buffer time of playing;

buffering times waiting for playing;

information whether the voice and the image are synchronous;

Optionally, the wireless-related QoE includes at least one of:

packet error rate PER at the frame or application data unit ADU level;

the air interface transmission duration of the frame or ADU;

delay for the frame of the I frame, the P frame or the B frame to reach the target protocol layer of the access network equipment;

the data size of the I frame, P frame, or B frame;

information of lost slices in each frame;

the ratio of the number of frames of the base layer to the enhancement layer;

the number of enhancement layer lost frames;

the mean or range of jitter;

the frame number of the target protocol layer reaching the access network equipment is later than the target reaching time, and the frame number of the target protocol layer reaching the access network equipment is in proportion to the frame number of the target protocol layer reaching the access network equipment;

Optionally, the processing module 310 is specifically configured to:

the target time range satisfies at least one of the following:

a multiple of the group of pictures GOP length;

Optionally, the sending module 330 is further configured to:

transmitting measurement configuration information to the terminal; the measurement configuration information includes at least one of:

the target transmission unit exists in the measurement duration;

measuring the video frames with the enhancement layers in the duration;

higher layer or application layer triggers.

Optionally, the sending module 330 is specifically configured to:

ending the measurement;

the measurement result is smaller or larger than a first threshold;

the classified or quantized measurement results meet a second target condition;

Optionally, the first target process includes at least one of:

time-frequency domain resource coordination among different terminals;

the access network device discards, delays or merges transmission or scheduling of the data packets;

optionally, the second device comprises at least one of: core network equipment and an application server.

The apparatus in this embodiment is configured to execute an embodiment of an access network device method, and specific implementation procedures and technical effects of the embodiment are similar to those of an embodiment of a terminal side method, and specific details of the embodiment of the access network device method may be referred to in the embodiment of the access network device method, which is not described herein again.

Fig. 11 is a third schematic structural diagram of the quality of experience reporting device for XR service provided in the present application. As shown in fig. 11, the quality of experience reporting device for XR service provided in this embodiment includes:

a receiving module 410, configured to receive reported quality of experience information of the XR service; the quality of experience information includes at least one of: application layer related quality of experience QoE, wireless related QoE and XR user quality of experience; the XR user experience quality is obtained based on the QoE related to the application layer and the QoE related to wireless;

and a processing module 420, configured to perform target processing on the quality of experience information of the XR service.

PDCP layer packet information corresponding to at least one frame or slice;

mapping relation between PDCP layer data packet and frame;

mapping relation between PDCP layer data packet and slice;

source coding information;

uplink video acquisition information;

waiting for the buffer time of playing;

buffering times waiting for playing;

information whether the voice and the image are synchronous;

Optionally, the wireless-related QoE includes at least one of:

packet error rate PER at the frame or application data unit ADU level;

the air interface transmission duration of the frame or ADU;

the data size of the I frame, P frame, or B frame;

information of lost slices in each frame;

the ratio of the number of frames of the base layer to the enhancement layer;

the number of enhancement layer lost frames;

the mean or range of jitter;

Optionally, the target processing includes source coding adjustment, including at least one of:

coding mode, coding rate, or video frame rate.

The apparatus in this embodiment is configured to execute a method embodiment of a core network device, and specific implementation procedures and technical effects of the method embodiment of the core network device are similar to those of the method embodiment of the core network device, and specific details of the method embodiment of the core network device may be referred to in the detailed description of the method embodiment of the core network device, which is not repeated herein.

The quality of experience reporting device of the XR service in the embodiment of the application may be an electronic device, for example, an electronic device with an operating system, or may be a component in the electronic device, for example, an integrated circuit or a chip. The electronic device may be a terminal, or may be other devices than a terminal. By way of example, terminals may include, but are not limited to, the types of terminals 11 listed above, other devices may be servers, network attached storage (Network Attached Storage, NAS), etc., and embodiments of the application are not specifically limited.

The quality of experience reporting device for XR service provided in this embodiment of the present application can implement each process implemented by the method embodiments of fig. N to n+x, and achieve the same technical effect, so that repetition is avoided, and no further description is given here.

Optionally, as shown in fig. 12, the embodiment of the present application further provides a communication device 1200, including a processor 1201 and a memory 1202, where the memory 1202 stores a program or an instruction that can be executed on the processor 1201, for example, when the communication device 1200 is a terminal, the program or the instruction is executed by the processor 1201 to implement each step of the foregoing embodiment of the quality of experience reporting method of XR service, and the same technical effect can be achieved. When the communication device 1200 is a network side device, the program or the instruction implements the steps of the foregoing embodiment of the method for reporting quality of experience of XR service when executed by the processor 1201, and the same technical effects can be achieved, so that repetition is avoided, and no further description is given here.

The embodiment of the application also provides a terminal, which comprises a processor and a communication interface, wherein the processor is used for collecting and measuring the experience quality information of the XR service; the quality of experience information includes at least one of: application layer related quality of experience QoE, wireless related QoE and XR user quality of experience; the XR user experience quality is obtained based on the QoE related to the application layer and the QoE related to wireless; the communication interface is used for reporting the experience quality information to the first equipment. The terminal embodiment corresponds to the terminal-side method embodiment, and each implementation process and implementation manner of the method embodiment can be applied to the terminal embodiment, and the same technical effects can be achieved. Specifically, fig. 13 is a schematic hardware structure of a terminal for implementing an embodiment of the present application.

The terminal 1000 includes, but is not limited to: at least some of the components of the radio frequency unit 1001, the network module 1002, the audio output unit 1003, the input unit 1004, the sensor 1005, the display unit 1006, the user input unit 1007, the interface unit 1008, the memory 1009, and the processor 1010, etc.

Those skilled in the art will appreciate that terminal 1000 can also include a power source (e.g., a battery) for powering the various components, which can be logically connected to processor 1010 by a power management system so as to perform functions such as managing charge, discharge, and power consumption by the power management system. The terminal structure shown in fig. 13 does not constitute a limitation of the terminal, and the terminal may include more or less components than shown, or may combine certain components, or may be arranged in different components, which will not be described in detail herein.

It should be understood that in the embodiment of the present application, the input unit 1004 may include a graphics processing unit (Graphics Processing Unit, GPU) 10041 and a microphone 10042, and the graphics processor 10041 processes image data of still pictures or videos obtained by an image capturing device (such as a camera) in a video capturing mode or an image capturing mode. The display unit 1006 may include a display panel 10061, and the display panel 10061 may be configured in the form of a liquid crystal display, an organic light emitting diode, or the like. The user input unit 1007 includes at least one of a touch panel 10071 and other input devices 10072. The touch panel 10071 is also referred to as a touch screen. The touch panel 10071 can include two portions, a touch detection device and a touch controller. Other input devices 10072 may include, but are not limited to, a physical keyboard, function keys (e.g., volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, and so forth, which are not described in detail herein.

In this embodiment, after receiving downlink data from the network side device, the radio frequency unit 1001 may transmit the downlink data to the processor 1010 for processing; in addition, the radio frequency unit 1001 may send uplink data to the network side device. In general, the radio frequency unit 1001 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, and the like.

The memory 1009 may be used to store software programs or instructions and various data. The memory 1009 may mainly include a first storage area storing programs or instructions, which may store an operating system, application programs or instructions (such as a sound playing function, an image playing function, etc.) required for at least one function, and a second storage area storing data. Further, the memory 1009 may include volatile memory or nonvolatile memory, or the memory 1009 may include both volatile and nonvolatile memory. Including high-speed random access Memory, and may also include non-volatile Memory, where the non-volatile Memory may be Read-Only Memory (ROM), programmable ROM (PROM), erasable Programmable ROM (EPROM), electrically Erasable Programmable EPROM (EEPROM), or flash Memory. The volatile memory may be random access memory (Random Access Memory, RAM), static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (ddr SDRAM), enhanced SDRAM (Enhanced SDRAM), synchronous DRAM (SLDRAM), and Direct RAM (DRRAM). The memory 1009 in embodiments of the present application includes, but is not limited to, these and any other suitable types of memory such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid-state storage device.

The processor 1010 may include one or more processing units; alternatively, the processor 1010 may integrate an application processor that primarily processes operations involving an operating system, a user interface, and applications or instructions, and a modem processor that primarily processes wireless communication signals, such as a baseband processor. It will be appreciated that the modem processor described above may not be integrated into the processor 1010.

Wherein the processor 1010 is configured to collect and measure quality of experience information of XR service; the quality of experience information includes at least one of: application layer related quality of experience QoE, wireless related QoE and XR user quality of experience; the XR user experience quality is obtained based on the QoE related to the application layer and the QoE related to wireless;

the radio frequency unit 1001 is configured to report quality of experience information to the first device.

In the above embodiment, the terminal collects and measures quality of experience information of XR service; the quality of experience information includes at least one of: application layer related quality of experience QoE, wireless related QoE and XR user quality of experience; the XR user experience quality is obtained based on QoE related to an application layer and QoE related to wireless; the experience quality information is strongly related to the XR service, so that the user experience quality of an XR user can be better reflected, and the user experience can be improved by reporting the experience quality information strongly related to the XR service and adaptively processing the XR service.

mapping relation between PDCP layer data packet and frame;

mapping relation between PDCP layer data packet and slice

Source coding information;

uplink video acquisition information;

waiting for the buffer time of playing;

buffering times waiting for playing;

information whether the voice and the image are synchronous;

Optionally, the wireless-related QoE includes at least one of:

packet error rate PER at the frame or application data unit ADU level;

the air interface transmission duration of the frame or ADU;

the data size of the I frame, P frame, or B frame;

information of lost slices in each frame;

The ratio of the number of frames of the base layer to the enhancement layer;

the number of enhancement layer lost frames;

the mean or range of jitter;

Optionally, the processor 1010 is specifically configured to:

The target time range satisfies at least one of the following:

a multiple of the group of pictures GOP length;

a multiple of the number of frames per second FPS;

a length less than the FPS;

Optionally, the measurement of quality of experience information of the XR service by the processor 1010 satisfies at least one of:

Optionally, the radio frequency unit 1001 is further configured to:

receiving measurement configuration information issued by access network equipment; the measurement configuration information includes at least one of:

the processor 1010 is specifically configured to:

the target transmission unit exists in the measurement duration;

measuring the video frames with the enhancement layers in the duration;

higher layer or application layer triggers.

Optionally, the radio frequency unit 1001 is specifically configured to:

ending the measurement;

the measurement result is smaller or larger than a first threshold;

the classified or quantized measurement results meet a second target condition;

a radio resource control, RRC, message;

l1 signaling or medium access control MAC messages;

cross-layer interaction;

an AT instruction;

a user plane link;

Optionally, the radio frequency unit 1001 is further configured to:

an RRC message;

a MAC CE; or (b)

DCI。

The embodiment of the application also provides access network equipment, which comprises a processor and a communication interface, wherein the processor is used for collecting and measuring the experience quality information of the XR service; the quality of experience information includes at least one of: application layer related quality of experience QoE, wireless related QoE and XR user quality of experience; the XR user experience quality is obtained based on the QoE related to the application layer and the QoE related to wireless; and/or the communication interface is used for receiving the experience quality information reported by the terminal;

The processor is further configured to perform target processing on the quality of experience information, and/or the communication interface is further configured to report the quality of experience information to a second device. The embodiment of the access network device corresponds to the embodiment of the method of the access network device, and each implementation process and implementation manner of the embodiment of the method of the access network device can be applied to the embodiment of the access network device, and the same technical effects can be achieved.

Specifically, the embodiment of the application also provides access network equipment. As shown in fig. 14, the access network apparatus 700 includes: an antenna 71, a radio frequency device 72, a baseband device 73, a processor 75 and a memory 75. The antenna 71 is connected to a radio frequency device 72. In the uplink direction, the radio frequency device 72 receives information via the antenna 71, and transmits the received information to the baseband device 73 for processing. In the downlink direction, the baseband device 73 processes information to be transmitted, and transmits the processed information to the radio frequency device 72, and the radio frequency device 72 processes the received information and transmits the processed information through the antenna 71.

The above-described band processing means may be located in a baseband means 73 and the method performed by the access network device in the above embodiment may be implemented in the baseband means 73, the baseband means 73 comprising a baseband processor 75 and a memory 75.

The baseband device 73 may, for example, comprise at least one baseband board, where a plurality of chips are disposed, as shown in fig. 14, where one chip, for example, a baseband processor 75, is connected to the memory 75 through a bus interface, so as to invoke a program in the memory 75 to perform the operation of the access network device shown in the foregoing method embodiment.

The baseband device 73 may also include a network interface 76 for interacting with the radio frequency device 72, such as a common public radio interface (common public radio interface, CPRI for short).

Specifically, the access network device 700 of the embodiment of the present invention further includes: instructions or programs stored in the memory 75 and capable of running on the processor 75, the processor 75 calls the instructions or programs in the memory 75 to execute the method executed by each module shown in fig. 10, and achieve the same technical effects, so that repetition is avoided and therefore are not described herein.

Specifically, the embodiment of the application also provides core network equipment. As shown in fig. 15, the core network device 800 includes: a processor 801, a network interface 802, and a memory 803. The network interface 802 is, for example, a common public radio interface (common public radio interface, CPRI).

Specifically, the core network device 800 of the embodiment of the present invention further includes: instructions or programs stored in the memory 803 and capable of running on the processor 801, the processor 801 calls the instructions or programs in the memory 803 to execute the method executed by each module shown in fig. 11, and achieve the same technical effect, so that repetition is avoided, and therefore, the description is omitted herein.

The embodiment of the application also provides a readable storage medium, on which a program or an instruction is stored, where the program or the instruction implements each process of the embodiment of the method for reporting quality of experience of XR service when executed by a processor, and the same technical effects can be achieved, so that repetition is avoided, and no redundant description is provided herein.

Wherein the processor is a processor in the terminal described in the above embodiment. The readable storage medium includes computer readable storage medium such as computer readable memory ROM, random access memory RAM, magnetic or optical disk, etc.

The embodiment of the application further provides a chip, the chip includes a processor and a communication interface, the communication interface is coupled with the processor, the processor is used for running a program or an instruction, implementing each process of the embodiment of the method for reporting quality of experience of XR service, and achieving the same technical effect, so as to avoid repetition, and no further description is provided here.

It should be understood that the chips referred to in the embodiments of the present application may also be referred to as system-on-chip chips, or the like.

The embodiments of the present application further provide a computer program/program product, where the computer program/program product is stored in a storage medium, and the computer program/program product is executed by at least one processor to implement each process of the foregoing embodiment of the quality of experience reporting method for XR service, and the same technical effects can be achieved, so that repetition is avoided, and details are not repeated here.

The embodiment of the application also provides a communication system, which comprises: the terminal can be used for executing the steps of the method for reporting the experience quality of the XR service, the access network device can be used for executing the steps of the method for reporting the experience quality of the XR service, and the core network device can be used for executing the steps of the method for reporting the experience quality of the XR service.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element. Furthermore, it should be noted that the scope of the methods and apparatus in the embodiments of the present application is not limited to performing the functions in the order shown or discussed, but may also include performing the functions in a substantially simultaneous manner or in an opposite order depending on the functions involved, e.g., the described methods may be performed in an order different from that described, and various steps may also be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solutions of the present application may be embodied essentially or in a part contributing to the prior art in the form of a computer software product stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk), comprising several instructions for causing a terminal (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to perform the method described in the embodiments of the present application.

The embodiments of the present application have been described above with reference to the accompanying drawings, but the present application is not limited to the above-described embodiments, which are merely illustrative and not restrictive, and many forms may be made by those of ordinary skill in the art without departing from the spirit of the present application and the scope of the claims, which are also within the protection of the present application.

Claims

1. The utility model provides a quality of experience reporting method of an extended reality XR service, which is characterized by comprising the following steps:

2. The method for reporting quality of experience for XR service of claim 1, wherein,

the application layer related quality of experience QoE comprises at least one of the following:

mapping relation between PDCP layer data packet and frame;

mapping relation between PDCP layer data packet and slice;

source coding information;

uplink video acquisition information;

waiting for the buffer time of playing;

buffering times waiting for playing;

information whether the voice and the image are synchronized.

3. The method for reporting quality of experience of XR service according to claim 1 or 2, wherein,

the wireless-related QoE includes at least one of:

Packet error rate PER at the frame or application data unit ADU level;

the air interface transmission duration of the frame or ADU;

the data size of the I frame, P frame, or B frame;

information of lost slices in each frame;

the ratio of the number of frames of the base layer to the enhancement layer;

the number of enhancement layer lost frames;

the mean or range of jitter;

the proportion of frames in which at least one packet of I, P or B frames is lost or late.

4. The method for reporting quality of experience of XR service according to claim 1 or 2, wherein the terminal collects and measures quality of experience information of XR service, comprising:

the terminal collects and measures the experience quality information of the XR service in a target time range;

The target time range satisfies at least one of the following:

a multiple of the group of pictures GOP length;

a multiple of the number of frames per second FPS;

a length less than the FPS;

5. The method for reporting quality of experience for XR service of claim 4, wherein,

the measurement of the quality of experience information of the XR service by the terminal meets at least one of the following:

6. The method for reporting quality of experience of XR service according to claim 1 or 2, wherein the method further comprises:

the terminal receives measurement configuration information issued by access network equipment;

the terminal collects and measures quality of experience information of XR service, including:

the terminal collects and measures the experience quality information of the XR service based on the measurement configuration information;

Wherein the measurement configuration information includes at least one of:

starting the collection and measurement of the quality of experience information after a first period of time;

7. The method for reporting quality of experience for XR service of claim 6, wherein,

the measurement configuration information is issued to the access network device by operation maintenance management (OAM), or the core network device activates the measurement configuration information configured by the OAM and issues the measurement configuration information to the access network device.

8. The method for reporting quality of experience for XR service of claim 6, wherein,

the measurement configuration information is activated by downlink control information DCI or a media access layer control unit (MAC CE) sent by the access network equipment; or alternatively, the first and second heat exchangers may be,

9. The method of claim 6, wherein the triggering conditions for the collection and measurement of quality of experience information include at least one of:

the target transmission unit exists in the measurement duration;

measuring the video frames with the enhancement layers in the duration;

higher layer or application layer triggers.

10. The method for reporting quality of experience of XR service according to claim 1 or 2, wherein the reporting the quality of experience information by the terminal to the first device comprises:

ending the measurement;

the measurement result is smaller or larger than a first threshold;

the classified or quantized measurement results meet a second target condition;

11. The method for reporting quality of experience of XR service according to claim 1 or 2, wherein,

the first device comprises at least one of: access network equipment, core network equipment, the terminal or an application server.

12. The method for reporting quality of experience for XR service of claim 11, wherein,

when the first device is an access network device, the quality of experience information is reported through at least one of the following:

a radio resource control, RRC, message;

l1 signaling or medium access control MAC messages;

cross-layer interaction;

an AT instruction;

a user plane link;

13. The method for reporting quality of experience of XR service according to claim 1 or 2, wherein the method further comprises:

an RRC message;

a MAC CE; or (b)

DCI。

14. The method for reporting quality of experience for XR service of claim 13, wherein,

The deactivation indication is triggered if at least one of the following is satisfied:

15. The method for reporting the experience quality of the XR service is characterized by comprising the following steps:

the access network equipment performs first target processing on the experience quality information; and/or the access network equipment reports the experience quality information to the second equipment.

16. The method for reporting quality of experience for XR service of claim 15, wherein,

PDCP layer packet information corresponding to at least one frame or slice;

Mapping relation between PDCP layer data packet and frame;

mapping relation between PDCP layer data packet and slice;

source coding information;

uplink video acquisition information;

waiting for the buffer time of playing;

buffering times waiting for playing;

information whether the voice and the image are synchronized.

17. The method for reporting quality of experience for XR service according to claim 15 or 16, wherein,

the wireless-related QoE includes at least one of:

packet error rate PER at the frame or application data unit ADU level;

the air interface transmission duration of the frame or ADU;

the data size of the I frame, P frame, or B frame;

information of lost slices in each frame;

the ratio of the number of frames of the base layer to the enhancement layer;

the number of enhancement layer lost frames;

the mean or range of jitter;

18. The method for reporting quality of experience of XR service according to claim 15 or 16, wherein the access network device collects and measures quality of experience information of XR service, comprising:

the target time range satisfies at least one of the following:

a multiple of the group of pictures GOP length;

a multiple of the number of frames per second FPS;

a length less than the FPS;

19. The method for reporting quality of experience for XR service according to claim 15 or 16, wherein,

the measurement of quality of experience information of the XR service by the access network device meets at least one of the following:

20. The method for reporting quality of experience of XR services according to claim 15 or 16, wherein the method further comprises:

21. The method for reporting quality of experience for XR service of claim 20, wherein,

22. The method for reporting quality of experience for XR service of claim 20, wherein,

23. The method of claim 20, wherein the triggering conditions for the collection and measurement of quality of experience information include at least one of:

the target transmission unit exists in the measurement duration;

measuring the video frames with the enhancement layers in the duration;

higher layer or application layer triggers.

24. The method for reporting quality of experience of XR service according to claim 15 or 16, wherein the reporting the quality of experience information by the access network device to the second device comprises:

ending the measurement;

the measurement result is smaller or larger than a first threshold;

the classified or quantized measurement results meet a second target condition;

25. The method for reporting quality of experience for XR service according to claim 15 or 16, wherein,

the first target process includes at least one of:

time-frequency domain resource coordination among different terminals;

26. The method for reporting quality of experience for XR service according to claim 15 or 16, wherein,

the second device comprises at least one of: core network equipment and an application server.

27. The method for reporting the experience quality of the XR service is characterized by comprising the following steps:

and the core network equipment performs second target processing on the experience quality information of the XR service.

28. The method for reporting quality of experience for XR service of claim 27, wherein,

PDCP layer packet information corresponding to at least one frame or slice;

mapping relation between PDCP layer data packet and frame;

mapping relation between PDCP layer data packet and slice;

source coding information;

uplink video acquisition information;

waiting for the buffer time of playing;

buffering times waiting for playing;

information whether the voice and the image are synchronized.

29. The method for reporting quality of experience for XR service according to claim 27 or 28, wherein,

the wireless-related QoE includes at least one of:

packet error rate PER at the frame or application data unit ADU level;

the air interface transmission duration of the frame or ADU;

the data size of the I frame, P frame, or B frame;

information of lost slices in each frame;

the ratio of the number of frames of the base layer to the enhancement layer;

the number of enhancement layer lost frames;

the mean or range of jitter;

30. The method for reporting quality of experience for XR service according to claim 27 or 28, wherein,

the second target process includes a source coding adjustment including at least one of:

coding mode, coding rate, or video frame rate.

31. An XR service quality of experience reporting device, comprising:

the processing module is used for collecting and measuring the experience quality information of the XR service by the terminal; the quality of experience information includes at least one of: application layer related quality of experience QoE, wireless related QoE and XR user quality of experience; the XR user experience quality is obtained based on the QoE related to the application layer and the QoE related to wireless;

32. An XR service quality of experience reporting device, comprising:

33. An XR service quality of experience reporting device, comprising:

34. A terminal comprising a processor and a memory storing a program or instructions executable on the processor, which when executed by the processor, performs the steps of the quality of experience reporting method of XR services as claimed in any one of claims 1 to 14.

35. An access network device comprising a processor and a memory storing a program or instructions executable on the processor, which when executed by the processor, performs the steps of the quality of experience reporting method of XR traffic as claimed in any one of claims 15 to 26.

36. A core network device comprising a processor and a memory storing a program or instructions executable on the processor, which when executed by the processor, performs the steps of the quality of experience reporting method of XR services according to any one of claims 27 to 30.

37. A readable storage medium, wherein a program or instructions is stored on the readable storage medium, which when executed by a processor, implements the quality of experience reporting method of an XR service according to any one of claims 1 to 14, or implements the quality of experience reporting method of an XR service according to any one of claims 15 to 26, or implements the steps of the quality of experience reporting method of an XR service according to any one of claims 27 to 30.