WO2023206445A1 - Ai监测装置以及方法 - Google Patents

Ai监测装置以及方法 Download PDF

Info

Publication number
WO2023206445A1
WO2023206445A1 PCT/CN2022/090505 CN2022090505W WO2023206445A1 WO 2023206445 A1 WO2023206445 A1 WO 2023206445A1 CN 2022090505 W CN2022090505 W CN 2022090505W WO 2023206445 A1 WO2023206445 A1 WO 2023206445A1
Authority
WO
WIPO (PCT)
Prior art keywords
model
information
terminal device
network device
performance
Prior art date
Application number
PCT/CN2022/090505
Other languages
English (en)
French (fr)
Inventor
王昕�
孙刚
张群
单宇佳
贾美艺
Original Assignee
富士通株式会社
王昕�
孙刚
张群
单宇佳
贾美艺
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 富士通株式会社, 王昕�, 孙刚, 张群, 单宇佳, 贾美艺 filed Critical 富士通株式会社
Priority to PCT/CN2022/090505 priority Critical patent/WO2023206445A1/zh
Publication of WO2023206445A1 publication Critical patent/WO2023206445A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04B—TRANSMISSION
    • H04B17/00—Monitoring; Testing
    • H04B17/30—Monitoring; Testing of propagation channels
    • H04B17/391—Modelling the propagation channel

Definitions

  • the embodiments of this application relate to the field of communication technology.
  • millimeter-wave frequency bands can provide larger bandwidth and become an important frequency band for 5G new wireless (NR, New Radio) systems. Due to its shorter wavelength, millimeter waves have different propagation characteristics from traditional low-frequency bands, such as higher propagation loss, poor reflection and diffraction performance, etc. Therefore, larger antenna arrays are usually used to form shaped beams with greater gain, overcome propagation losses, and ensure system coverage.
  • NR New Radio
  • AI Artificial Intelligence
  • ML Machine Learning
  • an AI encoder (AI encoder) is used on the terminal device side to encode/compress the CSI
  • an AI decoder (AI decoder) is used on the network device side to encode/compress the CSI. Decoding/decompressing can reduce feedback overhead.
  • AI/ML models are used to predict spatially optimal beam pairs based on the results of a small number of beam measurements, which can reduce system load and delay.
  • embodiments of the present application provide an AI monitoring device and method.
  • an AI monitoring method including:
  • Network equipment receives signals or information sent by terminal equipment
  • the network device monitors or performs performance evaluation on the AI/ML model in the network device and/or the terminal device according to the signal or information.
  • an AI monitoring device including:
  • a receiving unit that receives signals or information sent by the terminal device
  • a processing unit that monitors or performs performance evaluation of the AI/ML model in the network device and/or the terminal device according to the signal or information.
  • an AI monitoring method including:
  • the terminal device receives signals or information sent by the network device.
  • the terminal device monitors or performs performance evaluation on the network device and/or the AI/ML model in the terminal device according to the signal or information.
  • an AI monitoring device including:
  • a receiving unit that receives signals or information sent by the network device
  • a processing unit that monitors or performs performance evaluation of the AI/ML model in the network device and/or the terminal device according to the signal or information.
  • a communication system including:
  • a terminal device that receives signals or information sent by a network device; and monitors or performs performance evaluation of the AI/ML model in the network device and/or the terminal device based on the signal or information; and/or
  • a network device that receives a signal or information sent by a terminal device; and monitors or performs performance evaluation on the network device and/or the AI/ML model in the terminal device based on the signal or information.
  • the network device monitors or performs performance evaluation of the AI/ML model in the network device and/or the terminal device based on signals or information from or from the terminal device, or the terminal device Monitor or perform performance evaluation of the AI/ML model in the network device and/or the terminal device based on signals or information from or from the network device.
  • the operation of the AI/ML model can be monitored, the consistency of the AI/ML model operation can be maintained, and the robustness of the model operation can be improved.
  • Figure 1 is a schematic diagram of a communication system according to an embodiment of the present application.
  • FIG. 2 is a schematic diagram of the AI monitoring method according to the embodiment of the present application.
  • Figure 3 is a schematic diagram of an AI/ML model for network equipment monitoring terminal equipment according to an embodiment of the present application
  • Figure 4 is a schematic diagram of CSI estimation and reporting according to an embodiment of the present application.
  • Figure 5 is a schematic diagram of beam management or beam prediction according to an embodiment of the present application.
  • Figure 6 is an example diagram of beam estimation or prediction according to the embodiment of the present application.
  • Figure 7 is a schematic diagram of an AI/ML model for network equipment monitoring network equipment according to an embodiment of the present application.
  • Figure 8 is a schematic diagram of the AI monitoring method according to the embodiment of the present application.
  • Figure 9 is a schematic diagram of an AI/ML model for terminal equipment monitoring network equipment according to an embodiment of the present application.
  • Figure 10 is a schematic diagram of an AI/ML model for terminal equipment monitoring terminal equipment according to an embodiment of the present application.
  • FIG. 11 is a schematic diagram of the AI monitoring device according to the embodiment of the present application.
  • Figure 12 is another schematic diagram of the AI monitoring device according to the embodiment of the present application.
  • Figure 13 is a schematic diagram of network equipment according to an embodiment of the present application.
  • Figure 14 is a schematic diagram of a terminal device according to an embodiment of the present application.
  • the terms “first”, “second”, etc. are used to distinguish different elements from the title, but do not indicate the spatial arrangement or temporal order of these elements, and these elements should not be used by these terms. restricted.
  • the term “and/or” includes any and all combinations of one or more of the associated listed terms.
  • the terms “comprises,” “includes,” “having” and the like refer to the presence of stated features, elements, elements or components but do not exclude the presence or addition of one or more other features, elements, elements or components.
  • the term “communication network” or “wireless communication network” may refer to a network that complies with any of the following communication standards, such as Long Term Evolution (LTE, Long Term Evolution), Long Term Evolution Enhanced (LTE-A, LTE- Advanced), Wideband Code Division Multiple Access (WCDMA, Wideband Code Division Multiple Access), High-Speed Packet Access (HSPA, High-Speed Packet Access), etc.
  • LTE Long Term Evolution
  • LTE-A Long Term Evolution Enhanced
  • LTE-A Long Term Evolution Enhanced
  • WCDMA Wideband Code Division Multiple Access
  • High-Speed Packet Access High-Speed Packet Access
  • communication between devices in the communication system can be carried out according to any stage of communication protocols, which may include but are not limited to the following communication protocols: 1G (generation), 2G, 2.5G, 2.75G, 3G, 4G, 4.5G and 5G. , New Wireless (NR, New Radio), future 6G, etc., and/or other communication protocols currently known or to be developed in the future.
  • Network device refers to a device in a communication system that connects a terminal device to a communication network and provides services to the terminal device.
  • Network equipment may include but is not limited to the following equipment: base station (BS, Base Station), access point (AP, Access Point), transmission and reception point (TRP, Transmission Reception Point), broadcast transmitter, mobile management entity (MME, Mobile Management Entity), gateway, server, wireless network controller (RNC, Radio Network Controller), base station controller (BSC, Base Station Controller), etc.
  • the base station may include but is not limited to: Node B (NodeB or NB), evolved Node B (eNodeB or eNB) and 5G base station (gNB), etc.
  • it may also include remote radio head (RRH, Remote Radio Head) , Remote Radio Unit (RRU, Remote Radio Unit), relay or low-power node (such as femeto, pico, etc.).
  • RRH Remote Radio Head
  • RRU Remote Radio Unit
  • relay or low-power node such as femeto, pico, etc.
  • base station may include some or all of their functions, each of which may provide communications coverage to a specific geographic area.
  • the term "cell” may refer to a base station and/or its coverage area, depending on the context in which the term is used.
  • the term "user equipment” (UE, User Equipment) or “terminal equipment” (TE, Terminal Equipment or Terminal Device) refers to a device that accesses a communication network through a network device and receives network services.
  • Terminal equipment can be fixed or mobile, and can also be called mobile station (MS, Mobile Station), terminal, subscriber station (SS, Subscriber Station), access terminal (AT, Access Terminal), station, etc.
  • the terminal equipment may include but is not limited to the following equipment: cellular phone (Cellular Phone), personal digital assistant (PDA, Personal Digital Assistant), wireless modem, wireless communication equipment, handheld device, machine-type communication equipment, laptop computer, Cordless phones, smartphones, smart watches, digital cameras, and more.
  • cellular phone Cellular Phone
  • PDA Personal Digital Assistant
  • wireless modem wireless communication equipment
  • handheld device machine-type communication equipment
  • laptop computer Cordless phones
  • Cordless phones smartphones, smart watches, digital cameras, and more.
  • the terminal device can also be a machine or device for monitoring or measuring.
  • the terminal device can include but is not limited to: Machine Type Communication (MTC) terminals, Vehicle communication terminals, device-to-device (D2D, Device to Device) terminals, machine-to-machine (M2M, Machine to Machine) terminals, etc.
  • MTC Machine Type Communication
  • D2D Device to Device
  • M2M Machine to Machine
  • network side refers to one side of the network, which may be a certain base station or may include one or more network devices as above.
  • user side or “terminal side” or “terminal device side” refers to the side of the user or terminal, which may be a certain UE or may include one or more terminal devices as above.
  • device can refer to network equipment or terminal equipment.
  • Figure 1 is a schematic diagram of a communication system according to an embodiment of the present application, schematically illustrating a terminal device and a network device as an example.
  • the communication system 100 may include a network device 101 and terminal devices 102 and 103.
  • Figure 1 only takes two terminal devices and one network device as an example for illustration, but the embodiment of the present application is not limited thereto.
  • eMBB enhanced mobile broadband
  • mMTC massive machine type communication
  • URLLC Ultra-Reliable and Low -Latency Communication
  • Figure 1 shows that both terminal devices 102 and 103 are within the coverage of the network device 101, but the application is not limited thereto. Neither of the two terminal devices 102 and 103 may be within the coverage range of the network device 101, or one terminal device 102 may be within the coverage range of the network device 101 and the other terminal device 103 may be outside the coverage range of the network device 101.
  • the high-level signaling may be, for example, Radio Resource Control (RRC) signaling; for example, it is called an RRC message (RRC message), and for example, it includes MIB, system information (system information), and dedicated RRC message; or it is called RRC IE (RRC information element).
  • RRC Radio Resource Control
  • high-level signaling may also be MAC (Medium Access Control) signaling; or it may be called MAC CE (MAC control element).
  • RRC Radio Resource Control
  • RRC message RRC message
  • MIB system information (system information), and dedicated RRC message
  • RRC IE RRC information element
  • high-level signaling may also be MAC (Medium Access Control) signaling; or it may be called MAC CE (MAC control element).
  • MAC CE Medium Access Control
  • one or more AI/ML models can be configured and run in network devices and/or terminal devices.
  • the AI/ML model can be used for various signal processing functions of wireless communication, such as CSI estimation and reporting, beam management, beam prediction, etc.; this application is not limited thereto.
  • the embodiment of this application provides an AI monitoring method, which is explained from the network device side.
  • the AI/ML model is monitored by a network device.
  • FIG. 2 is a schematic diagram of the AI monitoring method according to the embodiment of the present application. As shown in Figure 2, the method includes:
  • the network device receives the signal or information sent by the terminal device
  • the network device monitors or performs performance evaluation on the AI/ML model in the network device and/or the terminal device according to the signal or information.
  • AI/ML models can be run separately for different signal processing functions.
  • the AI/ML model reported by CSI can have different model group identifiers, model identifiers, and version identifiers.
  • AI/ML models for beam management may have additional model group identifiers, model identifiers, and version identifiers.
  • the network device monitors or performs performance evaluation of the AI/ML model in the terminal device, and when the performance of the AI/ML model meets predetermined conditions, the network device reports to the The terminal device sends instruction information to stop the AI/ML model.
  • FIG 3 is a schematic diagram of an AI/ML model of a network device monitoring terminal device according to an embodiment of the present application.
  • the network device reception can receive a direct output signal from the AI/ML model of the terminal device or a signal related to the AI model output, such as a feedback signal in Figure 3.
  • the network device can monitor the performance of the AI/ML model on the terminal device side. When the performance fails to reach a certain indicator, the network device determines or infers that the processing performance of the AI/ML model on the terminal device side is not good, so it can send a stop instruction to the terminal device, and the terminal device stops the AI/ML model. . In addition, network devices can simultaneously or separately instruct terminal devices to switch to traditional non-AI processing modes to ensure stable and uninterrupted communication performance.
  • the signal or information sent by the terminal device is a signal or information related to the AI/ML model in the terminal device for a certain signal processing function. For example, it may be feedback information related to CSI, or an uplink signal related to beam management, and so on.
  • the indication information also includes identification information of the AI/ML model, and/or the indication information also instructs the terminal device to switch to non-AI/ML processing corresponding to a certain signal processing function.
  • the identification information of the AI/ML model includes at least one of the following: the signal processing function identification corresponding to the AI/ML model, the identification of the AI/ML model, the model group identification of the AI/ML model, The group identifier of the AI/ML model.
  • AI/ML models that consider multiple functions may run simultaneously, and the ON/OFF information indication of model monitoring contains identification information of the AI model. For example, it indicates that the AI/ML model with index 2 in model group 2 for CSI feedback will be stopped.
  • the AI/ML model runs on the terminal device and is used for channel state information (CSI) estimation or prediction; the network device responds to hybrid automatic repeat request (HARQ) feedback from the terminal device The information is monitored, and the performance of the AI/ML model is evaluated based on the HARQ feedback information.
  • CSI channel state information
  • HARQ hybrid automatic repeat request
  • Figure 4 is a schematic diagram of CSI estimation and reporting according to an embodiment of the present application. As shown in Figure 4, the method includes:
  • the network device sends CSI-RS
  • the terminal device performs CSI estimation or prediction
  • the network device configures and instructs the terminal device to perform CSI estimation and reporting;
  • the terminal device uses the AI/ML model to perform CSI estimation or prediction;
  • the network device receives the CSI reported by the terminal device
  • the reported CSI may include CQI, PMI, RI, CRI, etc., and the present application is not limited thereto.
  • the network device sends downlink information according to the reported CSI.
  • the network device receives the HARQ feedback information sent by the terminal device; wherein the HARQ feedback information is generated by the terminal device based on the downlink information.
  • the method may also include:
  • the network device monitors and performs performance evaluation of the AI/ML model based on HARQ ACK/NCK.
  • a predetermined condition eg, too many NACK feedbacks within a period of time
  • the method may also include:
  • the network device sends a stop instruction to the terminal device to stop the AI/ML model.
  • the network device side can perform AI/ML model monitoring based on the CSI of the opposite end.
  • the method may further include:
  • the network device receives the channel change indication information sent by the terminal device.
  • the channel change indication information includes the change information of the channel between the network device and the terminal device in the time domain and/or frequency domain and/or spatial domain. ;
  • the terminal device can report its own speed information, movement direction information, etc. to the network device, and the present application is not limited to this.
  • it can also be other information that changes in the time dimension, frequency dimension, and space dimension on the channel.
  • the network device generates measurement resource configuration information and/or reports resource configuration information according to the channel change indication information
  • the AI/ML model runs on the terminal device and is used for beam management or beam prediction; the network device monitors the uplink signal from the terminal device; and analyzes the uplink signal based on the uplink signal. Perform performance evaluation on the above AI/ML model.
  • Figure 5 is a schematic diagram of beam management or beam prediction according to an embodiment of the present application. As shown in Figure 5, the method includes:
  • the network device receives the number of repeated beam transmissions or the number of CSI-RS repeated transmissions sent by the terminal device;
  • the network device sends a reference signal according to the number of beam re-transmissions or the number of CSI-RS re-transmissions;
  • the terminal device uses the AI/ML model to perform beam estimation to obtain beam estimation information;
  • the terminal device uses receiving beams in different receiving directions to receive reference signals in the same direction or the same CSI-RS from the network device.
  • the terminal equipment uses the AI/ML model to estimate the best receiving beam direction, or the RS identifier of the airspace QCL of the best beam direction.
  • the network device receives the beam estimation information reported by the terminal device
  • the beam estimation information includes QCL information, RS identification, etc., and the present application is not limited thereto.
  • the network device receives the uplink signal sent by the terminal device; wherein the uplink signal is generated by the terminal device based on the beam estimation information.
  • the method may also include:
  • the network device can monitor and perform performance evaluation of the AI/ML model based on the uplink signal.
  • the network device compares the strength of the uplink signal generated based on the beam estimation information with the strength of the uplink signal in other directions to perform performance evaluation of the AI/ML model. For example, if the comparison result satisfies a predetermined condition, it may be determined that the beam information estimated or predicted via the AI/ML model is inaccurate, and thus the AI/ML model may be stopped.
  • the method may also include:
  • the network device sends a stop instruction to the terminal device to stop the AI/ML model.
  • the network device side can perform AI/ML model monitoring based on the beam information of the opposite end.
  • Terminal equipment can use AI/ML models for beam management, specifically using AI/ML models to achieve rapid selection or estimation of beam directions.
  • the terminal device may send to the network device the number of repeated transmissions corresponding to one beam, or the number of repeated transmissions of one CSI-RS.
  • Figure 6 is an example diagram of beam estimation or prediction according to the embodiment of the present application.
  • the terminal device can use receiving beams in different receiving directions (shown as 601 and 604 in Figure 6) to receive reference signals in the same direction or the same CSI-RS from the network device.
  • the terminal device receives the reference signal sent by the network device, uses the AI/ML model to estimate the intensity of the unsent beam (shown as 602 and 603 in Figure 6), and selects the strongest beam direction (shown in Figure 6) 602).
  • the terminal device can obtain the maximum beam direction, which saves 50% of the overhead compared to the traditional method of repeatedly sending four beams.
  • the terminal device can report to the network device the airspace QCL information of its selected beam direction, such as SSB ID or CSI-RS ID.
  • the network device can evaluate the performance of the model on the terminal device side by monitoring the strength of the uplink signal or the strength of the downlink signal corresponding to the beam direction. When a certain condition is met, the network device determines that the AI performance on the terminal device side is not good, and can notify the terminal device to stop running the AI/ML model.
  • the AI/ML model runs on the terminal device and is used for beam management or beam prediction; the network device monitors HARQ feedback information from the terminal device; and based on the HARQ feedback information Perform performance evaluation of the AI/ML model.
  • the network device configures and instructs the terminal device to perform beam prediction; wherein the terminal device uses the AI/ML model to perform beam prediction; receives the beam prediction information reported by the terminal device; and according to the report sending downlink information using the beam prediction information; and receiving HARQ feedback information sent by the terminal device; wherein the HARQ feedback information is generated by the terminal device based on the downlink information.
  • the network device receives channel change indication information sent by the terminal device, and the channel change indication information includes the channel between the network device and the terminal device in the time domain and/or frequency domain. and/or change information in the air domain; generate beam prediction configuration according to the channel change indication information; send the beam prediction configuration information to the terminal device.
  • the terminal device uses an AI/ML model to perform beam prediction.
  • the terminal device estimates the beam direction for a future period, or the QCL indication related to the transmission beam, and sends this information to the network device.
  • the terminal device can also send channel time domain change information to the network device, and the network device adjusts the density of the measurement RS accordingly.
  • the AI/ML model of the terminal device predicts the future beam direction or QCL information based on the RS and previous beam estimation results, and sends the corresponding RS ID to the network device; the network device refers to this information and uses the QCL direction at the relevant moment. Beam is sent.
  • the network device judges the performance of the AI/ML model by monitoring the HARQ feedback information of the terminal device. When a certain condition is met, the network device determines that the performance of the AI/ML model is not good, and notifies the terminal device to stop running the AI/ML model.
  • the network device schedules the terminal device to send an uplink SRS, and the SRS and the RS corresponding to other downlink directions are QCL.
  • the network device compares the strength of the SRS in other directions with the SRS in the downlink direction selected by the terminal device, and uses the reciprocity of the uplink and downlink channels to evaluate the performance of the AI/ML model regarding beam management or prediction.
  • the network device determines that the performance of the AI/ML model on the terminal device side is not good, and notifies the terminal device to stop running the AI/ML model.
  • the network device monitors or performs performance evaluation of the AI/ML model in the network device.
  • the network device stops the AI/ML model. ML model.
  • FIG. 7 is a schematic diagram of an AI/ML model for network equipment monitoring network equipment according to an embodiment of the present application.
  • the performance of AI/ML on the network device side can be realized by the model monitoring module within the network device.
  • the model monitoring module can configure or schedule terminal equipment to send model monitoring signals for network equipment to monitor the model.
  • the model monitoring module can also monitor the model through feedback information from the terminal device.
  • the network device determines or infers that the processing performance of the AI/ML model is not good, and the network device stops using the AI/ML model internally and uses non-AI processing methods instead.
  • the signal or information sent by the terminal device includes at least one of the following: sounding reference signal (SRS), reference signal received power (RSRP), HARQ feedback information, beam failure request information, beam failure recovery (BFR) )information.
  • SRS sounding reference signal
  • RSRP reference signal received power
  • BFR beam failure recovery
  • network equipment uses AI/ML models for downlink beam estimation.
  • the network device can send a small number of beams to the terminal device, report information through the RSRP of the terminal device and use the AI/ML model to estimate the intensity of the unsent beam direction and further find the maximum beam direction.
  • the use of AI/ML speeds up beam scanning and reduces overhead.
  • Network equipment can judge the accuracy of its AI/ML model for downlink beam estimation by measuring the uplink signal strength in the selected maximum beam direction and other beam directions, such as the strength of the SRS signal.
  • the reciprocity of the uplink and downlink channels can be used.
  • the model monitoring signal in Figure 7 may be an SRS signal.
  • the network device uses the AI/ML model to select the best beam
  • uses the beam to send data to the terminal device. If a beam failure occurs in the terminal device, it will perform beam failure recovery (BFR).
  • BFR beam failure recovery
  • the network device monitors the quality of the AI/ML model based on the statistics of beam failure recovery requests. When the performance fails to reach a certain indicator, the network device determines that the performance of the AI/ML model is not good, and the network device stops using the AI/ML model internally and uses non-AI processing methods instead.
  • network equipment uses AI/ML models for beam management. Specifically, the network device sends a set of beams (more than one beam) or a related set of CSI-RS, and then the network device receives the measurement results of the terminal device for the above signals. The network device uses the AI/ML model to generate an estimate of the unsent beam based on the measurement results, and then selects the best sending beam for the terminal device.
  • the network device can configure relevant AI/ML monitoring configuration and reporting configuration for the terminal device.
  • the terminal device reports AI/ML related performance information based on these configurations.
  • the network device may configure the RS configuration corresponding to one or more AI/ML-selected beams for the terminal device, and/or configure related metrics and thresholds.
  • the terminal device may be based on criteria for the beam failure determination process. Based on the measurement results of the beam corresponding to Beam-Failure-Detection-RS-ResourceConfig, the beam condition is judged and the beam failure recovery process is performed. Network equipment can evaluate the status of BFR. When certain conditions are met (such as too many beam failures per unit time, etc.), the network equipment determines that the performance of the AI/ML is not good and stops using the AI/ML model.
  • network equipment uses AI/ML models for uplink beam selection.
  • Network equipment can schedule terminal equipment to send a small number of SRS or beams, and use AI/ML to estimate the optimal terminal uplink beam direction.
  • the network device can send downlink beams and let the terminal device report RSRP to verify whether the direction selected by its AI/ML is the best direction, thereby judging the performance of the model.
  • the model monitoring signal in Figure 7 is the RSRP information reported based on the measurement of the CSI-RS signal for the downlink beam.
  • the above is a schematic explanation of the AI/ML model for network equipment monitoring.
  • the following is a schematic explanation of the monitoring of the AI/ML model in a certain cell or a certain area.
  • the AI/ML model is generally very robust, and model stoppage is relatively rare.
  • its AI/ML model generally does not serve one user, and the performance feedback from the user may not be accurate enough.
  • the network device side needs to make a decision based on feedback information from multiple terminal devices. In the same way, the network device side cannot determine that there is a performance problem with a certain model on the terminal device side by monitoring a certain model of the terminal device. It is necessary to stop using the same type of model when the performance of the same type of model in multiple terminal devices does not meet the requirements.
  • the network device monitors or evaluates the performance of the AI/ML model in the cell. When the number of terminal devices whose performance meets the predetermined conditions of the AI/ML model reaches a threshold, the network device Determine to stop the AI/ML model.
  • the network device monitors output signals or output-related signals of the AI/ML model of one or more terminal devices. When it is determined that the number of terminal devices with poor performance reaches a certain condition, the network device determines that this type of AI/ML model is not suitable for application in terminal devices in this community.
  • the network device monitors the AI/ML performance on the terminal device side through metric information such as HARQ NACK or beam failure recovery.
  • metric information such as HARQ NACK or beam failure recovery.
  • the network device Decide to stop AI/ML on the terminal device side, and notify all terminal devices using the AI/ML model to stop using the AI/ML model.
  • the network device broadcasts the identification information of the AI/ML model whose performance in the cell is lower than a threshold in the cell through system information, and/or broadcasts the identification information of the AI/ML model whose performance in the cell is higher than the threshold.
  • the identification information of the AI/ML model is broadcast in the cell through system information.
  • the network device includes the AI/ML model identification with poor performance into the exclusion list of the AI/ML model of the cell corresponding to the network device, and broadcasts it as system information in the cell.
  • the exclusion list includes AI model function identifiers and corresponding AI model identifiers.
  • it can further include the AI model group identification, intra-group identification, etc. corresponding to a certain AI function.
  • the model identification of the AI/ML model on the terminal device side that is identified by other network devices through model monitoring as having poor performance and needs to be deactivated can also be included in the exclusion list of the community AI/ML model as above.
  • the network device can also accumulate a list of models with particularly good AI/ML performance for terminal devices, put them into the allowed list of AI/ML models in the cell, and broadcast them in the cell as system information.
  • the allowed list includes AI model function identifiers and corresponding AI model identifiers.
  • it can further include the AI model group ID, intra-group ID, etc. corresponding to a certain AI function.
  • the network device configures cell-specific configuration information for one or more terminal devices, so that the one or more terminal devices feed back information for monitoring the AI/ML model according to the configuration information. Signal or message.
  • network equipment uses cell-specific configuration information to configure model monitoring configuration information for terminal equipment in the community, which is used to allow terminal equipment to feed back performance feedback or monitoring information for a certain functional AI.
  • This configuration may include measurement configuration, reporting configuration, and monitoring indicator configuration.
  • the monitoring indicator configuration may include monitored metric information, such as RSRP, HARQ-NACK, a counter of a certain event, a timer, and threshold information of the corresponding indicator.
  • the measurement information may include the measured RS type, such as SSB, CSI-RS, etc., and measurement resource configuration.
  • Reporting configuration includes reporting methods and resources, such as periodic reporting (periodic report), semi-persistent reporting (semi-persistent report), aperiodic reporting (aperiodic report), etc.
  • the terminal device After receiving the model monitoring configuration information, the terminal device can report according to the configuration.
  • the network device determines the performance of the AI/ML model based on the reported information received. Through the statistics reported by multiple terminal devices and reaching certain conditions, the network device determines that the performance of the AI/ML model is poor and stops the AI/ML model.
  • the network device sends the identification information of the AI/ML model whose performance in the cell is lower than the threshold to other cells or core network devices, and/or sends the identification information of the AI/ML model whose performance in the cell is higher than the threshold.
  • the identification information of the AI/ML model of the threshold is sent to other cells or core network equipment.
  • the exclusion list and/or the allow list of the AI/ML model can be sent to adjacent network devices or to the core network device.
  • the network device can recommend or share the model identifier to neighboring cells or core networks.
  • the network device monitors or evaluates the performance of the AI/ML model in the network device and/or the terminal device based on signals or information from the terminal device.
  • the operation of the AI/ML model can be monitored, the consistency of the AI/ML model operation can be maintained, and the robustness of the model operation can be improved.
  • the embodiment of the present application provides an AI monitoring method, which is explained from the terminal device side, and the same content as the embodiment of the first aspect will not be described again.
  • the AI/ML model is monitored by the terminal device.
  • FIG 8 is a schematic diagram of the AI monitoring method according to the embodiment of the present application. As shown in Figure 8, the method includes:
  • the terminal device receives the signal or information sent by the network device
  • the terminal device monitors or performs performance evaluation on the network device and/or the AI/ML model in the terminal device according to the signal or information.
  • the terminal device monitors or evaluates the performance of the AI/ML model in the network device. When the performance of the AI/ML model meets predetermined conditions, the terminal device reports to the network device Send a request message to stop the AI/ML model.
  • FIG. 9 is a schematic diagram of an AI/ML model for terminal equipment monitoring network equipment according to an embodiment of the present application.
  • the network device uses AI/ML to implement a certain function, and the terminal device can judge the performance of the AI/ML model on the network device side by receiving signals or information. When the performance fails to meet a certain indicator or condition, the terminal device determines that the performance of the AI/ML model is not good.
  • the terminal device may not be able to directly instruct the network device to stop its AI/ML model. Instead, it may send the relevant AI/ML model to the network device. A request to stop, and/or, a request to start related non-AI processing.
  • the request information includes at least one of the following: a stop request of the AI/ML model, identification information of the AI/ML model or model group, and corresponding signal processing of the AI/ML model. Function identification, non-AI processing start request, and non-AI processing identification information.
  • the terminal device receives configuration information for AI/ML monitoring sent by the network device.
  • the network device configures the measurement configuration, reporting configuration, and monitoring indicator configuration for the terminal device for model monitoring.
  • the monitoring indicator configuration may include monitored metric information, such as RSRP, HARQ-NACK, a counter of a certain event, a timer, and threshold information of the corresponding indicator.
  • the measurement information may include the measured RS type, such as SSB, CSI-RS, etc., and measurement resource configuration.
  • Reporting configuration includes reporting methods and resources, such as periodic reporting (periodic report), semi-persistent reporting (semi-persistent report), aperiodic reporting (aperiodic report), etc.
  • the terminal device monitors or performs performance evaluation of the AI/ML model in the terminal device, and when the performance of the AI/ML model meets predetermined conditions, the terminal device reports to the The network device sends a request message to stop the AI/ML model.
  • FIG 10 is a schematic diagram of an AI/ML model for terminal equipment monitoring terminal equipment according to an embodiment of the present application.
  • the terminal device uses AI/ML to implement a certain function.
  • the terminal device can judge the performance of the AI/ML model on the terminal device side by receiving signals or information from the network device. When the performance fails to meet a certain indicator or condition, the terminal device determines that the performance of the AI/ML model is not good. For example, the terminal device cannot directly stop its AI/ML model, but can send a request to the network device to stop the relevant AI/ML model. , and/or, send a request to enable related non-AI processing.
  • the terminal device receives an AI capability query request sent by the network device; and feeds back AI/ML capabilities and/or AI monitoring capabilities to the network device.
  • the network device inquires about AI/ML related capabilities on the terminal device side, including one or more of the following: AI function support inquiry, AI model monitoring capability inquiry, AI corresponding processing function inquiry, etc.
  • the terminal device can respond or report accordingly according to the query request.
  • the network device configures the measurement configuration, reporting configuration, and monitoring indicator configuration for the terminal device for model monitoring.
  • the monitoring indicator configuration may include monitored metric information, such as RSRP, HARQ-NACK, a counter of a certain event, a timer, and threshold information of the corresponding indicator.
  • the measurement information may include the measured RS type, such as SSB, CSI-RS, etc., and measurement resource configuration.
  • Reporting configuration includes reporting methods and resources, such as periodic reporting (periodic report), semi-persistent reporting (semi-persistent report), aperiodic reporting (aperiodic report), etc.
  • the terminal device monitors or evaluates the performance of the AI/ML model in the network device and/or the terminal device based on signals or information from the network device.
  • the operation of the AI/ML model can be monitored, the consistency of the AI/ML model operation can be maintained, and the robustness of the model operation can be improved.
  • An embodiment of the present application provides an AI monitoring device.
  • the device may be, for example, a network device, or may be some or some components or components configured on the network device. The same content as the embodiment of the first aspect will not be described again.
  • FIG 11 is a schematic diagram of an AI monitoring device according to an embodiment of the present application. As shown in Figure 11, the AI monitoring device 1100 includes:
  • Receiving unit 1101 which receives signals or information sent by the terminal device.
  • the processing unit 1102 is configured to monitor or perform performance evaluation of the AI/ML model in the network device and/or the terminal device according to the signal or information.
  • the AI/ML model in the terminal device is monitored or performance evaluated by a network device.
  • the device also includes:
  • the sending unit 1103 is configured to send instruction information to stop the AI/ML model to the terminal device when the performance of the AI/ML model meets a predetermined condition.
  • the signal or information sent by the terminal device is a signal or information related to the AI/ML model in the terminal device for a certain signal processing function.
  • the indication information also includes identification information of the AI/ML model, and/or the indication information also instructs the terminal device to switch to non-AI/ML processing corresponding to a certain signal processing function. ;
  • the identification information of the AI/ML model includes at least one of the following: the signal processing function identification corresponding to the AI/ML model, the identification of the AI/ML model, the model group identification of the AI/ML model, the The group identifier of the AI/ML model.
  • the AI/ML model runs on the terminal device and is used for channel state information (CSI) estimation or prediction; the network device responds to hybrid automatic repeat request (HARQ) from the terminal device. ) feedback information is monitored, and performance evaluation of the AI/ML model is performed based on the HARQ feedback information.
  • CSI channel state information
  • HARQ hybrid automatic repeat request
  • the network device configures and instructs the terminal device to perform CSI estimation and reporting; wherein the terminal device uses the AI/ML model to perform CSI estimation or prediction; and receives the CSI reported by the terminal device; Send downlink information according to the reported CSI; and receive HARQ feedback information sent by the terminal device; wherein the HARQ feedback information is generated by the terminal device based on the downlink information.
  • the network device receives channel change indication information sent by the terminal device, and the channel change indication information includes the channel between the network device and the terminal device in the time domain and/or frequency domain. and/or change information in the airspace; generating measurement resource configuration information and/or reporting resource configuration information according to the channel change indication information; sending the measurement resource configuration information and/or the reporting resource configuration information to the terminal device .
  • the AI/ML model runs on the terminal device and is used for beam management or beam prediction; the network device monitors the uplink signal from the terminal device; and analyzes the uplink signal based on the uplink signal. Perform performance evaluation on the above AI/ML model.
  • the network device receives the number of beam re-transmissions or the number of CSI-RS re-transmissions sent by the terminal device; and sends a reference signal according to the number of beam re-transmissions or the number of CSI-RS re-transmissions; wherein the The terminal equipment performs beam estimation using the AI/ML model to obtain beam estimation information; receives the beam estimation information reported by the terminal equipment; and receives an uplink signal sent by the terminal equipment; wherein the uplink signal is generated by the The terminal device generates it based on the beam estimation information.
  • the AI/ML model runs on the terminal device and is used for beam management or beam prediction; the network device monitors HARQ feedback information from the terminal device; and based on the HARQ feedback information Perform performance evaluation of the AI/ML model.
  • the network device configures and instructs the terminal device to perform beam prediction; wherein the terminal device uses the AI/ML model to perform beam prediction; receives the beam prediction information reported by the terminal device; and according to the report sending downlink information using the beam prediction information; and receiving HARQ feedback information sent by the terminal device; wherein the HARQ feedback information is generated by the terminal device based on the downlink information.
  • the network device monitors or performs performance evaluation of the AI/ML model in the network device.
  • the network device stops the AI. /ML model.
  • the signal or information sent by the terminal device includes at least one of the following: sounding reference signal (SRS), reference signal received power (RSRP), HARQ feedback information, beam failure request information, beam failure recovery (BFR) )information.
  • SRS sounding reference signal
  • RSRP reference signal received power
  • BFR beam failure recovery
  • the network device monitors or evaluates the performance of the AI/ML model in the cell. When the number of terminal devices whose performance of the AI/ML model meets predetermined conditions reaches a threshold, the network device determines Stop the AI/ML model.
  • the network device broadcasts the identification information of the AI/ML model whose performance in the cell is lower than a threshold in the cell through system information, and/or broadcasts the identification information of the AI/ML model whose performance in the cell is higher than the threshold.
  • the identification information of the AI/ML model of the threshold is broadcast in the cell through system information.
  • the network device configures cell-specific configuration information for one or more terminal devices, so that the one or more terminal devices provide feedback for monitoring AI/ML based on the configuration information. Signal or information from the model.
  • the network device sends the identification information of the AI/ML model whose performance in the cell is lower than the threshold to other cells or core network devices, and/or sends the identification information of the AI/ML model whose performance in the cell is higher than the threshold.
  • the identification information of the AI/ML model of the threshold is sent to other cells or core network equipment.
  • the AI monitoring device 1100 may also include other components or modules.
  • the specific contents of these components or modules please refer to related technologies.
  • FIG. 11 only illustrates the connection relationships or signal directions between various components or modules, but it should be clear to those skilled in the art that various related technologies such as bus connections can be used.
  • Each of the above components or modules can be implemented by hardware facilities such as a processor, a memory, a transmitter, a receiver, etc.; the implementation of this application is not limited to this.
  • the network device monitors or evaluates the performance of the AI/ML model in the network device and/or the terminal device based on signals or information from the terminal device.
  • the operation of the AI/ML model can be monitored, the consistency of the AI/ML model operation can be maintained, and the robustness of the model operation can be improved.
  • An embodiment of the present application provides an AI monitoring device.
  • the device may be, for example, a terminal device, or may be one or some parts or components configured in the terminal device.
  • the same content as the embodiments of the first and second aspects will not be described again.
  • FIG 12 is a schematic diagram of the AI monitoring device according to the embodiment of the present application. As shown in Figure 12, the AI monitoring device 1200 includes:
  • Receiving unit 1201 which receives signals or information sent by network devices.
  • the processing unit 1202 is configured to monitor or perform performance evaluation of the AI/ML model in the network device and/or terminal device according to the signal or information.
  • the terminal device monitors or performs performance evaluation on the AI/ML model in the network device.
  • the terminal device reports to the The network device sends a request message to stop the AI/ML model.
  • the terminal device monitors or performs performance evaluation of the AI/ML model in the terminal device, and when the performance of the AI/ML model meets predetermined conditions, the terminal device reports to the The network device sends a request message to stop the AI/ML model.
  • the AI monitoring device 1200 may also include other components or modules.
  • the specific contents of these components or modules please refer to related technologies.
  • FIG. 12 only illustrates the connection relationships or signal directions between various components or modules, but it should be clear to those skilled in the art that various related technologies such as bus connections can be used.
  • Each of the above components or modules can be implemented by hardware facilities such as a processor, a memory, a transmitter, a receiver, etc.; the implementation of this application is not limited to this.
  • the terminal device monitors or evaluates the performance of the AI/ML model in the network device and/or the terminal device based on signals or information from the network device.
  • the operation of the AI/ML model can be monitored, the consistency of the AI/ML model operation can be maintained, and the robustness of the model operation can be improved.
  • An embodiment of the present application also provides a communication system. Refer to FIG. 1 . Contents that are the same as those in the first to fourth embodiments will not be described again.
  • communication system 100 may include at least:
  • Terminal device 102 which receives signals or information sent by network devices; and monitors or performs performance evaluation on the network device and/or the AI/ML model in the terminal device according to the signals or information; and/or
  • Network device 101 which receives signals or information sent by terminal devices; and monitors or performs performance evaluation on the network device and/or the AI/ML model in the terminal device according to the signals or information.
  • the embodiment of the present application also provides a network device, which may be a base station, for example, but the present application is not limited thereto and may also be other network devices.
  • a network device which may be a base station, for example, but the present application is not limited thereto and may also be other network devices.
  • Figure 13 is a schematic diagram of the structure of a network device according to an embodiment of the present application.
  • the network device 1300 may include a processor 1310 (eg, a central processing unit CPU) and a memory 1320 ; the memory 1320 is coupled to the processor 1310 .
  • the memory 1320 can store various data; in addition, it also stores an information processing program 1330, and the program 1330 is executed under the control of the processor 1310.
  • the processor 1310 may be configured to execute a program to implement the AI monitoring method as described in the embodiment of the first aspect.
  • the processor 1310 may be configured to perform the following control: receive a signal or information sent by a terminal device; and monitor the network device and/or the AI/ML model in the terminal device according to the signal or information, or Performance evaluation.
  • the network device 1300 may also include: a transceiver 1340, an antenna 1350, etc.; the functions of the above components are similar to those of the existing technology and will not be described again here. It is worth noting that the network device 1300 does not necessarily include all components shown in Figure 13; in addition, the network device 1300 may also include components not shown in Figure 13, and reference may be made to the existing technology.
  • the embodiment of the present application also provides a terminal device, but the present application is not limited to this and may also be other devices.
  • Figure 14 is a schematic diagram of a terminal device according to an embodiment of the present application.
  • the terminal device 1400 may include a processor 1410 and a memory 1420; the memory 1420 stores data and programs and is coupled to the processor 1410. It is worth noting that this figure is exemplary; other types of structures may also be used to supplement or replace this structure to implement telecommunications functions or other functions.
  • the processor 1410 may be configured to execute a program to implement the AI monitoring method as described in the embodiment of the second aspect.
  • the processor 1410 may be configured to perform the following control: receive signals or information sent by a network device; and monitor the AI/ML model in the network device and/or the terminal device according to the signal or information, or Performance evaluation.
  • the terminal device 1400 may also include: a communication module 1430, an input unit 1440, a display 1450, and a power supply 1460.
  • the functions of the above components are similar to those in the prior art and will not be described again here. It is worth noting that the terminal device 1400 does not necessarily include all the components shown in Figure 14, and the above components are not required; in addition, the terminal device 1400 can also include components not shown in Figure 14, please refer to the current There is technology.
  • An embodiment of the present application also provides a computer program, wherein when the program is executed in a terminal device, the program causes the terminal device to execute the AI monitoring method described in the embodiment of the second aspect.
  • An embodiment of the present application also provides a storage medium storing a computer program, wherein the computer program causes the terminal device to execute the AI monitoring method described in the embodiment of the second aspect.
  • An embodiment of the present application also provides a computer program, wherein when the program is executed in a network device, the program causes the network device to execute the AI monitoring method described in the embodiment of the first aspect.
  • An embodiment of the present application also provides a storage medium storing a computer program, wherein the computer program causes the network device to execute the AI monitoring method described in the embodiment of the first aspect.
  • the above devices and methods of this application can be implemented by hardware, or can be implemented by hardware combined with software.
  • the present application relates to a computer-readable program that, when executed by a logic component, enables the logic component to implement the apparatus or component described above, or enables the logic component to implement the various methods described above or steps.
  • This application also involves storage media used to store the above programs, such as hard disks, magnetic disks, optical disks, DVDs, flash memories, etc.
  • the methods/devices described in connection with the embodiments of the present application may be directly embodied as hardware, a software module executed by a processor, or a combination of both.
  • one or more of the functional block diagrams and/or one or more combinations of the functional block diagrams shown in the figure may correspond to each software module of the computer program flow, or may correspond to each hardware module.
  • These software modules can respectively correspond to the various steps shown in the figure.
  • These hardware modules can be implemented by solidifying these software modules using a field programmable gate array (FPGA), for example.
  • FPGA field programmable gate array
  • the software module may be located in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.
  • a storage medium may be coupled to the processor such that the processor can read information from the storage medium and write information to the storage medium; or the storage medium may be an integral part of the processor.
  • the processor and storage media may be located in an ASIC.
  • the software module can be stored in the memory of the mobile terminal or in a memory card that can be inserted into the mobile terminal.
  • the software module can be stored in the MEGA-SIM card or the large-capacity flash memory device.
  • One or more of the functional blocks and/or one or more combinations of the functional blocks described in the accompanying drawings may be implemented as a general-purpose processor or a digital signal processor (DSP) for performing the functions described in this application. ), application specific integrated circuit (ASIC), field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, or any appropriate combination thereof.
  • DSP digital signal processor
  • ASIC application specific integrated circuit
  • FPGA field programmable gate array
  • One or more of the functional blocks and/or one or more combinations of the functional blocks described in the accompanying drawings can also be implemented as a combination of computing devices, for example, a combination of a DSP and a microprocessor, or multiple microprocessors. processor, one or more microprocessors combined with DSP communications, or any other such configuration.
  • An AI monitoring method including:
  • Network equipment receives signals or information sent by terminal equipment
  • the network device sends instruction information to stop the AI/ML model to the terminal device.
  • the indication information also includes identification information of the AI/ML model, and/or the indication information also instructs the terminal device to switch to a signal processing corresponding to a certain Non-AI/ML processing of features.
  • the identification information of the AI/ML model includes at least one of the following: a signal processing function identification corresponding to the AI/ML model, an identification of the AI/ML model, The model group identifier of the AI/ML model and the intra-group identifier of the AI/ML model.
  • the network device monitors Hybrid Automatic Repeat Request (HARQ) feedback information from the terminal device, and performs performance evaluation on the AI/ML model based on the HARQ feedback information.
  • HARQ Hybrid Automatic Repeat Request
  • the network device configures and instructs the terminal device to perform CSI estimation and reporting; wherein the terminal device uses the AI/ML model to perform CSI estimation or prediction;
  • the network device receives the channel change indication information sent by the terminal device.
  • the channel change indication information includes the channel between the network device and the terminal device in the time domain and/or frequency domain and/or spatial domain. change information;
  • the network device monitors the uplink signal from the terminal device; and performs performance evaluation on the AI/ML model based on the uplink signal.
  • the network device receives the number of repeated beam transmissions or the number of repeated CSI-RS transmissions sent by the terminal device;
  • the network device monitors HARQ feedback information from the terminal device; and performs performance evaluation on the AI/ML model based on the HARQ feedback information.
  • the network device configures and instructs the terminal device to perform beam prediction; wherein the terminal device uses the AI/ML model to perform beam prediction;
  • the network device receives the channel change indication information sent by the terminal device.
  • the channel change indication information includes the channel between the network device and the terminal device in the time domain and/or frequency domain and/or spatial domain. change information;
  • the network device stops the AI/ML model.
  • the signal or information sent by the terminal equipment includes at least one of the following: sounding reference signal (SRS), reference signal received power (RSRP), HARQ feedback information, beam failure request Information, Beam Failure Recovery (BFR) information.
  • SRS sounding reference signal
  • RSRP reference signal received power
  • BFR Beam Failure Recovery
  • the network device broadcasts the identification information of the AI/ML model whose performance in the cell is lower than the threshold in the cell through system information, and/or broadcasts the identification information of the AI/ML model whose performance is higher than the threshold in the cell.
  • the identification information of the model is broadcast in the cell through system information.
  • the network device configures cell-specific configuration information for one or more terminal devices, so that the one or more terminal devices feed back signals or information for monitoring the AI/ML model according to the configuration information.
  • the network device sends identification information of AI/ML models whose performance in the cell is lower than the threshold to other cells or core network devices, and/or sends AI/ML models whose performance in the cell is higher than the threshold.
  • the identification information of the ML model is sent to other cells or core network equipment.
  • An AI monitoring method including:
  • the terminal device receives signals or information sent by the network device.
  • the terminal device When the performance of the AI/ML model meets the predetermined condition, the terminal device sends request information to stop the AI/ML model to the network device.
  • the request information includes at least one of the following: a stop request of the AI/ML model, identification information of the AI/ML model or model group, the AI/ML model The identification of the signal processing function corresponding to the ML model, the start request of non-AI processing, and the identification information of non-AI processing.
  • the terminal device receives configuration information for AI/ML monitoring sent by the network device.
  • the terminal device When the performance of the AI/ML model meets the predetermined condition, the terminal device sends request information to stop the AI/ML model to the network device.
  • the terminal device receives the AI capability query request sent by the network device.
  • a network device including a memory and a processor, the memory stores a computer program, and the processor is configured to execute the computer program to implement the AI monitoring method as described in any one of appendices 1 to 20 .
  • a terminal device including a memory and a processor, the memory stores a computer program, and the processor is configured to execute the computer program to implement the AI monitoring method as described in any one of appendices 21 to 26 .

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Medical Informatics (AREA)
  • Artificial Intelligence (AREA)
  • Electromagnetism (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)
  • Debugging And Monitoring (AREA)

Abstract

本申请实施例提供一种AI监测装置以及方法。所述方法包括：网络设备接收终端设备发送的信号或信息；以及根据所述信号或信息对所述网络设备和/或所述终端设备中的AI/ML模型进行监测或性能评估。

Description

AI监测装置以及方法 技术领域
本申请实施例涉及通信技术领域。
背景技术
随着低频段频谱资源变得稀缺，毫米波频段能够提供更大带宽，成为了5G新无线(NR，New Radio)系统的重要频段。毫米波由于波长较短，具有与传统低频段不同的传播特性，例如更高的传播损耗，反射和衍射性能差等。因此通常会采用更大规模的天线阵列，以形成增益更大的赋形波束，克服传播损耗，确保系统覆盖。
伴随着人工智能(AI，Artificial Intelligence)和机器学习(ML，Machine Learning)技术的发展，将AI/ML技术应用到无线通信上，来解决传统方法的难点成为当前一个技术方向。AI/ML模型应用于无线通信系统，特别是应用于空口的传输是5G-Advanced以及6G阶段的新技术。
例如，对于信道状态信息(CSI，Channel State Information)上报而言，在终端设备侧使用AI编码器(AI encoder)对CSI进行编码/压缩，在网络设备侧使用AI解码器(AI decoder)对CSI进行解码/解压缩，能够降低反馈开销。再例如，对于波束管理(Beam Management)而言，利用AI/ML模型，根据少量波束测量的结果预测出空间上最优的波束对，能够减少系统的负荷和延时。
应该注意，上面对技术背景的介绍只是为了方便对本申请的技术方案进行清楚、完整的说明，并方便本领域技术人员的理解而阐述的。不能仅仅因为这些方案在本申请的背景技术部分进行了阐述而认为上述技术方案为本领域技术人员所公知。
发明内容
但是，发明人发现：作为基于数据集训练的AI/ML模型，如何适应多种无线应用的需求，如何应对千变万化的移动通信环境，对AI/ML方案本身带来很大挑战。对于丰富的无线通信场景，比如郊区、城区、室内、工厂、矿山等，离线训练好的AI/ML模型很难确保在各种情况下保持一致的性能。因此需要监视AI/ML模型运行的性能，在必要时停止某个AI/ML模型的使用。
针对上述问题的至少之一，本申请实施例提供一种AI监测装置以及方法。
根据本申请实施例的一个方面，提供一种AI监测方法，包括：
网络设备接收终端设备发送的信号或信息；以及
所述网络设备根据所述信号或信息对所述网络设备和/或所述终端设备中的AI/ML模型进行监测或性能评估。
根据本申请实施例的另一个方面，提供一种AI监测装置，包括：
接收单元，其接收终端设备发送的信号或信息；以及
处理单元，其根据所述信号或信息对所述网络设备和/或所述终端设备中的AI/ML模型进行监测或性能评估。
根据本申请实施例的另一个方面，提供一种AI监测方法，包括：
终端设备接收网络设备发送的信号或信息；以及
所述终端设备根据所述信号或信息对所述网络设备和/或所述终端设备中的AI/ML模型进行监测或性能评估。
根据本申请实施例的另一个方面，提供一种AI监测装置，包括：
接收单元，其接收网络设备发送的信号或信息；以及
处理单元，其根据所述信号或信息对所述网络设备和/或所述终端设备中的AI/ML模型进行监测或性能评估。
根据本申请实施例的另一个方面，提供一种通信系统，包括：
终端设备，其接收网络设备发送的信号或信息；以及根据所述信号或信息对所述网络设备和/或所述终端设备中的AI/ML模型进行监测或性能评估；和/或
网络设备，其接收终端设备发送的信号或信息；以及根据所述信号或信息对所述网络设备和/或所述终端设备中的AI/ML模型进行监测或性能评估。
本申请实施例的有益效果之一在于：网络设备根据或来自终端设备的信号或信息对所述网络设备和/或所述终端设备中的AI/ML模型进行监测或性能评估，或者，终端设备根据或来自网络设备的信号或信息对所述网络设备和/或所述终端设备中的AI/ML模型进行监测或性能评估。由此，能够监视AI/ML模型的运行，保持AI/ML模型运行的一致性，提高模型运行的鲁棒性。
参照后文的说明和附图，详细公开了本申请的特定实施方式，指明了本申请的原理可以被采用的方式。应该理解，本申请的实施方式在范围上并不因而受到限制。在 所附权利要求的精神和条款的范围内，本申请的实施方式包括许多改变、修改和等同。
针对一种实施方式描述和/或示出的特征可以以相同或类似的方式在一个或更多个其它实施方式中使用，与其它实施方式中的特征相组合，或替代其它实施方式中的特征。
应该强调，术语“包括/包含”在本文使用时指特征、整件、步骤或组件的存在，但并不排除一个或更多个其它特征、整件、步骤或组件的存在或附加。
附图说明
在本申请实施例的一个附图或一种实施方式中描述的元素和特征可以与一个或更多个其它附图或实施方式中示出的元素和特征相结合。此外，在附图中，类似的标号表示几个附图中对应的部件，并可用于指示多于一种实施方式中使用的对应部件。
图1是本申请实施例的通信系统的示意图；
图2是本申请实施例的AI监测方法的一示意图；
图3是本申请实施例的网络设备监测终端设备的AI/ML模型的一示意图；
图4是本申请实施例的CSI估计和上报的一示意图；
图5是本申请实施例的波束管理或波束预测的一示意图；
图6是本申请实施例的波束估计或预测的一示例图；
图7是本申请实施例的网络设备监测网络设备的AI/ML模型的一示意图；
图8是本申请实施例的AI监测方法的一示意图；
图9是本申请实施例的终端设备监测网络设备的AI/ML模型的一示意图；
图10是本申请实施例的终端设备监测终端设备的AI/ML模型的一示意图；
图11是本申请实施例的AI监测装置的一示意图；
图12是本申请实施例的AI监测装置的另一示意图；
图13是本申请实施例的网络设备的一示意图；
图14是本申请实施例的终端设备的一示意图。
具体实施方式
参照附图，通过下面的说明书，本申请的前述以及其它特征将变得明显。在说明书和附图中，具体公开了本申请的特定实施方式，其表明了其中可以采用本申请的原 则的部分实施方式，应了解的是，本申请不限于所描述的实施方式，相反，本申请包括落入所附权利要求的范围内的全部修改、变型以及等同物。
在本申请实施例中，术语“第一”、“第二”等用于对不同元素从称谓上进行区分，但并不表示这些元素的空间排列或时间顺序等，这些元素不应被这些术语所限制。术语“和/或”包括相关联列出的术语的一种或多个中的任何一个和所有组合。术语“包含”、“包括”、“具有”等是指所陈述的特征、元素、元件或组件的存在，但并不排除存在或添加一个或多个其他特征、元素、元件或组件。
在本申请实施例中，单数形式“一”、“该”等包括复数形式，应广义地理解为“一种”或“一类”而并不是限定为“一个”的含义；此外术语“所述”应理解为既包括单数形式也包括复数形式，除非上下文另外明确指出。此外术语“根据”应理解为“至少部分根据……”，术语“基于”应理解为“至少部分基于……”，除非上下文另外明确指出。
在本申请实施例中，术语“通信网络”或“无线通信网络”可以指符合如下任意通信标准的网络，例如长期演进(LTE，Long Term Evolution)、增强的长期演进(LTE-A，LTE-Advanced)、宽带码分多址接入(WCDMA，Wideband Code Division Multiple Access)、高速报文接入(HSPA，High-Speed Packet Access)等等。
并且，通信系统中设备之间的通信可以根据任意阶段的通信协议进行，例如可以包括但不限于如下通信协议：1G(generation)、2G、2.5G、2.75G、3G、4G、4.5G以及5G、新无线(NR，New Radio)、未来的6G等等，和/或其他目前已知或未来将被开发的通信协议。
在本申请实施例中，术语“网络设备”例如是指通信系统中将终端设备接入通信网络并为该终端设备提供服务的设备。网络设备可以包括但不限于如下设备：基站(BS，Base Station)、接入点(AP、Access Point)、发送接收点(TRP，Transmission Reception Point)、广播发射机、移动管理实体(MME、Mobile Management Entity)、网关、服务器、无线网络控制器(RNC，Radio Network Controller)、基站控制器(BSC，Base Station Controller)等等。
其中，基站可以包括但不限于：节点B(NodeB或NB)、演进节点B(eNodeB或eNB)以及5G基站(gNB)，等等，此外还可包括远端无线头(RRH，Remote Radio Head)、远端无线单元(RRU，Remote Radio Unit)、中继(relay)或者低功率节点(例如femeto、pico等等)。并且术语“基站”可以包括它们的一些或所有功能，每个基站 可以对特定的地理区域提供通信覆盖。术语“小区”可以指的是基站和/或其覆盖区域，这取决于使用该术语的上下文。
在本申请实施例中，术语“用户设备”(UE，User Equipment)或者“终端设备”(TE，Terminal Equipment或Terminal Device)例如是指通过网络设备接入通信网络并接收网络服务的设备。终端设备可以是固定的或移动的，并且也可以称为移动台(MS，Mobile Station)、终端、用户台(SS，Subscriber Station)、接入终端(AT，Access Terminal)、站，等等。
其中，终端设备可以包括但不限于如下设备：蜂窝电话(Cellular Phone)、个人数字助理(PDA，Personal Digital Assistant)、无线调制解调器、无线通信设备、手持设备、机器型通信设备、膝上型计算机、无绳电话、智能手机、智能手表、数字相机，等等。
再例如，在物联网(IoT，Internet of Things)等场景下，终端设备还可以是进行监控或测量的机器或装置，例如可以包括但不限于：机器类通信(MTC，Machine Type Communication)终端、车载通信终端、设备到设备(D2D，Device to Device)终端、机器到机器(M2M，Machine to Machine)终端，等等。
此外，术语“网络侧”或“网络设备侧”是指网络的一侧，可以是某一基站，也可以包括如上的一个或多个网络设备。术语“用户侧”或“终端侧”或“终端设备侧”是指用户或终端的一侧，可以是某一UE，也可以包括如上的一个或多个终端设备。本文在没有特别指出的情况下，“设备”可以指网络设备，也可以指终端设备。
以下通过示例对本申请实施例的场景进行说明，但本申请不限于此。
图1是本申请实施例的通信系统的示意图，示意性说明了以终端设备和网络设备为例的情况，如图1所示，通信系统100可以包括网络设备101和终端设备102、103。为简单起见，图1仅以两个终端设备和一个网络设备为例进行说明，但本申请实施例不限于此。
在本申请实施例中，网络设备101和终端设备102、103之间可以进行现有的业务或者未来可实施的业务发送。例如，这些业务可以包括但不限于：增强的移动宽带(eMBB，enhanced Mobile Broadband)、大规模机器类型通信(mMTC，massive Machine Type Communication)和高可靠低时延通信(URLLC，Ultra-Reliable and Low-Latency Communication)，等等。
值得注意的是，图1示出了两个终端设备102、103均处于网络设备101的覆盖范围内，但本申请不限于此。两个终端设备102、103可以均不在网络设备101的覆盖范围内，或者一个终端设备102在网络设备101的覆盖范围之内而另一个终端设备103在网络设备101的覆盖范围之外。
在本申请实施例中，高层信令例如可以是无线资源控制(RRC)信令；例如称为RRC消息(RRC message)，例如包括MIB、系统信息(system information)、专用RRC消息；或者称为RRC IE(RRC information element)。高层信令例如还可以是MAC(Medium Access Control)信令；或者称为MAC CE(MAC control element)。但本申请不限于此。
在本申请实施例中，网络设备和/或终端设备中可以配置并运行一个或多个AI/ML模型。AI/ML模型可以用于无线通信的各种信号处理功能，例如CSI估计和上报、波束管理和波束预测等等；本申请不限于此。
第一方面的实施例
本申请实施例提供一种AI监测方法，从网络设备侧进行说明。在第一方面的实施例中，由网络设备对AI/ML模型进行监测(monitor)。
图2是本申请实施例的AI监测方法的一示意图，如图2所示，该方法包括：
201，网络设备接收终端设备发送的信号或信息；
202，所述网络设备根据所述信号或信息，对所述网络设备和/或所述终端设备中的AI/ML模型进行监测或性能评估。
值得注意的是，以上附图2仅对本申请实施例进行了示意性说明，但本申请不限于此。例如可以适当地调整各个操作之间的执行顺序，此外还可以增加其他的一些操作或者减少其中的某些操作。本领域的技术人员可以根据上述内容进行适当地变型，而不仅限于上述附图2的记载。
在一些实施例中，针对不同信号处理功能可以分别运行AI/ML模型。例如，针对CSI上报的AI/ML模型，可以具有不同的模型组标识、模型标识和版本标识。针对波束管理的AI/ML模型，可以具有另外的模型组标识、模型标识和版本标识。
在一些实施例中，所述网络设备对所述终端设备中的AI/ML模型进行监测或性能评估，在所述AI/ML模型的性能满足预定条件的情况下，所述网络设备向所述终 端设备发送停止所述AI/ML模型的指示信息。
图3是本申请实施例的网络设备监测终端设备的AI/ML模型的一示意图。如图3所示，网络设备接收可以接收来自终端设备的AI/ML模型的直接输出信号或与该AI模型输出有关的信号，如图3中的反馈信号。
网络设备根据该反馈信号，可以监测终端设备侧的AI/ML模型的性能。当性能达不到某一指标时，网络设备判定或推理(inference)终端设备侧的AI/ML模型的处理性能不好，因此可以发送停止指示给终端设备，由终端设备停止该AI/ML模型。此外，网络设备可以同时或分别指示终端设备切换到非AI处理的传统方式运行，由此保证通信性能的稳定和不间断性。
在一些实施例中，所述终端设备发送的信号或信息为与所述终端设备中的AI/ML模型相关的针对某一信号处理功能的信号或信息。例如，可以是与CSI相关的反馈信息，或者，是与波束管理相关的上行信号，等等。
在一些实施例中，所述指示信息还包括所述AI/ML模型的标识信息，和/或，所述指示信息还指示所述终端设备切换到对应某一信号处理功能的非AI/ML处理。其中，所述AI/ML模型的标识信息包括如下至少之一：所述AI/ML模型对应的信号处理功能标识、所述AI/ML模型的标识、所述AI/ML模型的模型组标识、所述AI/ML模型的组内标识。
例如，考虑多种功能的AI/ML模型可能同时运行，模型监测的ON/OFF信息指示包含有AI模型的标识信息。例如，指示用于CSI反馈的模型组2中的索引为2的AI/ML模型将被停止。
以下先以CSI的估计和上报为例进行说明。
在一些实施例中，所述AI/ML模型运行在终端设备并被用于信道状态信息(CSI)估计或预测；所述网络设备对来自所述终端设备的混合自动重传请求(HARQ)反馈信息进行监测，并根据所述HARQ反馈信息对所述AI/ML模型进行性能评估。
图4是本申请实施例的CSI估计和上报的一示意图，如图4所示，该方法包括：
404，网络设备发送CSI-RS；
405，终端设备进行CSI估计或预测；
其中，网络设备配置和指示所述终端设备进行CSI估计和上报；所述终端设备使用所述AI/ML模型进行CSI估计或预测；
406，网络设备接收所述终端设备上报的CSI；
该上报的CSI可以包括CQI、PMI、RI、CRI等等，本申请不限于此。
407，网络设备根据上报的所述CSI发送下行信息；以及
408，网络设备接收所述终端设备发送的HARQ反馈信息；其中所述HARQ反馈信息由所述终端设备基于所述下行信息而生成。
如图4所示，该方法还可以包括：
409，网络设备根据HARQ ACK/NCK进行AI/ML模型的监测和性能评估。
例如，如果满足预定条件(例如一段时间内NACK反馈的过多)，可以判定经由AI/ML模型估计或预测的CSI不准确，由此可以停止该AI/ML模型。
如图4所示，该方法还可以包括：
410，网络设备向终端设备发送停止AI/ML模型的停止指示。
由此，网络设备侧可以根据对端的CSI进行AI/ML模型监测。
在一些实施例中，如图4所示，该方法还可以包括：
401，网络设备接收终端设备发送的信道变化指示信息，所述信道变化指示信息包括所述网络设备和所述终端设备之间的信道在时域和/或频域和/或空域上的变化信息；
例如，终端设备可以将自身的速度信息、运动方向信息等上报给网络设备，本申请不限于此。此外，还可以是信道上的时间维度、频率维度、空间维度上变化的其他信息。
402，网络设备根据所述信道变化指示信息，生成测量资源配置信息和/或上报资源配置信息；
403，向所述终端设备发送所述测量资源配置信息和/或所述上报资源配置信息。
值得注意的是，以上附图4仅对本申请实施例进行了示意性说明，但本申请不限于此。例如可以适当地调整各个操作之间的执行顺序，此外还可以增加其他的一些操作或者减少其中的某些操作。本领域的技术人员可以根据上述内容进行适当地变型，而不仅限于上述附图4的记载。
以下再以波束管理或波束预测为例进行说明。
在一些实施例中，所述AI/ML模型运行在终端设备并被用于波束管理或波束预测；所述网络设备对来自所述终端设备的上行信号进行监测；并根据所述上行信号对 所述AI/ML模型进行性能评估。
图5是本申请实施例的波束管理或波束预测的一示意图，如图5所示，该方法包括：
501，网络设备接收终端设备发送的波束重复发送数目或CSI-RS重复发送数目；
502，网络设备根据所述波束重复发送数目或CSI-RS重复发送数目发送参考信号；
503，终端设备使用所述AI/ML模型进行波束估计以获得波束估计信息；
例如，终端设备利用不同接收方向的接收波束，接收来自网络设备的同一方向或同一CSI-RS的参考信号。终端设备利用AI/ML模型估计出最佳接收波束方向，或最佳波束方向的空域QCL的RS标识。
504，网络设备接收所述终端设备上报的波束估计信息；
例如，该波束估计信息包括QCL信息、RS标识等等，本申请不限于此。
505，网络设备接收所述终端设备发送的上行信号；其中所述上行信号由所述终端设备基于所述波束估计信息而生成。
如图5所示，所述方法还可以包括：
506，网络设备可以根据该上行信号进行AI/ML模型的监测和性能评估。
例如，网络设备将基于所述波束估计信息生成的上行信号的强度与其他方向的上行信号的强度进行比较，以对所述AI/ML模型进行性能评估。例如，如果比较结果满足预定条件，则可以判定经由AI/ML模型估计或预测的波束信息不准确，由此可以停止该AI/ML模型。
如图5所示，该方法还可以包括：
507，网络设备向终端设备发送停止AI/ML模型的停止指示。
由此，网络设备侧可以根据对端的波束信息进行AI/ML模型监测。
值得注意的是，以上附图5仅对本申请实施例进行了示意性说明，但本申请不限于此。例如可以适当地调整各个操作之间的执行顺序，此外还可以增加其他的一些操作或者减少其中的某些操作。本领域的技术人员可以根据上述内容进行适当地变型，而不仅限于上述附图5的记载。
终端设备可以用AI/ML模型进行波束管理，特别的利用AI/ML模型来实现波束方向的快速选择或估计。终端设备可以发送给网络设备对应一个波束重复发送的数目， 或一个CSI-RS的重复发送数目。
图6是本申请实施例的波束估计或预测的一示例图。如图6所示，终端设备可以利用不同接收方向的接收波束(如图6中的601和604所示)，接收来自网络设备的同一方向或同一CSI-RS的参考信号。终端设备通过接收网络设备发送的参考信号，利用AI/ML模型估计出未发送波束(如图6中的602和603所示)的强度，并选择出最强的波束方向(如图6中的602)。
由于网络设备只发送了两个波束，终端设备就能够获得最大波束方向，比传统方法重复发送四次波束的波束选择，节省了50％的开销。终端设备可以汇报给网络设备其所选择波束方向的空域QCL信息，例如SSB ID或CSI-RS ID。
网络设备可以通过监测对应该波束方向上行信号的强度或下行信号的强度，来评估终端设备侧的模型的性能。当满足某一条件后，网络设备判断终端设备侧的AI性能不好，可以通知终端设备停止该AI/ML模型的运行。
在一些实施例中，所述AI/ML模型运行在终端设备并被用于波束管理或波束预测；所述网络设备对来自所述终端设备的HARQ反馈信息进行监测；并根据所述HARQ反馈信息对所述AI/ML模型进行性能评估。
在一些实施例中，所述网络设备配置和指示所述终端设备进行波束预测；其中所述终端设备使用所述AI/ML模型进行波束预测；接收所述终端设备上报的波束预测信息；根据上报的所述波束预测信息发送下行信息；以及接收所述终端设备发送的HARQ反馈信息；其中所述HARQ反馈信息由所述终端设备基于所述下行信息而生成。
在一些实施例中，所述网络设备接收所述终端设备发送的信道变化指示信息，所述信道变化指示信息包括所述网络设备和所述终端设备之间的信道在时域和/或频域和/或空域上的变化信息；根据所述信道变化指示信息生成波束预测配置；向所述终端设备发送所述波束预测配置信息。
例如，终端设备使用AI/ML模型进行波束预测，终端设备估计出未来一个时段的波束方向，或，与发送波束相关的QCL指示，并将这些信息发送给网络设备。为了预测波束，终端设备还可以发送信道时域变化信息给网络设备，网络设备据此调整测量用RS的密度。
再例如，终端设备的AI/ML模型基于RS和以前波束估计结果，预测未来波束方 向或QCL信息，并将对应RS ID发送给网络设备；网络设备参照这些信息，在相关时刻采用该QCL方向的波束进行发送。
网络设备通过监测终端设备的HARQ反馈信息，来判断AI/ML模型的性能。当满足某一条件后，网络设备判断该AI/ML模型的性能不好，通知终端设备停止该AI/ML模型的运行。
再例如，网络设备调度终端设备发送上行SRS，该SRS与其他下行方向对应的RS是QCL的。网络设备比较其他方向的SRS与终端设备选择的下行方向的SRS的强度，利用上下行信道互易性来评估AI/ML模型有关波束管理或预测的性能。当满足某一条件后，网络设备判断终端设备侧的AI/ML模型的性能不好，通知终端设备停止该AI/ML模型的运行。
在一些实施例中，网络设备对所述网络设备中的AI/ML模型进行监测或性能评估，在所述AI/ML模型的性能满足预定条件的情况下，所述网络设备停止所述AI/ML模型。
图7是本申请实施例的网络设备监测网络设备的AI/ML模型的一示意图。如图7所示，网络设备侧AI/ML的性能可以由网络设备内的模型监测模块实现。模型监测模块可以配置或调度终端设备来发送模型监测信号，用于网络设备对模型的监测。或者，模型监测模块也可以通过终端设备的反馈信息进行模型的监测。当性能达不到某一指标时，网络设备判定或推理(inference)该AI/ML模型的处理性能不好，网络设备内部停止使用AI/ML模型，改用非AI处理的方式。
在一些实施例中，所述终端设备发送的信号或信息包括如下至少之一：探测参考信号(SRS)、参考信号接收功率(RSRP)、HARQ反馈信息、波束失败请求信息、波束失败恢复(BFR)信息。
例如，网络设备使用AI/ML模型进行下行波束估计。网络设备可以通过发送少量的波束给终端设备，通过终端设备的RSRP上报信息并利用AI/ML模型，来估计未发送波束方向的强度，并进一步找出最大波束方向。AI/ML的利用加快了波束扫描的速度也降低了开销。
网络设备可以通过测量所选择的最大波束方向和其他波束方向的上行信号强度，比如SRS信号的强度来判断其AI/ML模型对于下行波束估计的准确度，这里可以利用上下行信道的互易性。图7中的模型监测信号可以是SRS信号。当性能达不到某 一指标时，网络设备判定该AI/ML模型性能不好，网络设备内部停止使用AI/ML模型，改用非AI处理的方式。
或者，在网络设备利用AI/ML模型选择了最佳波束后，网络设备使用该波束向终端设备发送数据。终端设备如果发生波束失败，将进行波束失败恢复(BFR)，网络设备根据波束失败恢复请求的统计，来监测AI/ML模型的质量。当性能达不到某一指标时，网络设备判定该AI/ML模型的性能不好，网络设备内部停止使用AI/ML模型，改用非AI处理的方式。
再例如，网络设备使用AI/ML模型进行波束管理。具体地，网络设备发送一组波束(多于一个波束)或相关的一组CSI-RS，然后网络设备接收终端设备针对上述信号的测量结果。网络设备使用AI/ML模型，基于所述测量结果产生对于未发送波束的估计，进而选择出对于终端设备而言的最佳发送波束。
为了监测AI/ML的性能，网络设备可以为终端设备配置相关的AI/ML监测配置和汇报配置。终端设备根据这些配置来汇报AI/ML相关的性能信息。网络设备可以为终端设备配置一个或多个基于AI/ML选择的波束所对应的RS配置，和/或，配置相关度量(metric)及门限(threshold)。
终端设备可以基于波束失败判断过程的准则。根据对应Beam-Failure-Detection-RS-ResourceConfig的波束的测量结果，来判断波束情况，并进行波束失败恢复(beam failure recovery)的过程。网络设备可以评估BFR的状况，当一定条件满足时(例如单位时间波束失败次数过多等)，网络设备判断该AI/ML的性能不好，并停止使用该AI/ML模型。
再例如，网络设备使用AI/ML模型进行上行波束选择。网络设备可以调度终端设备发送少量的SRS或波束，利用AI/ML来估计出最佳的终端上行波束方向。为了验证模型性能，网络设备可以通过发送下行波束，让终端设备上报RSRP，来验证其AI/ML选择的方向是否是最佳方向，由此判断模型的性能。例如，图7中的模型监测信号是对应下行波束用的CSI-RS信号的测量上报的RSRP信息。当性能达不到某一指标时，网络设备判定该AI/ML模型处理性能不好，网络设备内部停止使用AI/ML模型，改用非AI处理的方式。
以上对于网络设备监测AI/ML模型进行了示意性说明，以下再示意性说明对于某个小区或某个区域内的AI/ML模型的监测。
例如，无论是网络设备侧还是终端设备侧，其AI/ML模型一般是非常鲁棒的，模型停止是比较少出现的情况。对于网络设备侧来说，其AI/ML模型一般不是针对一个用户来服务的，来自用户的性能反馈可能不够准确。
因此，网络设备侧需要根据多个终端设备的反馈信息做出判定。同理，网络设备侧也不能通过一个终端设备的某个模型的监测，就判定终端设备侧某个模型的性能有问题。需要在多个终端设备中的同一类模型的性能都不满足要求的情况下，再停止该类模型的使用。
在一些实施例中，所述网络设备对小区内的AI/ML模型进行监测或性能评估，在所述AI/ML模型的性能满足预定条件的终端设备数目达到阈值的情况下，所述网络设备确定停止所述AI/ML模型。
例如，网络设备监测一个或多个终端设备的AI/ML模型的输出信号或与输出相关的信号。当判断出性能不好的终端设备的数目达到一定条件时，网络设备判断该类AI/ML模型不适合应用在本小区的终端设备中。
例如，网络设备通过HARQ NACK或波束失败恢复等度量信息，来监视终端设备侧的AI/ML性能，当监视到多个用户都出现相关度量的性能不好，且达到某一条件时，网络设备决定停止终端设备侧的AI/ML，并告知使用该AI/ML模型的所有终端设备停止使用该AI/ML模型。
在一些实施例中，网络设备将所述小区内性能低于阈值的AI/ML模型的标识信息通过系统信息在所述小区内广播，和/或，将所述小区内性能高于所述阈值的AI/ML模型的标识信息通过系统信息在所述小区内广播。
例如，网络设备将性能不好的AI/ML模型标识收入该网络设备所对应小区的AI/ML模型的排除名单，作为系统信息在小区广播。排除名单包括AI模型功能标识、对应的AI模型标识。此外，还可进一步包含对应某一AI功能的AI模型组标识、组内标识等。同样的，对于其他网络设备通过模型监测识别出的性能不佳，且判断为需要停用的终端设备侧AI/ML模型的模型标识，也可以如上收入到小区AI/ML模型的排除名单。
再例如，网络设备也可以积累终端设备使用AI/ML性能特别好的模型列表，并放入小区AI/ML模型的允许名单中，作为系统信息在小区广播。允许名单包括AI模型功能标识、对应的AI模型标识。此外，还可进一步包含对应某一AI功能的AI模 型组标识、组内标识等。
在一些实施例中，网络设备为一个或多个终端设备配置小区特定(cell-specific)的配置信息，使得所述一个或多个终端设备根据所述配置信息反馈用于监测AI/ML模型的信号或信息。
例如，网络设备采用cell-specific配置信息，为小区内的终端设备配置模型监测配置信息，用于让终端设备反馈用于某功能AI的性能反馈或监视信息。该配置可能包含测量配置和上报配置以及监测指标配置。
其中，监测指标配置可以包括监测的度量信息，例如RSRP、HARQ-NACK、某一事件的计数器、定时器、相应该指标的门限值信息。测量信息可以包括测量的RS类型，例如SSB，CSI-RS等、测量资源配置。上报配置包括上报方式和资源，例如周期上报(periodic report)、半持续上报(semi-persistent report)、非周期上报(aperiodic report)等。
终端设备收到模型监测配置信息后，可以依据配置进行上报。网络设备根据接收到的上报信息判断AI/ML模型的性能。通过多个终端设备上报的统计，并达到一定条件时，网络设备确定AI/ML模型性能不好，并停止该AI/ML模型。
在一些实施例中，所述网络设备将所述小区内性能低于所述阈值的AI/ML模型的标识信息发送给其他小区或者核心网设备，和/或，将所述小区内性能高于所述阈值的AI/ML模型的标识信息发送给其他小区或者核心网设备。
例如，AI/ML模型的排除名单和/或允许名单可以发送给相邻的网络设备，也可以发送给核心网设备。
再例如，在上述模型监控配置信息中，还可配置优秀模型的监测度量和门限值指标，以及相应的测量配置和上报配置。当收到多个终端设备对于优秀模型的上报统计后，网络设备可以将该模型标识推荐或分享到相邻小区或核心网。
以上各个实施例仅对本申请实施例进行了示例性说明，但本申请不限于此，还可以在以上各个实施例的基础上进行适当的变型。例如，可以单独使用上述各个实施例，也可以将以上各个实施例中的一种或多种结合起来。
由上述实施例可知，网络设备根据来自终端设备的信号或信息对所述网络设备和/或所述终端设备中的AI/ML模型进行监测或性能评估。由此，能够监视AI/ML模型的运行，保持AI/ML模型运行的一致性，提高模型运行的鲁棒性。
第二方面的实施例
本申请实施例提供一种AI监测方法，从终端设备侧进行说明，与第一方面的实施例相同的内容不再赘述。在第二方面的实施例中，由终端设备对AI/ML模型进行监测。
图8是本申请实施例的AI监测方法的一示意图，如图8所示，该方法包括：
801，终端设备接收网络设备发送的信号或信息；
802，所述终端设备根据所述信号或信息，对所述网络设备和/或所述终端设备中的AI/ML模型进行监测或性能评估。
值得注意的是，以上附图8仅对本申请实施例进行了示意性说明，但本申请不限于此。例如可以适当地调整各个操作之间的执行顺序，此外还可以增加其他的一些操作或者减少其中的某些操作。本领域的技术人员可以根据上述内容进行适当地变型，而不仅限于上述附图8的记载。
在一些实施例中，终端设备对所述网络设备中的AI/ML模型进行监测或性能评估，在所述AI/ML模型的性能满足预定条件的情况下，所述终端设备向所述网络设备发送停止所述AI/ML模型的请求信息。
图9是本申请实施例的终端设备监测网络设备的AI/ML模型的一示意图。如图9所示，网络设备使用AI/ML实现某项功能，终端设备可以通过接收信号或信息来判断网络设备侧AI/ML模型的性能。当性能达不到某一指标或条件时，终端设备判定该AI/ML模型的性能不好，终端设备可能不能直接指示网络设备停止其AI/ML模型，可以向网络设备发送相关AI/ML模型停止的请求，和/或，发送开启相关非AI处理的请求。
在一些实施例中，所述请求信息包括如下至少之一：所述AI/ML模型的停止请求，所述AI/ML模型或模型组的标识信息，所述AI/ML模型的对应的信号处理功能的标识、非AI处理的开启请求、非AI处理的标识信息。
在一些实施例中，终端设备接收网络设备发送的用于AI/ML监测的配置信息。
例如，网络设备为终端设备配置用于模型监测的测量配置、上报配置以及监测指标配置。其中，监测指标配置可以包括监测的度量信息，例如RSRP、HARQ-NACK、某一事件的计数器、定时器、相应该指标的门限值信息。测量信息可以包括测量的RS 类型，例如SSB、CSI-RS等、测量资源配置。上报配置包括上报方式和资源，例如周期上报(periodic report)、半持续上报(semi-persistent report)、非周期上报(aperiodic report)等。
在一些实施例中，所述终端设备对所述终端设备中的AI/ML模型进行监测或性能评估，在所述AI/ML模型的性能满足预定条件的情况下，所述终端设备向所述网络设备发送停止所述AI/ML模型的请求信息。
图10是本申请实施例的终端设备监测终端设备的AI/ML模型的一示意图。如图10所示，终端设备使用AI/ML实现某项功能，终端设备可以通过接收来自网络设备的信号或信息，来判断终端设备侧AI/ML模型的性能。当性能达不到某一指标或条件时，终端设备判定该AI/ML模型的性能不好，终端设备例如不能直接停止其AI/ML模型，可以向网络设备发送相关AI/ML模型停止的请求，和/或，发送开启相关非AI处理的请求。
在一些实施例中，所述终端设备接收所述网络设备发送的AI能力查询请求；以及向所述网络设备反馈AI/ML能力和/或AI监测能力。
例如，网络设备问询终端设备侧的AI/ML相关能力，包括以下一项或多项：AI功能支持查询、AI模型监测能力查询、AI对应处理功能查询，等等。终端设备可以根据查询请求相应地进行响应或汇报。
再例如，网络设备在终端设备具有AI能力且能够进行模型监测的情况下，为终端设备配置用于模型监测的测量配置、上报配置以及监测指标配置。其中，监测指标配置可以包括监测的度量信息，例如RSRP、HARQ-NACK、某一事件的计数器、定时器、相应该指标的门限值信息。测量信息可以包括测量的RS类型，例如SSB、CSI-RS等、测量资源配置。上报配置包括上报方式和资源，例如周期上报(periodic report)、半持续上报(semi-persistent report)、非周期上报(aperiodic report)等。
以上各个实施例仅对本申请实施例进行了示例性说明，但本申请不限于此，还可以在以上各个实施例的基础上进行适当的变型。例如，可以单独使用上述各个实施例，也可以将以上各个实施例中的一种或多种结合起来。
由上述实施例可知，终端设备根据来自网络设备的信号或信息对所述网络设备和/或所述终端设备中的AI/ML模型进行监测或性能评估。由此，能够监视AI/ML模型的运行，保持AI/ML模型运行的一致性，提高模型运行的鲁棒性。
第三方面的实施例
本申请实施例提供一种AI监测装置。该装置例如可以是网络设备，也可以是配置于网络设备的某个或某些部件或者组件，与第一方面的实施例相同的内容不再赘述。
图11是本申请实施例的AI监测装置的一示意图。如图11所示，AI监测装置1100包括：
接收单元1101，其接收终端设备发送的信号或信息；以及
处理单元1102，其根据所述信号或信息对所述网络设备和/或所述终端设备中的AI/ML模型进行监测或性能评估。
在一些实施例中，由网络设备对所述终端设备中的AI/ML模型进行监测或性能评估。如图11所示，该装置还包括：
发送单元1103，其在所述AI/ML模型的性能满足预定条件的情况下，向所述终端设备发送停止所述AI/ML模型的指示信息。
在一些实施例中，所述终端设备发送的信号或信息为与所述终端设备中的AI/ML模型相关的针对某一信号处理功能的信号或信息。
在一些实施例中，所述指示信息还包括所述AI/ML模型的标识信息，和/或，所述指示信息还指示所述终端设备切换到对应某一信号处理功能的非AI/ML处理；
所述AI/ML模型的标识信息包括如下至少之一：所述AI/ML模型对应的信号处理功能标识、所述AI/ML模型的标识、所述AI/ML模型的模型组标识、所述AI/ML模型的组内标识。
在一些实施例中，所述AI/ML模型运行在所述终端设备并被用于信道状态信息(CSI)估计或预测；所述网络设备对来自所述终端设备的混合自动重传请求(HARQ)反馈信息进行监测，并根据所述HARQ反馈信息对所述AI/ML模型进行性能评估。
在一些实施例中，所述网络设备配置和指示所述终端设备进行CSI估计和上报；其中所述终端设备使用所述AI/ML模型进行CSI估计或预测；接收所述终端设备上报的CSI；根据上报的所述CSI发送下行信息；以及接收所述终端设备发送的HARQ反馈信息；其中所述HARQ反馈信息由所述终端设备基于所述下行信息而生成。
在一些实施例中，所述网络设备接收所述终端设备发送的信道变化指示信息，所述信道变化指示信息包括所述网络设备和所述终端设备之间的信道在时域和/或频域 和/或空域上的变化信息；根据所述信道变化指示信息生成测量资源配置信息和/或上报资源配置信息；向所述终端设备发送所述测量资源配置信息和/或所述上报资源配置信息。
在一些实施例中，所述AI/ML模型运行在终端设备并被用于波束管理或波束预测；所述网络设备对来自所述终端设备的上行信号进行监测；并根据所述上行信号对所述AI/ML模型进行性能评估。
在一些实施例中，所述网络设备接收所述终端设备发送的波束重复发送数目或CSI-RS重复发送数目；根据所述波束重复发送数目或CSI-RS重复发送数目发送参考信号；其中所述终端设备使用所述AI/ML模型进行波束估计以获得波束估计信息；接收所述终端设备上报的所述波束估计信息；以及接收所述终端设备发送的上行信号；其中所述上行信号由所述终端设备基于所述波束估计信息而生成。
在一些实施例中，所述AI/ML模型运行在终端设备并被用于波束管理或波束预测；所述网络设备对来自所述终端设备的HARQ反馈信息进行监测；并根据所述HARQ反馈信息对所述AI/ML模型进行性能评估。
在一些实施例中，所述网络设备配置和指示所述终端设备进行波束预测；其中所述终端设备使用所述AI/ML模型进行波束预测；接收所述终端设备上报的波束预测信息；根据上报的所述波束预测信息发送下行信息；以及接收所述终端设备发送的HARQ反馈信息；其中所述HARQ反馈信息由所述终端设备基于所述下行信息而生成。
在一些实施例中，由网络设备对所述网络设备中的AI/ML模型进行监测或性能评估，在所述AI/ML模型的性能满足预定条件的情况下，所述网络设备停止所述AI/ML模型。
在一些实施例中，所述终端设备发送的信号或信息包括如下至少之一：探测参考信号(SRS)、参考信号接收功率(RSRP)、HARQ反馈信息、波束失败请求信息、波束失败恢复(BFR)信息。
在一些实施例中，由网络设备对小区内的AI/ML模型进行监测或性能评估，在所述AI/ML模型的性能满足预定条件的终端设备数目达到阈值的情况下，所述网络设备确定停止所述AI/ML模型。
在一些实施例中，所述网络设备将所述小区内性能低于阈值的AI/ML模型的标 识信息通过系统信息在所述小区内广播，和/或，将所述小区内性能高于所述阈值的AI/ML模型的标识信息通过系统信息在所述小区内广播。
在一些实施例中，所述网络设备为一个或多个终端设备配置小区特定(cell-specific)的配置信息，使得所述一个或多个终端设备根据所述配置信息反馈用于监测AI/ML模型的信号或信息。
在一些实施例中，所述网络设备将所述小区内性能低于所述阈值的AI/ML模型的标识信息发送给其他小区或者核心网设备，和/或，将所述小区内性能高于所述阈值的AI/ML模型的标识信息发送给其他小区或者核心网设备。
以上各个实施例仅对本申请实施例进行了示例性说明，但本申请不限于此，还可以在以上各个实施例的基础上进行适当的变型。例如，可以单独使用上述各个实施例，也可以将以上各个实施例中的一种或多种结合起来。
值得注意的是，以上仅对与本申请相关的各部件或模块进行了说明，但本申请不限于此。AI监测装置1100还可以包括其他部件或者模块，关于这些部件或者模块的具体内容，可以参考相关技术。
此外，为了简单起见，图11中仅示例性示出了各个部件或模块之间的连接关系或信号走向，但是本领域技术人员应该清楚的是，可以采用总线连接等各种相关技术。上述各个部件或模块可以通过例如处理器、存储器、发射机、接收机等硬件设施来实现；本申请实施并不对此进行限制。
由上述实施例可知，网络设备根据来自终端设备的信号或信息对所述网络设备和/或所述终端设备中的AI/ML模型进行监测或性能评估。由此，能够监视AI/ML模型的运行，保持AI/ML模型运行的一致性，提高模型运行的鲁棒性。
第四方面的实施例
本申请实施例提供一种AI监测装置。该装置例如可以是终端设备，也可以是配置于终端设备的某个或某些部件或者组件，与第一、二方面的实施例相同的内容不再赘述。
图12是本申请实施例的AI监测装置的一示意图。如图12所示，AI监测装置1200包括：
接收单元1201，其接收网络设备发送的信号或信息；以及
处理单元1202，其根据所述信号或信息对所述网络设备和/或终端设备中的AI/ML模型进行监测或性能评估。
在一些实施例中，所述终端设备对所述网络设备中的AI/ML模型进行监测或性能评估，在所述AI/ML模型的性能满足预定条件的情况下，所述终端设备向所述网络设备发送停止所述AI/ML模型的请求信息。
在一些实施例中，所述终端设备对所述终端设备中的AI/ML模型进行监测或性能评估，在所述AI/ML模型的性能满足预定条件的情况下，所述终端设备向所述网络设备发送停止所述AI/ML模型的请求信息。
以上各个实施例仅对本申请实施例进行了示例性说明，但本申请不限于此，还可以在以上各个实施例的基础上进行适当的变型。例如，可以单独使用上述各个实施例，也可以将以上各个实施例中的一种或多种结合起来。
值得注意的是，以上仅对与本申请相关的各部件或模块进行了说明，但本申请不限于此。AI监测装置1200还可以包括其他部件或者模块，关于这些部件或者模块的具体内容，可以参考相关技术。
此外，为了简单起见，图12中仅示例性示出了各个部件或模块之间的连接关系或信号走向，但是本领域技术人员应该清楚的是，可以采用总线连接等各种相关技术。上述各个部件或模块可以通过例如处理器、存储器、发射机、接收机等硬件设施来实现；本申请实施并不对此进行限制。
由上述实施例可知，终端设备根据来自网络设备的信号或信息对所述网络设备和/或所述终端设备中的AI/ML模型进行监测或性能评估。由此，能够监视AI/ML模型的运行，保持AI/ML模型运行的一致性，提高模型运行的鲁棒性。
第五方面的实施例
本申请实施例还提供一种通信系统，可以参考图1，与第一至四方面的实施例相同的内容不再赘述。
在一些实施例中，通信系统100至少可以包括：
终端设备102，其接收网络设备发送的信号或信息；以及根据所述信号或信息对所述网络设备和/或所述终端设备中的AI/ML模型进行监测或性能评估；和/或
网络设备101，其接收终端设备发送的信号或信息；以及根据所述信号或信息对 所述网络设备和/或所述终端设备中的AI/ML模型进行监测或性能评估。
本申请实施例还提供一种网络设备，例如可以是基站，但本申请不限于此，还可以是其他的网络设备。
图13是本申请实施例的网络设备的构成示意图。如图13所示，网络设备1300可以包括：处理器1310(例如中央处理器CPU)和存储器1320；存储器1320耦合到处理器1310。其中该存储器1320可存储各种数据；此外还存储信息处理的程序1330，并且在处理器1310的控制下执行该程序1330。
例如，处理器1310可以被配置为执行程序而实现如第一方面的实施例所述的AI监测方法。例如处理器1310可以被配置为进行如下的控制：接收终端设备发送的信号或信息；以及根据所述信号或信息对所述网络设备和/或所述终端设备中的AI/ML模型进行监测或性能评估。
此外，如图13所示，网络设备1300还可以包括：收发机1340和天线1350等；其中，上述部件的功能与现有技术类似，此处不再赘述。值得注意的是，网络设备1300也并不是必须要包括图13中所示的所有部件；此外，网络设备1300还可以包括图13中没有示出的部件，可以参考现有技术。
本申请实施例还提供一种终端设备，但本申请不限于此，还可以是其他的设备。
图14是本申请实施例的终端设备的示意图。如图14所示，该终端设备1400可以包括处理器1410和存储器1420；存储器1420存储有数据和程序，并耦合到处理器1410。值得注意的是，该图是示例性的；还可以使用其他类型的结构，来补充或代替该结构，以实现电信功能或其他功能。
例如，处理器1410可以被配置为执行程序而实现如第二方面的实施例所述的AI监测方法。例如处理器1410可以被配置为进行如下的控制：接收网络设备发送的信号或信息；以及根据所述信号或信息对所述网络设备和/或所述终端设备中的AI/ML模型进行监测或性能评估。
如图14所示，该终端设备1400还可以包括：通信模块1430、输入单元1440、显示器1450、电源1460。其中，上述部件的功能与现有技术类似，此处不再赘述。值得注意的是，终端设备1400也并不是必须要包括图14中所示的所有部件，上述部件并不是必需的；此外，终端设备1400还可以包括图14中没有示出的部件，可以参考现有技术。
本申请实施例还提供一种计算机程序，其中当在终端设备中执行所述程序时，所述程序使得所述终端设备执行第二方面的实施例所述的AI监测方法。
本申请实施例还提供一种存储有计算机程序的存储介质，其中所述计算机程序使得终端设备执行第二方面的实施例所述的AI监测方法。
本申请实施例还提供一种计算机程序，其中当在网络设备中执行所述程序时，所述程序使得所述网络设备执行第一方面的实施例所述的AI监测方法。
本申请实施例还提供一种存储有计算机程序的存储介质，其中所述计算机程序使得网络设备执行第一方面的实施例所述的AI监测方法。
本申请以上的装置和方法可以由硬件实现，也可以由硬件结合软件实现。本申请涉及这样的计算机可读程序，当该程序被逻辑部件所执行时，能够使该逻辑部件实现上文所述的装置或构成部件，或使该逻辑部件实现上文所述的各种方法或步骤。本申请还涉及用于存储以上程序的存储介质，如硬盘、磁盘、光盘、DVD、flash存储器等。
结合本申请实施例描述的方法/装置可直接体现为硬件、由处理器执行的软件模块或二者组合。例如，图中所示的功能框图中的一个或多个和/或功能框图的一个或多个组合，既可以对应于计算机程序流程的各个软件模块，亦可以对应于各个硬件模块。这些软件模块，可以分别对应于图中所示的各个步骤。这些硬件模块例如可利用现场可编程门阵列(FPGA)将这些软件模块固化而实现。
软件模块可以位于RAM存储器、闪存、ROM存储器、EPROM存储器、EEPROM存储器、寄存器、硬盘、移动磁盘、CD-ROM或者本领域已知的任何其它形式的存储介质。可以将一种存储介质耦接至处理器，从而使处理器能够从该存储介质读取信息，且可向该存储介质写入信息；或者该存储介质可以是处理器的组成部分。处理器和存储介质可以位于ASIC中。该软件模块可以存储在移动终端的存储器中，也可以存储在可插入移动终端的存储卡中。例如，若设备(如移动终端)采用的是较大容量的MEGA-SIM卡或者大容量的闪存装置，则该软件模块可存储在该MEGA-SIM卡或者大容量的闪存装置中。
针对附图中描述的功能方框中的一个或多个和/或功能方框的一个或多个组合，可以实现为用于执行本申请所描述功能的通用处理器、数字信号处理器(DSP)、专用集成电路(ASIC)、现场可编程门阵列(FPGA)或者其它可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件或者其任意适当组合。针对附图描述的功能方框中 的一个或多个和/或功能方框的一个或多个组合，还可以实现为计算设备的组合，例如，DSP和微处理器的组合、多个微处理器、与DSP通信结合的一个或多个微处理器或者任何其它这种配置。
以上结合具体的实施方式对本申请进行了描述，但本领域技术人员应该清楚，这些描述都是示例性的，并不是对本申请保护范围的限制。本领域技术人员可以根据本申请的精神和原理对本申请做出各种变型和修改，这些变型和修改也在本申请的范围内。
关于包括以上实施例的实施方式，还公开下述的附记：
1.一种AI监测方法，包括：
网络设备接收终端设备发送的信号或信息；以及
根据所述信号或信息对所述网络设备和/或所述终端设备中的AI/ML模型进行监测或性能评估。
2.根据附记1所述的方法，其中，所述网络设备对所述终端设备中的AI/ML模型进行监测或性能评估，所述方法还包括：
在所述AI/ML模型的性能满足预定条件的情况下，所述网络设备向所述终端设备发送停止所述AI/ML模型的指示信息。
3.根据附记2所述的方法，其中，所述终端设备发送的信号或信息为与所述终端设备中的AI/ML模型相关的针对某一信号处理功能的信号或信息。
4.根据附记2所述的方法，其中，所述指示信息还包括所述AI/ML模型的标识信息，和/或，所述指示信息还指示所述终端设备切换到对应某一信号处理功能的非AI/ML处理。
5.根据附记2所述的方法，其中，所述AI/ML模型的标识信息包括如下至少之一：所述AI/ML模型对应的信号处理功能标识、所述AI/ML模型的标识、所述AI/ML模型的模型组标识、所述AI/ML模型的组内标识。
6.根据附记2至5任一项所述的方法，其中，所述AI/ML模型运行在终端设备并被用于信道状态信息(CSI)估计或预测；
所述网络设备对来自所述终端设备的混合自动重传请求(HARQ)反馈信息进行监测，并根据所述HARQ反馈信息对所述AI/ML模型进行性能评估。
7.根据附记6所述的方法，其中，所述方法还包括：
所述网络设备配置和指示所述终端设备进行CSI估计和上报；其中所述终端设备使用所述AI/ML模型进行CSI估计或预测；
接收所述终端设备上报的CSI；
根据上报的所述CSI发送下行信息；以及
接收所述终端设备发送的HARQ反馈信息；其中所述HARQ反馈信息由所述终端设备基于所述下行信息而生成。
8.根据附记7所述的方法，其中，所述方法还包括：
所述网络设备接收所述终端设备发送的信道变化指示信息，所述信道变化指示信息包括所述网络设备和所述终端设备之间的信道在时域和/或频域和/或空域上的变化信息；
根据所述信道变化指示信息生成测量资源配置信息和/或上报资源配置信息；
向所述终端设备发送所述测量资源配置信息和/或所述上报资源配置信息。
9.根据附记2至5任一项所述的方法，其中，所述AI/ML模型运行在终端设备并被用于波束管理或波束预测；
所述网络设备对来自所述终端设备的上行信号进行监测；并根据所述上行信号对所述AI/ML模型进行性能评估。
10.根据附记9所述的方法，其中，所述方法还包括：
所述网络设备接收所述终端设备发送的波束重复发送数目或CSI-RS重复发送数目；
根据所述波束重复发送数目或CSI-RS重复发送数目发送参考信号；其中所述终端设备使用所述AI/ML模型进行波束估计以获得波束估计信息；
接收所述终端设备上报的所述波束估计信息；以及
接收所述终端设备发送的上行信号；其中所述上行信号由所述终端设备基于所述波束估计信息而生成。
11.根据附记10所述的方法，其中，所述网络设备将基于所述波束估计信息生成的上行信号的强度与其他方向的上行信号的强度进行比较，以对所述AI/ML模型进行性能评估。
12.根据附记2至5任一项所述的方法，其中，所述AI/ML模型运行在终端设备并被用于波束管理或波束预测；
所述网络设备对来自所述终端设备的HARQ反馈信息进行监测；并根据所述HARQ反馈信息对所述AI/ML模型进行性能评估。
13.根据附记12所述的方法，其中，所述方法还包括：
所述网络设备配置和指示所述终端设备进行波束预测；其中所述终端设备使用所述AI/ML模型进行波束预测；
接收所述终端设备上报的波束预测信息；
根据上报的所述波束预测信息发送下行信息；以及
接收所述终端设备发送的HARQ反馈信息；其中所述HARQ反馈信息由所述终端设备基于所述下行信息而生成。
14.根据附记13所述的方法，其中，所述方法还包括：
所述网络设备接收所述终端设备发送的信道变化指示信息，所述信道变化指示信息包括所述网络设备和所述终端设备之间的信道在时域和/或频域和/或空域上的变化信息；
根据所述信道变化指示信息生成波束预测配置；
向所述终端设备发送所述波束预测配置信息。
15.根据附记1所述的方法，其中，所述网络设备对所述网络设备中的AI/ML模型进行监测或性能评估，所述方法还包括：
在所述AI/ML模型的性能满足预定条件的情况下，所述网络设备停止所述AI/ML模型。
16.根据附记15所述的方法，其中，所述终端设备发送的信号或信息包括如下至少之一：探测参考信号(SRS)、参考信号接收功率(RSRP)、HARQ反馈信息、波束失败请求信息、波束失败恢复(BFR)信息。
17.根据附记1至16任一项所述的方法，其中，所述网络设备对小区内的AI/ML模型进行监测或性能评估，在所述AI/ML模型的性能满足预定条件的终端设备数目达到阈值的情况下，所述网络设备确定停止所述AI/ML模型。
18.根据附记17所述的方法，其中，所述方法还包括：
所述网络设备将所述小区内性能低于阈值的AI/ML模型的标识信息通过系统信息在所述小区内广播，和/或，将所述小区内性能高于所述阈值的AI/ML模型的标识信息通过系统信息在所述小区内广播。
19.根据附记17所述的方法，其中，所述方法还包括：
所述网络设备为一个或多个终端设备配置小区特定(cell-specific)的配置信息，使得所述一个或多个终端设备根据所述配置信息反馈用于监测AI/ML模型的信号或信息。
20.根据附记17所述的方法，其中，所述方法还包括：
所述网络设备将所述小区内性能低于所述阈值的AI/ML模型的标识信息发送给其他小区或者核心网设备，和/或，将所述小区内性能高于所述阈值的AI/ML模型的标识信息发送给其他小区或者核心网设备。
21.一种AI监测方法，包括：
终端设备接收网络设备发送的信号或信息；以及
根据所述信号或信息对所述网络设备和/或所述终端设备中的AI/ML模型进行监测或性能评估。
22.根据附记21所述的方法，其中，所述终端设备对所述网络设备中的AI/ML模型进行监测或性能评估，所述方法还包括：
在所述AI/ML模型的性能满足预定条件的情况下，所述终端设备向所述网络设备发送停止所述AI/ML模型的请求信息。
23.根据附记22所述的方法，其中，所述请求信息包括如下至少之一：所述AI/ML模型的停止请求，所述AI/ML模型或模型组的标识信息，所述AI/ML模型的对应的信号处理功能的标识、非AI处理的开启请求、非AI处理的标识信息。
24.根据附记22所述的方法，其中，所述方法还包括：
所述终端设备接收所述网络设备发送的用于AI/ML监测的配置信息。
25.根据附记21所述的方法，其中，所述终端设备对所述终端设备中的AI/ML模型进行监测或性能评估，所述方法还包括：
在所述AI/ML模型的性能满足预定条件的情况下，所述终端设备向所述网络设备发送停止所述AI/ML模型的请求信息。
26.根据附记21至25任一项所述的方法，其中，所述方法还包括：
所述终端设备接收所述网络设备发送的AI能力查询请求；以及
向所述网络设备反馈AI/ML能力和/或AI监测能力。
27.一种网络设备，包括存储器和处理器，所述存储器存储有计算机程序，所述 处理器被配置为执行所述计算机程序而实现如附记1至20任一项所述的AI监测方法。
28.一种终端设备，包括存储器和处理器，所述存储器存储有计算机程序，所述处理器被配置为执行所述计算机程序而实现如附记21至26任一项所述的AI监测方法。

Claims (20)

  1. 一种AI监测装置，包括：
    接收单元，其接收终端设备发送的信号或信息；以及
    处理单元，其根据所述信号或信息对网络设备和/或所述终端设备中的AI/ML模型进行监测或性能评估。
  2. 根据权利要求1所述的装置，其中，由网络设备对所述终端设备中的AI/ML模型进行监测或性能评估，所述装置还包括：
    发送单元，其在所述AI/ML模型的性能满足预定条件的情况下，向所述终端设备发送停止所述AI/ML模型的指示信息。
  3. 根据权利要求2所述的装置，其中，所述终端设备发送的信号或信息为与所述终端设备中的AI/ML模型相关的针对某一信号处理功能的信号或信息。
  4. 根据权利要求2所述的装置，其中，所述指示信息还包括所述AI/ML模型的标识信息，和/或，所述指示信息还指示所述终端设备切换到对应某一信号处理功能的非AI/ML处理；
    所述AI/ML模型的标识信息包括如下至少之一：所述AI/ML模型对应的信号处理功能标识、所述AI/ML模型的标识、所述AI/ML模型的模型组标识、所述AI/ML模型的组内标识。
  5. 根据权利要求2所述的装置，其中，所述AI/ML模型运行在所述终端设备并被用于信道状态信息估计或预测；所述网络设备对来自所述终端设备的混合自动重传请求反馈信息进行监测，并根据所述混合自动重传请求反馈信息对所述AI/ML模型进行性能评估。
  6. 根据权利要求5所述的装置，其中，所述网络设备配置和指示所述终端设备进行信道状态信息估计和上报；其中所述终端设备使用所述AI/ML模型进行信道状态信息估计或预测；接收所述终端设备上报的信道状态信息；根据上报的所述信道状态信息发送下行信息；以及接收所述终端设备发送的混合自动重传请求反馈信息；其中所述混合自动重传请求反馈信息由所述终端设备基于所述下行信息而生成。
  7. 根据权利要求6所述的装置，其中，所述网络设备接收所述终端设备发送的信道变化指示信息，所述信道变化指示信息包括所述网络设备和所述终端设备之间的 信道在时域和/或频域和/或空域上的变化信息；根据所述信道变化指示信息生成测量资源配置信息和/或上报资源配置信息；向所述终端设备发送所述测量资源配置信息和/或所述上报资源配置信息。
  8. 根据权利要求2所述的装置，其中，所述AI/ML模型运行在终端设备并被用于波束管理或波束预测；所述网络设备对来自所述终端设备的上行信号进行监测；并根据所述上行信号对所述AI/ML模型进行性能评估。
  9. 根据权利要求8所述的装置，其中，所述网络设备接收所述终端设备发送的波束重复发送数目或信道状态信息参考信号重复发送数目；根据所述波束重复发送数目或信道状态信息参考信号重复发送数目发送参考信号；其中所述终端设备使用所述AI/ML模型进行波束估计以获得波束估计信息；接收所述终端设备上报的所述波束估计信息；以及接收所述终端设备发送的上行信号；其中所述上行信号由所述终端设备基于所述波束估计信息而生成。
  10. 根据权利要求2所述的装置，其中，所述AI/ML模型运行在终端设备并被用于波束管理或波束预测；所述网络设备对来自所述终端设备的混合自动重传请求反馈信息进行监测；并根据所述混合自动重传请求反馈信息对所述AI/ML模型进行性能评估。
  11. 根据权利要求10所述的装置，其中，所述网络设备配置和指示所述终端设备进行波束预测；其中所述终端设备使用所述AI/ML模型进行波束预测；接收所述终端设备上报的波束预测信息；根据上报的所述波束预测信息发送下行信息；以及接收所述终端设备发送的混合自动重传请求反馈信息；其中所述混合自动重传请求反馈信息由所述终端设备基于所述下行信息而生成。
  12. 根据权利要求1所述的装置，其中，由网络设备对所述网络设备中的AI/ML模型进行监测或性能评估，在所述AI/ML模型的性能满足预定条件的情况下，所述网络设备停止所述AI/ML模型。
  13. 根据权利要求12所述的装置，其中，所述终端设备发送的信号或信息包括如下至少之一：探测参考信号、参考信号接收功率、混合自动重传请求反馈信息、波束失败请求信息、波束失败恢复信息。
  14. 根据权利要求1所述的装置，其中，由网络设备对小区内的AI/ML模型进行监测或性能评估，在所述AI/ML模型的性能满足预定条件的终端设备数目达到阈 值的情况下，所述网络设备确定停止所述AI/ML模型。
  15. 根据权利要求14所述的装置，其中，所述网络设备将所述小区内性能低于阈值的AI/ML模型的标识信息通过系统信息在所述小区内广播，和/或，将所述小区内性能高于所述阈值的AI/ML模型的标识信息通过系统信息在所述小区内广播。
  16. 根据权利要求15所述的装置，其中，所述网络设备为一个或多个终端设备配置小区特定的配置信息，使得所述一个或多个终端设备根据所述配置信息反馈用于监测AI/ML模型的信号或信息。
  17. 根据权利要求15所述的装置，其中，所述网络设备将所述小区内性能低于所述阈值的AI/ML模型的标识信息发送给其他小区或者核心网设备，和/或，将所述小区内性能高于所述阈值的AI/ML模型的标识信息发送给其他小区或者核心网设备。
  18. 一种AI监测装置，包括：
    接收单元，其接收网络设备发送的信号或信息；以及
    处理单元，其根据所述信号或信息对所述网络设备和/或终端设备中的AI/ML模型进行监测或性能评估。
  19. 根据权利要求18所述的装置，其中，所述终端设备对所述网络设备中的AI/ML模型进行监测或性能评估，在所述AI/ML模型的性能满足预定条件的情况下，所述终端设备向所述网络设备发送停止所述AI/ML模型的请求信息；
    和/或，所述终端设备对所述终端设备中的AI/ML模型进行监测或性能评估，在所述AI/ML模型的性能满足预定条件的情况下，所述终端设备向所述网络设备发送停止所述AI/ML模型的请求信息。
  20. 一种通信系统，包括：
    终端设备，其接收网络设备发送的信号或信息；以及根据所述信号或信息对所述网络设备和/或所述终端设备中的AI/ML模型进行监测或性能评估；和/或
    网络设备，其接收终端设备发送的信号或信息；以及根据所述信号或信息对所述网络设备和/或所述终端设备中的AI/ML模型进行监测或性能评估。
PCT/CN2022/090505 2022-04-29 2022-04-29 Ai监测装置以及方法 WO2023206445A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2022/090505 WO2023206445A1 (zh) 2022-04-29 2022-04-29 Ai监测装置以及方法

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2022/090505 WO2023206445A1 (zh) 2022-04-29 2022-04-29 Ai监测装置以及方法

Publications (1)

Publication Number Publication Date
WO2023206445A1 true WO2023206445A1 (zh) 2023-11-02

Family

ID=88516752

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/090505 WO2023206445A1 (zh) 2022-04-29 2022-04-29 Ai监测装置以及方法

Country Status (1)

Country Link
WO (1) WO2023206445A1 (zh)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170347279A1 (en) * 2016-05-27 2017-11-30 Alcatel-Lucent Usa Inc. MONITORING AND MANAGEMENT OF eMBMS SYSTEMS
CN110661556A (zh) * 2018-06-29 2020-01-07 华为技术有限公司 发送和接收信道状态信息的方法和通信装置
CN113938232A (zh) * 2020-07-13 2022-01-14 华为技术有限公司 通信的方法及通信装置
CN114363921A (zh) * 2020-10-13 2022-04-15 维沃移动通信有限公司 Ai网络参数的配置方法和设备

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170347279A1 (en) * 2016-05-27 2017-11-30 Alcatel-Lucent Usa Inc. MONITORING AND MANAGEMENT OF eMBMS SYSTEMS
CN110661556A (zh) * 2018-06-29 2020-01-07 华为技术有限公司 发送和接收信道状态信息的方法和通信装置
CN113938232A (zh) * 2020-07-13 2022-01-14 华为技术有限公司 通信的方法及通信装置
CN114363921A (zh) * 2020-10-13 2022-04-15 维沃移动通信有限公司 Ai网络参数的配置方法和设备

Similar Documents

Publication Publication Date Title
RU2711033C1 (ru) Способы и устройства обеспечения покрытия лучом устройства связи, работающего в сети беспроводной связи
US8725153B2 (en) Methods and devices for adjusting resource management procedures in heterogeneous communication networks
CN113396615B (zh) 用于ue省电的方法
CN109804664B (zh) 用于无线网络中的测量和测量报告的方法和装置
US20220006539A1 (en) Signal transmission method, apparatus, electronic device and computer readable storage medium
WO2020221342A1 (en) Electronic device and method for beam management with power saving
JP2020503803A (ja) 通信方法、ネットワークデバイス、および端末デバイス
US20210227542A1 (en) Antenna configuration in a communication network
WO2021147641A1 (zh) 一种调整波束的方法及装置
US20230180128A1 (en) Valuation for ue assistance information
CN112567839A (zh) 评估无线链路质量的方法、参数配置方法、装置和系统
CN114982274A (zh) 早期测量报告下的ue节能机制
CN111434071B (zh) 信号接收装置及方法、通信系统
EP4014457B1 (en) Method and apparatus for transmission configuration
WO2020227913A1 (en) Method and apparatus for beam management
WO2023206445A1 (zh) Ai监测装置以及方法
EP4307757A1 (en) Communication method and communication apparatus
EP4236427A1 (en) Array selection method, terminal, network device, and storage medium
WO2023207026A1 (zh) 信息指示和处理方法以及装置
WO2023206437A1 (zh) 信息传输方法与装置
WO2023206272A1 (zh) 波束信息的发送和接收方法、装置和通信系统
WO2024031290A1 (zh) 信息收发方法与装置
WO2023236143A1 (zh) 信息收发方法与装置
WO2024016230A1 (zh) 信息收发方法与装置
WO2024031692A1 (zh) Ai/ml模型的监测方法和装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22939293

Country of ref document: EP

Kind code of ref document: A1