WO2022016406A1 - 信息传输方法、装置及通信设备 - Google Patents

信息传输方法、装置及通信设备 Download PDF

Info

Publication number
WO2022016406A1
WO2022016406A1 PCT/CN2020/103429 CN2020103429W WO2022016406A1 WO 2022016406 A1 WO2022016406 A1 WO 2022016406A1 CN 2020103429 W CN2020103429 W CN 2020103429W WO 2022016406 A1 WO2022016406 A1 WO 2022016406A1
Authority
WO
WIPO (PCT)
Prior art keywords
input
modalities
input data
modality
weight
Prior art date
Application number
PCT/CN2020/103429
Other languages
English (en)
French (fr)
Inventor
洪伟
张明
Original Assignee
北京小米移动软件有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京小米移动软件有限公司 filed Critical 北京小米移动软件有限公司
Priority to US18/010,791 priority Critical patent/US20230239726A1/en
Priority to CN202080001633.8A priority patent/CN114342335B/zh
Priority to PCT/CN2020/103429 priority patent/WO2022016406A1/zh
Priority to EP20945998.1A priority patent/EP4187930A4/en
Publication of WO2022016406A1 publication Critical patent/WO2022016406A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W28/00—Network traffic management; Network resource management
    • H04W28/02—Traffic management, e.g. flow control or congestion control
    • H04W28/0268—Traffic management, e.g. flow control or congestion control using specific QoS parameters for wireless networks, e.g. QoS class identifier [QCI] or guaranteed bit rate [GBR]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00—Traffic control in data switching networks
    • H04L47/10—Flow control; Congestion control
    • H04L47/24—Traffic characterised by specific attributes, e.g. priority or QoS

Definitions

  • the present application relates to the field of wireless communication technologies, but is not limited to the field of wireless communication technologies, and in particular, relates to information transmission methods, apparatuses, and communication equipment.
  • Multi-modal interaction can be to send input from multiple devices or multiple inputs from one device to a centralized processing device or function, and comprehensively process these multi-modal inputs. , and finally obtain one or more outputs that meet the user's needs. Among them, multiple outputs can also be performed by multiple devices or by one device.
  • embodiments of the present disclosure provide an information transmission method, apparatus, and communication device.
  • an information transmission method includes:
  • the processing result of the input data is determined based on the input data of at least one of the input modalities and the weight of the input modalities corresponding to each of the input data.
  • an information transmission apparatus wherein the apparatus includes: a first acquisition module and a determination module, wherein,
  • the first obtaining module is configured to obtain weights corresponding to different input modalities
  • the determining module is configured to determine a processing result of the input data based on the input data of at least one input modality and the weight of the input modality corresponding to each of the input data respectively.
  • a communication device including a processor, a transceiver, a memory, and an executable program stored on the memory and capable of being executed by the processor, the processor executing the executable program When the program is executed, the steps of the information transmission method described in the first aspect are executed.
  • a storage medium on which an executable program is stored, and when the executable program is executed by a processor, the steps of the information transmission method according to the first aspect are implemented.
  • the multimodal processing device obtains the weights corresponding to different input modalities; based on the input data of at least one of the input modalities, corresponding to each of the input data respectively
  • the weight of the input modality determines the processing result of the input data.
  • FIG. 1 is a schematic structural diagram of a wireless communication system according to an exemplary embodiment
  • FIG. 2 is a schematic flowchart of an information transmission method according to an exemplary embodiment
  • FIG. 3 is a multimodal processing architecture diagram according to an exemplary embodiment
  • FIG. 4 is a block diagram of an information transmission apparatus according to an exemplary embodiment
  • Fig. 5 is a block diagram of an apparatus for information transmission according to an exemplary embodiment.
  • first, second, third, etc. may be used in embodiments of the present disclosure to describe various pieces of information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other.
  • the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information.
  • the word "if” as used herein can be interpreted as "at the time of” or "when” or "in response to determining.”
  • FIG. 1 shows a schematic structural diagram of a wireless communication system provided by an embodiment of the present disclosure.
  • the wireless communication system is a communication system based on cellular mobile communication technology, and the wireless communication system may include: several terminals 11 and several base stations 12 .
  • the terminal 11 may be a device that provides voice and/or data connectivity to the user.
  • the terminal 11 may communicate with one or more core networks via a radio access network (RAN), and the terminal 11 may be an IoT terminal such as a sensor device, a mobile phone (or "cellular" phone) and a
  • RAN radio access network
  • the computer of the IoT terminal for example, may be a fixed, portable, pocket, hand-held, built-in computer or a vehicle-mounted device.
  • a station For example, a station (Station, STA), a subscriber unit (subscriber unit), a subscriber station (subscriber station), a mobile station (mobile station), a mobile station (mobile), a remote station (remote station), an access point, a remote terminal ( remote terminal), access terminal (access terminal), user device (user terminal), user agent (user agent), user equipment (user device), or user equipment (user equipment, UE).
  • the terminal 11 may also be a device of an unmanned aerial vehicle.
  • the terminal 11 may also be a vehicle-mounted device, for example, a trip computer with a wireless communication function, or a wireless communication device externally connected to the trip computer.
  • the terminal 11 may also be a roadside device, for example, a street light, a signal light, or other roadside devices with a wireless communication function.
  • the base station 12 may be a network-side device in a wireless communication system.
  • the wireless communication system may be a fourth generation mobile communication (the 4th generation mobile communication, 4G) system, also known as a long term evolution (Long Term Evolution, LTE) system; or, the wireless communication system may also be a 5G system, Also known as new radio (NR) system or 5G NR system.
  • the wireless communication system may also be a next-generation system of the 5G system.
  • the access network in the 5G system can be called NG-RAN (New Generation-Radio Access Network, a new generation of radio access network).
  • the MTC system may be a network-side device in a wireless communication system.
  • the base station 12 may be an evolved base station (eNB) used in the 4G system.
  • the base station 12 may also be a base station (gNB) that adopts a centralized distributed architecture in a 5G system.
  • eNB evolved base station
  • gNB base station
  • the base station 12 adopts a centralized distributed architecture it usually includes a centralized unit (central unit, CU) and at least two distributed units (distributed unit, DU).
  • the centralized unit is provided with a protocol stack of a Packet Data Convergence Protocol (PDCP) layer, a Radio Link Control Protocol (Radio Link Control, RLC) layer, and a Media Access Control (Media Access Control, MAC) layer; distribution A physical (Physical, PHY) layer protocol stack is set in the unit, and the specific implementation manner of the base station 12 is not limited in this embodiment of the present disclosure.
  • PDCP Packet Data Convergence Protocol
  • RLC Radio Link Control Protocol
  • MAC Media Access Control
  • distribution A physical (Physical, PHY) layer protocol stack is set in the unit, and the specific implementation manner of the base station 12 is not limited in this embodiment of the present disclosure.
  • a wireless connection can be established between the base station 12 and the terminal 11 through a wireless air interface.
  • the wireless air interface is a wireless air interface based on the fourth generation mobile communication network technology (4G) standard; or, the wireless air interface is a wireless air interface based on the fifth generation mobile communication network technology (5G) standard, such as
  • the wireless air interface is a new air interface; alternatively, the wireless air interface may also be a wireless air interface based on a 5G next-generation mobile communication network technology standard.
  • an E2E (End to End, end-to-end) connection may also be established between the terminals 11 .
  • V2V vehicle to vehicle, vehicle-to-vehicle
  • V2I vehicle to Infrastructure, vehicle-to-roadside equipment
  • V2P vehicle to pedestrian, vehicle-to-person communication in vehicle-to-everything (V2X) communication etc. scene.
  • the above wireless communication system may further include a network management device 13 .
  • the network management device 13 may be a core network device in a wireless communication system, for example, the network management device 13 may be a mobility management entity (Mobility Management Entity) in an evolved packet core network (Evolved Packet Core, EPC). MME).
  • the network management device may also be other core network devices, such as a serving gateway (Serving GateWay, SGW), a public data network gateway (Public Data Network GateWay, PGW), a policy and charging rule functional unit (Policy and Charging Rules) Function, PCRF) or home subscriber server (Home Subscriber Server, HSS), etc.
  • the implementation form of the network management device 13 is not limited in this embodiment of the present disclosure.
  • the executive bodies involved in the embodiments of the present disclosure include, but are not limited to, access network devices that support cellular mobile communications, such as base stations, etc., and core network devices.
  • the application scenario of the embodiment of the present disclosure is that, since the centralized processing device or function needs to be comprehensively processed based on the input of multiple modalities at the same time to obtain the multi-modal output, the reduction of the quality of one modal input will affect the quality of the entire multi-modal output. .
  • This embodiment provides an information transmission method that can be applied to a base station and/or a core in a mobile communication network. As shown in FIG. 2 , the information transmission method may include:
  • Step 201 Obtain weights corresponding to different input modalities
  • Step 202 Determine a processing result of the input data based on the input data of at least one input modality and the weight of the input modality corresponding to each input data.
  • each source or form of input data can be referred to as an input modality.
  • Input modalities can be distinguished based on the medium of the input data, for example, voice, video, text, etc. can be called an input modality; modalities can also be distinguished based on different acquisition methods of input data, for example: through sensors, radar, infrared
  • the input data obtained from the accelerometer and the like can be respectively referred to as input data of an input mode.
  • Each of the above can be called a modality.
  • Multi-modal data processing equipment such as access network equipment and/or core network equipment in the cellular mobile communication network can process the input data of at least one input modality to obtain a multi-modal processing result, and perform a multi-modal processing result on the processing result. output.
  • the access network device may include a base station, etc., and the input data may be processed by a processing device with processing capability in the base station.
  • Input data acquired through image sensors, audio sensors, etc. can be sent by user equipment (UE, User Equipment) such as mobile phone terminals and IoT terminals in the cellular mobile communication network.
  • UE User Equipment
  • the input data of the at least one input modality may be input data of at least one input modality obtained respectively by multiple UEs, or may be input data of at least one input modality obtained by one UE.
  • audio input data and video input data may be acquired by one mobile phone terminal; audio input data may also be acquired by one IoT terminal, and video input data may be acquired by another IoT terminal.
  • the multi-modal data processing device can determine the multi-modal processing result by means of big data analysis or AI processing on the input data of multiple input modalities. For example, in real-time speech recognition or translation, the multimodal data processing device can perform speech recognition and image recognition corresponding to the audio input data and video input data sent by the UE, and perform big data analysis or AI processing on the recognition results, etc. And then determine the semantic or translation result.
  • input data for different input modalities have different degrees of correlation with the processing results.
  • the audio input data has a high degree of correlation
  • the video input data can be used as an auxiliary recognition means.
  • different weights can be set for different input modalities. Input modalities with a high degree of correlation with the processing results are weighted higher, and input modalities with a low degree of correlation with the processing results are weighted lower.
  • the multimodal data processing device can pre-store the weights of different input modalities. Pre-stored weights of different input modalities are obtained during multimodal data processing.
  • the multimodal data processing device may determine output results based on weights of input data of different input modalities. For example, in real-time translation, the weight of audio input data can be set to 90%, and the weight of video input data can be set to 10%; comprehensively analyze the audio input data and video input data to determine the translation result.
  • the weights corresponding to different input modalities are determined based on the quality of service QOS of different input modalities.
  • QoS may include transmission delay, transmission rate, and the like.
  • the QoS of different input modalities may be different, that is, the transmission delay and/or transmission rate of input data of different input modalities are different. Therefore, the information content of input data of different input modalities received at the same time is unequal.
  • the weight of the input modalities may be determined based on the quality of service QoS of different input modalities, so as to balance the unequal situation of the information amount of different input data due to QoS.
  • the QoS of audio input data, video input data and haptic input data may be different, the QoS of audio input data and haptic input data is higher, and the QoS of video input data is lower.
  • the video input data may have less information than the voice data and tactile data.
  • the weight of the video input data can be increased. For example, audio input data may be weighted 30%, video input data may be weighted 40%, and haptic input data may be weighted 30%.
  • the weights include:
  • Different weight values are used to indicate absolute weights of the importance of different input modalities.
  • the relative weight can be a proportion weight, that is, the role of one input modality in the evaluation of all input modality can be reflected in a proportionate manner. For example: there are three modalities of audio input, video input and haptic input, the weight of audio input is 30%, the weight of video input is 40% and the weight of haptic input is 30%. Using relative weights can reflect the role of one input modality in the evaluation of all input modalities.
  • the absolute weight can use different weight values to indicate the importance of different input modalities. For example, there are three modalities of audio input, video input and tactile input. The weight of audio input is 3, the weight of video input is 4, and the weight of tactile input is 3.
  • the absolute weight of a single input mode can not be affected by other modes. When the existing input mode changes, such as adding a new input mode or removing an input mode, the absolute weight of the input mode does not change. In this way, the adaptability of the weights can be improved.
  • determining the processing result of the input data based on the input data of at least one input modality and the weight of the input modality corresponding to each input data including:
  • the processing result of the input data is determined based on the input data of at least one input modality received through the mobile communication network, and the weight of the input modality corresponding to each input data.
  • User equipment such as a mobile phone terminal and an IoT terminal in a cellular mobile communication network can send input data of multiple input modalities to an access network device or a core network device through the mobile communication network.
  • the input data of the multiple input modalities may be input data of each input modalities acquired by multiple UEs respectively, or may be input data of each input modalities acquired by one UE.
  • obtaining weights corresponding to different input modalities includes:
  • the centralized processing device or the centralized processing function may determine the weight corresponding to each input mode based on the multimodal processing requirement, and send the weight to the access network device or the core network device.
  • the centralized processing device may be a cloud processing device or the like.
  • the centralized processing function can be a function module used in the cloud for multi-modal processing, etc.
  • the weights corresponding to each input mode can also be determined by multi-modal input devices such as mobile phone terminals, Internet of Things terminals and other user equipment based on multi-modal processing requirements and sent to access network devices or core network devices.
  • multi-modal input devices such as mobile phone terminals, Internet of Things terminals and other user equipment based on multi-modal processing requirements and sent to access network devices or core network devices.
  • the weight corresponding to each input mode may also be pre-agreed and stored in the access network device or the core network device.
  • the information transmission method further includes:
  • the input data of at least one first input modality in the at least one input modality is used to compensate the input data of at least one second input modality in the at least one input modality.
  • multi-modal data processing equipment needs to perform comprehensive processing based on the input processing of multiple input modalities at the same time to obtain multi-modal processing results.
  • the reduction in the amount of data will affect the quality of the processing results of the entire multimodal output.
  • the input quality may include transmission delay and/or transmission rate, etc.
  • using the input data of the first input modality to compensate the input data of the second input modality may include: by improving the input quality of the input data of the first input modality, compensating for the input of the input data of the second input modality quality etc.
  • Compensation rules can be set based on the correspondence between different input modalities. For example, the input data of a plurality of first input modalities can be used to compensate the input data of a second input modality, and the input data of a plurality of first input modalities can also be used to compensate the input data of a plurality of second input modalities.
  • the compensation rule can also specify the corresponding relationship of compensation. For example, after the input mode A decreases, it can be compensated by increasing the input mode B, or it can be compensated by increasing the input mode C.
  • the compensation rules may specify the amount of data to be compensated and/or the quality of input, etc.
  • the compensation rule may specify the compensation amount of the input data of the first input modality for compensating the input data of the second input modality, and the like.
  • the compensation amount may be the data amount or the input quality or the like.
  • the data amount or transmission quality of the audio input modality decreases during the transmission process, and the base station or the core network can improve the data amount or the transmission quality of the video input modality.
  • using input data of at least one first input modality in at least one input modality to compensate for input data of at least one second input modality in at least one input modality including:
  • the QoS of the input data of the at least one second input modality is compensated using the QoS of the input data of the at least one first input modality.
  • the base station or the core network can improve the transmission quality of the first modality input, for example, improve the QoS of other modalities input. In this way, the influence on the multi-modal processing result due to the change of the QoS of the input modality can be reduced, thereby improving the accuracy of the multi-modal processing result.
  • the compensation rule is determined based on the weight of the first input modality and the weight of the second input modality.
  • the compensation rules may specify the amount of data to be compensated and/or the quality of input, etc.
  • the compensation rule may specify the compensation amount of the input data of the first input modality for compensating the input data of the second input modality, and the like.
  • the compensation rule can determine the compensation amount based on the weight. For example, for the second input modality with a relatively large relative weight, more compensation amounts of the first input modality with a relatively small relative weight can be used for compensation. For the second input modality with a relatively small relative weight, the compensation amount of the first input modality with a relatively large relative weight can be used for compensation.
  • the video input modality is weighted 40%. If the QoS of the audio input modality is reduced by 10%, since the weight of the video input modality is larger, only a smaller amount of compensation for the video input modality is required to compensate, for example, the QoS of the video input modality can be increased by 5% to compensate.
  • the compensation amount between different input modalities is determined based on the weight, so that the input data between different modalities in multi-modal data processing can meet the needs of multi-modal processing, and the input quality or data due to input modalities is reduced.
  • the influence of quantity change on the multimodal processing results is improved, and the quality of the multimodal processing results is improved.
  • the information transmission method further includes at least one of the following:
  • the centralized processing device or the centralized processing function may determine the compensation rule based on the weight of each input mode, etc., and send it to the access network device or the core network device.
  • the compensation rules can also be determined by multi-modal input devices such as mobile phone terminals and IoT end-user devices based on the weights of each input modality and the like, and sent to the access network device or the core network device.
  • the compensation rules can also be pre-agreed and stored in the access network device or the core network device.
  • the present invention proposes a method for ensuring output quality.
  • the base station and/or the core network obtain the weights of multiple input modalities.
  • the weight can be sent to the base station or core network by the centralized processing device or function.
  • the weight can be sent to the base station or the core network by the multimodal input device.
  • the weight can be a relative value or an absolute value.
  • the weight can be measured based on QoS, such as delay, rate, etc.
  • audio input accounts for 30% For example, audio input accounts for 30%, video input accounts for 40%, and tactile input accounts for 30%.
  • the base station can also obtain a compensation relationship between multiple input modalities, that is, compensation rules.
  • the compensation relationship can be sent to the base station or the core network by the centralized processing device or function.
  • the base station or the core network can improve the transmission quality of other modality inputs, such as improving the QoS of other modality inputs.
  • the QoS of audio input is reduced by 10%, and the QoS of video input is increased by 5%, the overall output quality or accuracy can be guaranteed.
  • Impairments caused by reduced input quality of one or more modalities can be compensated for by improving the transmission quality of the input from one or more modalities.
  • the specific compensation algorithm may be based on implementation, or may be based on the compensation relationship between multiple input modalities.
  • FIG. 4 is a schematic structural diagram of the composition of the information transmission apparatus 100 provided by the embodiment of the present invention; as shown in FIG. 4 , the apparatus 100 includes: a first acquisition module 110 and determining module 120, wherein,
  • the first obtaining module 110 is configured to obtain weights corresponding to different input modalities
  • the determination module 120 is configured to determine the processing result of the input data based on the input data of at least one input modality and the weight of the input modality corresponding to each input data.
  • the weights corresponding to different input modalities are determined based on the quality of service (QoS) of the different input modalities.
  • QoS quality of service
  • the weights include:
  • the determining module 120 includes:
  • the determination sub-module 121 is configured to determine the processing result of the input data based on the input data of at least one input modality received through the mobile communication network and the weight of the input modality corresponding to each input data.
  • the first obtaining module 110 includes:
  • the second obtaining sub-module 112 is configured to obtain pre-agreed weights corresponding to different input modalities.
  • the apparatus 100 further includes:
  • the compensation module 130 includes:
  • the compensation sub-module 131 is configured to use the QoS of the input data of the at least one first input modality to compensate the QoS of the input data of the at least one second input modality.
  • the apparatus 100 further includes at least one of the following:
  • a receiving module 140 configured to receive indication information indicating a compensation rule
  • the second obtaining module 150 is configured to obtain pre-agreed compensation rules.
  • the first obtaining module 110, the determining module 120, the compensation module 130, the receiving module 140, the second obtaining module 150, etc. may be processed by one or more central processing units (CPU, Central Processing Unit), graphics processor (GPU, Graphics Processing Unit), baseband processor (BP, baseband processor), application specific integrated circuit (ASIC, Application Specific Integrated Circuit), DSP, programmable logic device (PLD, Programmable Logic Device), complex programmable logic Device (CPLD, Complex Programmable Logic Device), Field Programmable Gate Array (FPGA, Field-Programmable Gate Array), General Purpose Processor, Controller, Micro Controller (MCU, Micro Controller Unit), Microprocessor (Microprocessor), or other electronic components, and can also be implemented in combination with one or more radio frequency (RF, radio frequency) antennas, for performing the aforementioned method.
  • CPU Central Processing Unit
  • GPU Graphics Processing Unit
  • BP baseband processor
  • ASIC Application Specific Integrated Circuit
  • DSP programmable logic device
  • PLD Programmable Logic Device
  • FIG. 5 is a block diagram of an apparatus 3000 for information transmission according to an exemplary embodiment.
  • apparatus 3000 may be a mobile phone, computer, digital broadcast terminal, messaging device, game console, tablet device, medical device, fitness device, personal digital assistant, and the like.
  • the apparatus 3000 may include one or more of the following components: a processing component 3002, a memory 3004, a power supply component 3006, a multimedia component 3008, an audio component 3010, an input/output (I/O) interface 3012, a sensor component 3014, And the communication component 3016.
  • a processing component 3002 a memory 3004, a power supply component 3006, a multimedia component 3008, an audio component 3010, an input/output (I/O) interface 3012, a sensor component 3014, And the communication component 3016.
  • the processing component 3002 generally controls the overall operation of the apparatus 3000, such as operations associated with display, phone calls, data communications, camera operations, and recording operations.
  • the processing component 3002 can include one or more processors 3020 to execute instructions to perform all or some of the steps of the methods described above.
  • processing component 3002 may include one or more modules that facilitate interaction between processing component 3002 and other components.
  • processing component 3002 may include a multimedia module to facilitate interaction between multimedia component 3008 and processing component 3002.
  • Power supply assembly 3006 provides power to various components of device 3000.
  • Power supply components 3006 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to device 3000.
  • Multimedia component 3008 includes a screen that provides an output interface between device 3000 and the user.
  • the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user.
  • the touch panel includes one or more touch sensors to sense touch, swipe, and gestures on the touch panel. A touch sensor can sense not only the boundaries of a touch or swipe action, but also the duration and pressure associated with the touch or swipe action.
  • the multimedia component 3008 includes a front-facing camera and/or a rear-facing camera. When the device 3000 is in an operation mode, such as a shooting mode or a video mode, the front camera and/or the rear camera may receive external multimedia data. Each of the front and rear cameras can be a fixed optical lens system or have focal length and optical zoom capability.
  • Audio component 3010 is configured to output and/or input audio signals.
  • audio component 3010 includes a microphone (MIC) that is configured to receive external audio signals when device 3000 is in operating modes, such as call mode, recording mode, and voice recognition mode.
  • the received audio signal may be further stored in memory 3004 or transmitted via communication component 3016.
  • the audio component 3010 also includes a speaker for outputting audio signals.
  • the I/O interface 3012 provides an interface between the processing component 3002 and a peripheral interface module, which may be a keyboard, a click wheel, a button, and the like. These buttons may include, but are not limited to: home button, volume buttons, start button, and lock button.
  • Sensor assembly 3014 includes one or more sensors for providing status assessment of various aspects of device 3000 .
  • the sensor component 3014 can detect the on/off state of the device 3000, the relative positioning of components, such as the display and keypad of the device 3000, the sensor component 3014 can also detect a change in the position of the device 3000 or a component of the device 3000, the user The presence or absence of contact with the device 3000, the orientation or acceleration/deceleration of the device 3000 and the temperature change of the device 3000.
  • Sensor assembly 3014 may include a proximity sensor configured to detect the presence of nearby objects in the absence of any physical contact.
  • Sensor assembly 3014 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications.
  • the sensor component 3014 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.
  • Communication component 3016 is configured to facilitate wired or wireless communication between apparatus 3000 and other devices.
  • the apparatus 3000 may access a wireless network based on a communication standard, such as Wi-Fi, 2G or 3G, or a combination thereof.
  • the communication component 3016 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel.
  • the communication component 3016 also includes a near field communication (NFC) module to facilitate short-range communication.
  • NFC near field communication
  • the NFC module may be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.
  • RFID radio frequency identification
  • IrDA infrared data association
  • UWB ultra-wideband
  • Bluetooth Bluetooth
  • apparatus 3000 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable A gate array (FPGA), controller, microcontroller, microprocessor or other electronic component implementation is used to perform the above method.
  • ASICs application specific integrated circuits
  • DSPs digital signal processors
  • DSPDs digital signal processing devices
  • PLDs programmable logic devices
  • FPGA field programmable A gate array
  • controller microcontroller, microprocessor or other electronic component implementation is used to perform the above method.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Mobile Radio Communication Systems (AREA)

Abstract

本公开实施例是关于信息传输方法、装置及通信设备。该方法包括：获取不同输入模态对应的权重；基于至少一个所述输入模态的输入数据，和每个所述输入数据分别对应的所述输入模态的权重，确定所述输入数据的处理结果。

Description

信息传输方法、装置及通信设备 技术领域
本申请涉及无线通信技术领域但不限于无线通信技术领域，尤其涉及信息传输方法、装置及通信设备。
背景技术
智慧交互是下一代移动通信，如第六代(6G，6 th Generation)的一个重要的应用场景。智慧交互是智能体(包括人与智能设备)之间产生的智能交互。现有的智能体交互大多是被动的，依赖于需求的输入，比如人与智慧家居的语音和视觉交互，并且输入都是单模态的。在6G时代，多模态交互将会成为一种常态，多模态交互可以是将多个设备的输入或一个设备的多种输入，发送到集中处理设备或者功能，综合处理这些多模态输入，最终来获得满足用户需求的一种或者多种输出。其中，多种输出也可以通过多个设备进行多输出或者一个设备进行多输出。
发明内容
有鉴于此，本公开实施例提供了一种信息传输方法、装置及通信设备。
根据本公开实施例的第一方面，提供一种信息传输方法，其中，所述方法包括：
获取不同输入模态对应的权重；
基于至少一个所述输入模态的输入数据，和每个所述输入数据分别对应的所述输入模态的权重，确定所述输入数据的处理结果。
根据本公开实施例的第二方面，提供一种信息传输装置，其中，所述装置包括：第一获取模块和确定模块，其中，
所述第一获取模块，配置为获取不同输入模态对应的权重；
所述确定模块，配置为基于至少一个所述输入模态的输入数据，和每个所述输入数据分别对应的所述输入模态的权重，确定所述输入数据的处理结果。
根据本公开实施例的第三方面，提供一种通信设备，包括处理器、收发器、存储器及存储在存储器上并能够有所述处理器运行的可执行程序，所述处理器运行所述可执行程序时执行如第一方面所述信息传输方法的步骤。
根据本公开实施例的第四方面，提供一种存储介质，其上存储由可执行程序，所述可执行程序被处理器执行时实现如第一方面所述信息传输方法的步骤。
本公开实施例提供的信息传输方法、装置及存储介质，多模态处理设备获取不同输入模态对应的权重；基于至少一个所述输入模态的输入数据，和每个所述输入数据分别对应的所述输入模态的权重，确定所述输入数据的处理结果。如此，通过对不同输入模态的输入数据设置权重，区分不同输入模态的输入数据的重要程度，采用权重调整不同输入模态的输入数据的互补性，从而提高多模态处理结果的准确性。
应当理解的是，以上的一般描述和后文的细节描述仅是示例性和解释性的，并不能限制本公开实施例。
附图说明
此处的附图被并入说明书中并构成本说明书的一部分，示出了符合本发明实施例，并与说明书一起用于解释本发明实施例的原理。
图1是根据一示例性实施例示出的一种无线通信系统的结构示意图；
图2是根据一示例性实施例示出的一种信息传输方法的流程示意图；
图3是根据一示例性实施例示出的多模态处理架构图；
图4是根据一示例性实施例示出的一种信息传输装置的框图；
图5是根据一示例性实施例示出的一种用于信息传输的装置的框图。
具体实施方式
这里将详细地对示例性实施例进行说明，其示例表示在附图中。下面的描述涉及附图时，除非另有表示，不同附图中的相同数字表示相同或相似的要素。以下示例性实施例中所描述的实施方式并不代表与本发明实施例相一致的所有实施方式。相反，它们仅是与如所附权利要求书中所详述的、本发明实施例的一些方面相一致的装置和方法的例子。
在本公开实施例使用的术语是仅仅出于描述特定实施例的目的，而非旨在限制本公开实施例。在本公开实施例和所附权利要求书中所使用的单数形式的“一种”、“所述”和“该”也旨在包括多数形式，除非上下文清楚地表示其他含义。还应当理解，本文中使用的术语“和/或”是指并包含一个或多个相关联的列出项目的任何或所有可能组合。
应当理解，尽管在本公开实施例可能采用术语第一、第二、第三等来描述各种信息，但这些信息不应限于这些术语。这些术语仅用来将同一类型的信息彼此区分开。例如，在不脱离本公开实施例范围的情况下，第一信息也可以被称为第二信息，类似地，第二信息也可以被称为第一信息。取决于语境，如在此所使用的词语“如果”可以被解释成为“在……时”或“当……时”或“响应于确定”。
请参考图1，其示出了本公开实施例提供的一种无线通信系统的结构示意图。如图1所示，无线通信系统是基于蜂窝移动通信技术的通信系统，该无线通信系统可以包括：若干个终端11以及若干个基站12。
其中，终端11可以是指向用户提供语音和/或数据连通性的设备。终端11可以经无线接入网(Radio Access Network，RAN)与一个或多个核心网进行通信，终端11可以是物联网终端，如传感器设备、移动电话(或称为 “蜂窝”电话)和具有物联网终端的计算机，例如，可以是固定式、便携式、袖珍式、手持式、计算机内置的或者车载的装置。例如，站(Station，STA)、订户单元(subscriber unit)、订户站(subscriber station)，移动站(mobile station)、移动台(mobile)、远程站(remote station)、接入点、远程终端(remote terminal)、接入终端(access terminal)、用户装置(user terminal)、用户代理(user agent)、用户设备(user device)、或用户终端(user equipment，UE)。或者，终端11也可以是无人飞行器的设备。或者，终端11也可以是车载设备，比如，可以是具有无线通信功能的行车电脑，或者是外接行车电脑的无线通信设备。或者，终端11也可以是路边设备，比如，可以是具有无线通信功能的路灯、信号灯或者其它路边设备等。
基站12可以是无线通信系统中的网络侧设备。其中，该无线通信系统可以是第四代移动通信技术(the 4th generation mobile communication，4G)系统，又称长期演进(Long Term Evolution，LTE)系统；或者，该无线通信系统也可以是5G系统，又称新空口(new radio，NR)系统或5G NR系统。或者，该无线通信系统也可以是5G系统的再下一代系统。其中，5G系统中的接入网可以称为NG-RAN(New Generation-Radio Access Network，新一代无线接入网)。或者，MTC系统。
其中，基站12可以是4G系统中采用的演进型基站(eNB)。或者，基站12也可以是5G系统中采用集中分布式架构的基站(gNB)。当基站12采用集中分布式架构时，通常包括集中单元(central unit，CU)和至少两个分布单元(distributed unit，DU)。集中单元中设置有分组数据汇聚协议(Packet Data Convergence Protocol，PDCP)层、无线链路层控制协议(Radio Link Control，RLC)层、媒体访问控制(Media Access Control，MAC)层的协议栈；分布单元中设置有物理(Physical，PHY)层协议栈，本公开实施例对基站12的具体实现方式不加以限定。
基站12和终端11之间可以通过无线空口建立无线连接。在不同的实施方式中，该无线空口是基于第四代移动通信网络技术(4G)标准的无线空口；或者，该无线空口是基于第五代移动通信网络技术(5G)标准的无线空口，比如该无线空口是新空口；或者，该无线空口也可以是基于5G的更下一代移动通信网络技术标准的无线空口。
在一些实施例中，终端11之间还可以建立E2E(End to End，端到端)连接。比如车联网通信(vehicle to everything，V2X)中的V2V(vehicle to vehicle，车对车)通信、V2I(vehicle to Infrastructure，车对路边设备)通信和V2P(vehicle to pedestrian，车对人)通信等场景。
在一些实施例中，上述无线通信系统还可以包含网络管理设备13。
若干个基站12分别与网络管理设备13相连。其中，网络管理设备13可以是无线通信系统中的核心网设备，比如，该网络管理设备13可以是演进的数据分组核心网(Evolved Packet Core，EPC)中的移动性管理实体(Mobility Management Entity，MME)。或者，该网络管理设备也可以是其它的核心网设备，比如服务网关(Serving GateWay，SGW)、公用数据网网关(Public Data Network GateWay，PGW)、策略与计费规则功能单元(Policy and Charging Rules Function，PCRF)或者归属签约用户服务器(Home Subscriber Server，HSS)等。对于网络管理设备13的实现形态，本公开实施例不做限定。
本公开实施例涉及的执行主体包括但不限于：支持蜂窝移动通信的接入网设备如基站等，以及核心网设备等。
本公开实施例的应用场景为，由于集中处理设备或者功能需要同时基于多个模态的输入综合处理才能得到多模态输出，因此一个模态输入质量的降低会影响整个多模态输出的质量。
本实施例提供一种信息传输方法可以应用于移动通信网络中的基站和/ 或核心中，如图2所示，信息传输方法可以包括：
步骤201：获取不同输入模态对应的权重；
步骤202：基于至少一个输入模态的输入数据，和每个输入数据分别对应的输入模态的权重，确定输入数据的处理结果。
这里，每种输入数据的来源或者形式，可以称为一种输入模态。可以基于输入数据的媒介区分输入模态，例如：语音、视频、文字等分别可以称为一种输入模态；也可以基于输入数据不同获取方式区分模态，例如：分别通过传感器，雷达、红外和加速度计等得到的输入数据可以分别称为一种输入模态的输入数据。以上的每一种都可以称为一种模态。
可以由蜂窝移动通信网络中的接入网设备和/或核心网设备等多模态数据处理设备，对至少一个输入模态的输入数据进行处理得到多模态的处理结果，并对处理结果进行输出。接入网设备可以包括基站等，可以由基站中具有处理能力的处理设备对输入数据进行处理。
可以由蜂窝移动通信网络中的手机终端、物联网终端等用户设备(UE，User Equipment)发送通过图像传感器、音频传感器等获取的输入数据。
至少一个输入模态的输入数据可以是多个UE分别获取的至少一个输入模态的输入数据，也可以是一个UE获取的至少一个输入模态的输入数据。例如，可以由一个手机终端获取音频输入数据和视频输入数据；也可以由一个物联网终端获取音频输入数据，由另一个物联网终端获取视频输入数据。
多模态数据处理设备可以对多个输入模态的输入数据采用大数据分析或AI处理等方式确定多模态处理结果。例如，在进行实时语音识别或翻译中，多模态数据处理设备可以对应UE发送的音频输入数据和视频输入数据，进行语音识别和图像识别，并对识别结果进行大数据分析或AI处理等，进而确定出语义或者翻译结果。
在多模态数据处理中，对不同输入模态的输入数据与处理结果的相关程度不同。例如，对于语音识别处理，音频输入数据的相关程度较高，视频输入数据可以做为辅助识别手段。这里，针对不同的多模态数据处理需求，可以对不同的输入模态设置不同的权重。与处理结果的相关程度较高的输入模态的权重较高，与处理结果的相关程度较低的输入模态的权重较低。通过对不同输入模态的输入数据设置权重，在多模态数据处理中，提高不同输入模态的输入数据的互补性，进而提高处理结果的准确性。
多模态数据处理设备可以预存不同输入模态的权重。在多模态数据处理过程过获取预存的不同输入模态的权重。
在多模态数据处理中，多模态数据处理设备可以基于不同输入模态的输入数据的权重来确定输出结果。例如，在进行实时翻译中，可以设置音频输入数据的权重为90％，设置视频输入数据的权重为10％；综合分析音频输入数据和视频输入数据确定翻译结果。
如此，通过对不同输入模态的输入数据设置权重，区分不同输入模态的输入数据的重要程度，采用权重调整不同输入模态的输入数据的互补性，从而提高多模态处理结果的准确性。
在一个实施例中，不同输入模态对应的权重，是基于不同输入模态的服务质量QOS确定的。
这里，QoS可以包括传输时延和传输速率等。在多模态数据处理中，不同输入模态的QoS可以不同，即不同输入模态的输入数据的传输时延和或传输速率等不同。因此，在同一时间内接收到的不同输入模态的输入数据的信息量是不对等的。
可以基于不同输入模态的服务质量QOS，确定输入模态的权重，用于平衡由于QoS产生的不同输入数据的信息量的不对等情况。
示例性，蜂窝移动通信网络中，音频输入数据、视频输入数据和触觉 输入数据的QoS可以不同，音频输入数据和触觉输入数据的QoS较高，视频输入数据的QoS较低。在进行实时多模态处理时，视频输入数据可能出现的信息量少于语音数据和触觉数据，此时，可以提高视频输入数据的权重。例如，音频输入数据的权重可以占30％，视频输入数据的权重可以占40％，触觉输入数据的权重可以占30％。
如此，通过权重可以平衡由于不同输入模态的不同QoS产生的不同输入模态的输入数据的差异，进而提高多模态处理结果的准确性。
在一个实施例中，权重包括：
用于表征每个所述输入模态占全部所述输入模态比重的相对权重；
或，
采用不同权重数值来指示不同所述输入模态的重要程度的绝对权重。
这里，相对权重可以是比重权数，即可以采用占比的方式反应一个输入模态在所有输入模态评价中的作用。例如：共有音频输入、视频输入和触觉输入三个模态，音频输入的权重为30％，视频输入的权重为40％和触觉输入的权重为30％。采用相对权重可以反应一个输入模态在所有输入模态评价中的作用。
绝对权重可以采用不同的权重数值来指示不同输入模态的重要程度。例如：共有音频输入、视频输入和触觉输入三个模态，音频输入的权重为3，视频输入的权重为4，触觉输入的权重为3。单个输入模态的绝对权重可以不受其他模态的影响，当现有的输入模态发生变化，如加入新输入模态或去除一个输入模态时，输入模态的绝对权重不发生变化。如此，可以提高权重的适应性。
在一个实施例中，基于至少一个输入模态的输入数据，和每个输入数据分别对应的输入模态的权重，确定输入数据的处理结果，包括：
基于通过移动通信网络接收的至少一个输入模态的输入数据，和每个 输入数据分别对应的输入模态的权重，确定输入数据的处理结果。
可以由蜂窝移动通信网络中的手机终端、物联网终端等用户设备(UE，User Equipment)通过移动通信网络向接入网设备或核心网设备发送多个输入模态的输入数据。
多个输入模态的输入数据可以是多个UE分别获取的各输入模态的输入数据，也可以是一个UE获取的各输入模态的输入数据。
在一个实施例中，获取不同输入模态对应的权重，包括：
接收指示不同输入模态对应的权重的指示信息；
获取预先商定的不同输入模态对应的权重。
如图3所示，集中处理设备或者集中处理功能可以基于多模态处理的需求确定各输入模态对应的权重，并发送给接入网设备或核心网设备。集中处理设备可以是云端处理设备等。集中处理功能可以是云端用于多模态处理的功能模块等。
各输入模态对应的权重也可以由手机终端、物联网终端等用户设备等多模态输入设备基于多模态处理的需求确定并发送给接入网设备或核心网设备。
各输入模态对应的权重也可以预先约定，并存储在接入网设备或核心网设备。
在一个实施例中，信息传输方法还包括：
基于补偿规则，采用至少一个输入模态中至少一第一输入模态的输入数据，补偿至少一个输入模态中至少一第二输入模态的输入数据。
在多模态数据处理中，多模态数据处理设备需要同时基于多个输入模态的输入处理综合处理才能得到多模态的处理结果，因此，如果一个输入模态的输入数据的输入质量或数据量的降低会影响整个多模态输出的处理结果的质量。输入质量可以包括传输时延和/或传输速率等。
这里，采用第一输入模态的输入数据，补偿第二输入模态的输入数据，可以包括：通过提高第一输入模态的输入数据的输入质量，补偿第二输入模态的输入数据的输入质量等。
补偿规则可以基于不同输入模态的对应关系设置。例如，可以采用多个第一输入模态的输入数据补偿一个第二输入模态的输入数据，也可以采用多个第一输入模态的输入数据补偿多个第二输入模态的输入数据。
补偿规则也可以规定补偿的对应关系，例如：输入模态A降低后，可以通过输入模态B提高来弥补，也可以通过输入模态C提高来弥补。
补偿规则可以规定补偿的数据量和/或输入质量等。例如：补偿规则可以规定用于补偿第二输入模态的输入数据的第一输入模态的输入数据的补偿量等。这里，补偿量可以是数据量也可以是输入质量等。
示例性的，音频输入模态在传输过程中数据量或传输质量降低，基站或者核心网可以提高视频输入模态的数据量或传输质量。
如此，通过不同输入模态之间的补偿，降低由于输入模态的输入质量或数据量变化对多模态处理结果的影响，提高多模态处理结果质量。
在一个实施例中，基于补偿规则，采用至少一个输入模态中至少一个第一输入模态的输入数据，补偿至少一个输入模态中至少一第二输入模态的输入数据，包括：
采用至少一个第一输入模态的输入数据的QoS，补偿至少一个第二输入模态的输入数据的QoS。
示例性的，当第二输入模态在传输过程中传输质量降低，例如QoS值降低，基站或者核心网可以提高第一模态输入的传输质量，例如提高其他模态输入的QoS。如此，可以减少由于输入模态的QoS的变化对多模态处理结果的影响，进而提高多模态处理结果的准确性。
在一个实施例中，补偿规则是基于第一输入模态的权重和第二输入模 态的权重确定的。
补偿规则可以规定补偿的数据量和/或输入质量等。例如：补偿规则可以规定用于补偿第二输入模态的输入数据的第一输入模态的输入数据的补偿量等。
补偿规则可以基于权重确定补偿量，例如，针对相对权重较大的第二输入模态，可以采用较多相对权重较小第一输入模态的补偿量进行补偿。针对相对权重较小的第二输入模态，可以采用较少相对权重较大第一输入模态的补偿量进行补偿。
示例性的：如果音频输入模态的权重占30％，视频输入模态的权重占40％。如果音频输入模态的QoS降低了10％，由于视频输入模态权重较大，因此，只需要较少的视频输入模态补偿量进行补偿，例如，可以将视频输入模态的QoS提高5％进行补偿。
如此，基于权重确定不同输入模态之间的补偿量，使得在多模态数据处理中不同模态之间的输入数据可以满足多模态处理的需求，降低由于输入模态的输入质量或数据量变化对多模态处理结果的影响，提高多模态处理结果质量。
在一个实施例中，信息传输方法还包括以下至少之一：
接收指示补偿规则的指示信息；
获取预先商定的补偿规则。
集中处理设备或者集中处理功能可以基于各输入模态的权重等确定补偿规则，并发送给接入网设备或核心网设备。
补偿规则也可以由手机终端、物联网终端用户设备等多模态输入设备基于各输入模态的权重等确定并发送给接入网设备或核心网设备。
补偿规则也可以预先约定，并存储在接入网设备或核心网设备。
以下结合上述任意实施例提供一个具体示例：
本发明提出了一种保证输出质量的方法。
1、基站和/或核心网获得多个输入模态的权重。
1)该权重可以由集中处理设备或者功能发送给基站或核心网。
2)该权重可以由多模态输入设备发送给基站或核心网。
3)该权重可以是相对值也可以是绝对值。
4)该权重可以基于QoS来衡量，例如时延、速率等。
5)例如音频输入占30％，视频输入占40，触觉输入占30％。
2、基站还可以获得多个输入模态之间的弥补关系，即补偿规则。
1)例如输入A降低a后可以通过B提高b来弥补，也可以通过C提高c来弥补。
2)该弥补关系可以由集中处理设备或者功能发送给基站或核心网。
3、当某个模态在传输过程中传输质量降低，例如QoS值降低，基站或者核心网可以提高其他模态输入的传输质量，例如提高其他模态输入的QoS。
例如音频输入的QoS降低了10％，则把视频输入的QoS提高5％，就可以保证整体输出的质量或者准确性。
可以通过提高一个或多个模态的输入的传输质量来弥补一个或多个模态的输入质量降低带来的损害。
具体的弥补算法可以基于实现，也可以基于多个输入模态之间的弥补关系。
本发明实施例还提供了一种信息传输装置，应用于基站，图4为本发明实施例提供的信息传输装置100的组成结构示意图；如图4所示，装置100包括：第一获取模块110和确定模块120，其中，
第一获取模块110，配置为获取不同输入模态对应的权重；
确定模块120，配置为基于至少一个输入模态的输入数据，和每个输入 数据分别对应的输入模态的权重，确定输入数据的处理结果。
在一个实施例中，不同输入模态对应的权重，是基于不同输入模态的服务质量QoS确定的。
在一个实施例中，权重包括：
用于表征每个所述输入模态占全部所述输入模态比重的相对权重；
或，
采用不同权重数值来指示不同所述输入模态的重要程度的绝对权重。
在一个实施例中，确定模块120，包括：
确定子模块121，配置为基于通过移动通信网络接收的至少一个输入模态的输入数据，和每个输入数据分别对应的输入模态的权重，确定输入数据的处理结果。
在一个实施例中，第一获取模块110，包括：
第一获取子模块111，配置为接收指示不同输入模态对应的权重的指示信息；
第二获取子模块112，配置为获取预先商定的不同输入模态对应的权重。
在一个实施例中，装置100还包括：
补偿模块130，配置为基于补偿规则，采用至少一个输入模态中至少一第一输入模态的输入数据，补偿至少一个输入模态中至少一第二输入模态的输入数据。
在一个实施例中，补偿模块130，包括：
补偿子模块131，配置为采用至少一个第一输入模态的输入数据的QoS，补偿至少一个第二输入模态的输入数据的QoS。
在一个实施例中，补偿规则是基于第一输入模态的权重和第二输入模态的权重确定的。
在一个实施例中，装置100还包括以下至少之一：
接收模块140，配置为接收指示补偿规则的指示信息；
第二获取模块150，配置为获取预先商定的补偿规则。
在示例性实施例中，第一获取模块110、确定模块120、补偿模块130、接收模块140和第二获取模块150等可以被一个或多个中央处理器(CPU，Central Processing Unit)、图形处理器(GPU，Graphics Processing Unit)、基带处理器(BP，baseband processor)、应用专用集成电路(ASIC，Application Specific Integrated Circuit)、DSP、可编程逻辑器件(PLD，Programmable Logic Device)、复杂可编程逻辑器件(CPLD，Complex Programmable Logic Device)、现场可编程门阵列(FPGA，Field-Programmable Gate Array)、通用处理器、控制器、微控制器(MCU，Micro Controller Unit)、微处理器(Microprocessor)、或其他电子元件实现，也可以结合一个或多个射频(RF，radio frequency)天线实现，用于执行前述方法。
图5是根据一示例性实施例示出的一种用于信息传输的装置3000的框图。例如，装置3000可以是移动电话，计算机，数字广播终端，消息收发设备，游戏控制台，平板设备，医疗设备，健身设备，个人数字助理等。
参照图5，装置3000可以包括以下一个或多个组件：处理组件3002，存储器3004，电源组件3006，多媒体组件3008，音频组件3010，输入/输出(I/O)的接口3012，传感器组件3014，以及通信组件3016。
处理组件3002通常控制装置3000的整体操作，诸如与显示，电话呼叫，数据通信，相机操作和记录操作相关联的操作。处理组件3002可以包括一个或多个处理器3020来执行指令，以完成上述的方法的全部或部分步骤。此外，处理组件3002可以包括一个或多个模块，便于处理组件3002和其他组件之间的交互。例如，处理组件3002可以包括多媒体模块，以方便多媒体组件3008和处理组件3002之间的交互。
存储器3004被配置为存储各种类型的数据以支持在设备3000的操作。这些数据的示例包括用于在装置3000上操作的任何应用程序或方法的指令，联系人数据，电话簿数据，消息，图片，视频等。存储器3004可以由任何类型的易失性或非易失性存储设备或者它们的组合实现，如静态随机存取存储器(SRAM)，电可擦除可编程只读存储器(EEPROM)，可擦除可编程只读存储器(EPROM)，可编程只读存储器(PROM)，只读存储器(ROM)，磁存储器，快闪存储器，磁盘或光盘。
电源组件3006为装置3000的各种组件提供电力。电源组件3006可以包括电源管理系统，一个或多个电源，及其他与为装置3000生成、管理和分配电力相关联的组件。
多媒体组件3008包括在装置3000和用户之间的提供一个输出接口的屏幕。在一些实施例中，屏幕可以包括液晶显示器(LCD)和触摸面板(TP)。如果屏幕包括触摸面板，屏幕可以被实现为触摸屏，以接收来自用户的输入信号。触摸面板包括一个或多个触摸传感器以感测触摸、滑动和触摸面板上的手势。触摸传感器可以不仅感测触摸或滑动动作的边界，而且还检测与触摸或滑动操作相关的持续时间和压力。在一些实施例中，多媒体组件3008包括一个前置摄像头和/或后置摄像头。当设备3000处于操作模式，如拍摄模式或视频模式时，前置摄像头和/或后置摄像头可以接收外部的多媒体数据。每个前置摄像头和后置摄像头可以是一个固定的光学透镜系统或具有焦距和光学变焦能力。
音频组件3010被配置为输出和/或输入音频信号。例如，音频组件3010包括一个麦克风(MIC)，当装置3000处于操作模式，如呼叫模式、记录模式和语音识别模式时，麦克风被配置为接收外部音频信号。所接收的音频信号可以被进一步存储在存储器3004或经由通信组件3016发送。在一些实施例中，音频组件3010还包括一个扬声器，用于输出音频信号。
I/O接口3012为处理组件3002和外围接口模块之间提供接口，上述外围接口模块可以是键盘，点击轮，按钮等。这些按钮可包括但不限于：主页按钮、音量按钮、启动按钮和锁定按钮。
传感器组件3014包括一个或多个传感器，用于为装置3000提供各个方面的状态评估。例如，传感器组件3014可以检测到设备3000的打开/关闭状态，组件的相对定位，例如组件为装置3000的显示器和小键盘，传感器组件3014还可以检测装置3000或装置3000一个组件的位置改变，用户与装置3000接触的存在或不存在，装置3000方位或加速/减速和装置3000的温度变化。传感器组件3014可以包括接近传感器，被配置用来在没有任何的物理接触时检测附近物体的存在。传感器组件3014还可以包括光传感器，如CMOS或CCD图像传感器，用于在成像应用中使用。在一些实施例中，该传感器组件3014还可以包括加速度传感器，陀螺仪传感器，磁传感器，压力传感器或温度传感器。
通信组件3016被配置为便于装置3000和其他设备之间有线或无线方式的通信。装置3000可以接入基于通信标准的无线网络，如Wi-Fi，2G或3G，或它们的组合。在一个示例性实施例中，通信组件3016经由广播信道接收来自外部广播管理系统的广播信号或广播相关信息。在一个示例性实施例中，通信组件3016还包括近场通信(NFC)模块，以促进短程通信。例如，在NFC模块可基于射频识别(RFID)技术，红外数据协会(IrDA)技术，超宽带(UWB)技术，蓝牙(BT)技术和其他技术来实现。
在示例性实施例中，装置3000可以被一个或多个应用专用集成电路(ASIC)、数字信号处理器(DSP)、数字信号处理设备(DSPD)、可编程逻辑器件(PLD)、现场可编程门阵列(FPGA)、控制器、微控制器、微处理器或其他电子元件实现，用于执行上述方法。
在示例性实施例中，还提供了一种包括指令的非临时性计算机可读存 储介质，例如包括指令的存储器3004，上述指令可由装置3000的处理器3020执行以完成上述方法。例如，非临时性计算机可读存储介质可以是ROM、随机存取存储器(RAM)、CD-ROM、磁带、软盘和光数据存储设备等。
本领域技术人员在考虑说明书及实践这里公开的发明后，将容易想到本发明实施例的其它实施方案。本申请旨在涵盖本发明实施例的任何变型、用途或者适应性变化，这些变型、用途或者适应性变化遵循本发明实施例的一般性原理并包括本公开实施例未公开的本技术领域中的公知常识或惯用技术手段。此外，对于本技术领域的普通技术人员来说，在不脱离本公开原理的前提下，还可以对本公开中各个实施方式的步骤或模块进行替换或组合，这些替换、组合也应视为本公开的保护范围。说明书和实施例仅被视为示例性的，本发明实施例的要求保护的范围和精神由下面的权利要求指出。
应当理解的是，本发明实施例并不局限于上面已经描述并在附图中示出的精确结构，并且可以在不脱离其范围进行各种修改和改变。本发明实施例的范围仅由所附的权利要求来限制。

Claims (19)

  1. 一种信息传输方法，其中，所述方法包括：
    获取不同输入模态对应的权重；
    基于至少一个所述输入模态的输入数据，和每个所述输入数据分别对应的所述输入模态的权重，确定所述输入数据的处理结果。
  2. 根据权利要求1所述的方法，其中，
    不同所述输入模态对应的权重，是基于不同所述输入模态的服务质量QOS确定的。
  3. 根据权利要求1所述的方法，其中，所述权重包括：
    用于表征每个所述输入模态占全部所述输入模态比重的相对权重；
    或，
    采用不同权重数值来指示不同所述输入模态的重要程度的绝对权重。
  4. 根据权利要求1所述的方法，其中，所述基于至少一个所述输入模态的输入数据，和每个所述输入数据分别对应的所述输入模态的权重，确定所述输入数据的处理结果，包括：
    基于通过移动通信网络接收的至少一个所述输入模态的输入数据，和每个所述输入数据分别对应的所述输入模态的权重，确定所述输入数据的处理结果。
  5. 根据权利要求1所述的方法，其中，所述获取不同输入模态对应的权重，包括：
    接收指示不同所述输入模态对应的权重的指示信息；
    获取预先商定的不同所述输入模态对应的权重。
  6. 根据权利要求1至5任一项所述的方法，其中，所述方法还包括：
    基于补偿规则，采用至少一个所述输入模态中至少一第一输入模态的输入数据，补偿至少一个所述输入模态中至少一第二输入模态的输入数据。
  7. 根据权利要求6所述的方法，其中，所述基于补偿规则，采用至少一个所述输入模态中至少一个第一输入模态的输入数据，补偿至少一个所述输入模态中至少一第二输入模态的输入数据，包括：
    采用至少一个所述第一输入模态的输入数据的QoS，补偿所述至少一个所述第二输入模态的输入数据的QoS。
  8. 根据权利要求6所述的方法，其中，
    所述补偿规则是基于所述第一输入模态的权重和第二输入模态的权重确定的。
  9. 根据权利要求6所述的方法，其中，所述方法还包括以下至少之一：
    接收指示所述补偿规则的指示信息；
    获取预先商定的所述补偿规则。
  10. 一种信息传输装置，其中，所述装置包括：第一获取模块和确定模块，其中，
    所述第一获取模块，配置为获取不同输入模态对应的权重；
    所述确定模块，配置为基于至少一个所述输入模态的输入数据，和每个所述输入数据分别对应的所述输入模态的权重，确定所述输入数据的处理结果。
  11. 根据权利要求10所述的装置，其中，
    不同所述输入模态对应的权重，是基于不同所述输入模态的服务质量QOS确定的。
  12. 根据权利要求10所述的装置，其中，所述权重包括：
    用于表征每个所述输入模态占全部所述输入模态比重的相对权重；
    或，
    采用不同权重数值来指示不同所述输入模态的重要程度的绝对权重。
  13. 根据权利要求10所述的装置，其中，所述确定模块，包括：
    确定子模块，配置为基于通过移动通信网络接收的至少一个所述输入模态的输入数据，和每个所述输入数据分别对应的所述输入模态的权重，确定所述输入数据的处理结果。
  14. 根据权利要求10所述的装置，其中，所述第一获取模块，包括：
    第一获取子模块，配置为接收指示不同所述输入模态对应的权重的指示信息；
    第二获取子模块，配置为获取预先商定的不同所述输入模态对应的权重。
  15. 根据权利要求10至14任一项所述的装置，其中，所述装置还包括：
    补偿模块，配置为基于补偿规则，采用至少一个所述输入模态中至少一第一输入模态的输入数据，补偿至少一个所述输入模态中至少一第二输入模态的输入数据。
  16. 根据权利要求15所述的装置，其中，所述补偿模块，包括：
    补偿子模块，配置为采用至少一个所述第一输入模态的输入数据的QoS，补偿所述至少一个所述第二输入模态的输入数据的QoS。
  17. 根据权利要求15所述的装置，其中，
    所述补偿规则是基于所述第一输入模态的权重和第二输入模态的权重确定的。
  18. 根据权利要求15所述的装置，其中，所述装置还包括以下至少之一：
    接收模块，配置为接收指示所述补偿规则的指示信息；
    第二获取模块，配置为获取预先商定的所述补偿规则。
  19. 一种通信设备，包括处理器、收发器、存储器及存储在存储器上并能够有所述处理器运行的可执行程序，所述处理器运行所述可执行程序 时执行如权利要求1至9任一项所述信息传输方法的步骤。
PCT/CN2020/103429 2020-07-22 2020-07-22 信息传输方法、装置及通信设备 WO2022016406A1 (zh)

Priority Applications (4)

Application Number Priority Date Filing Date Title
US18/010,791 US20230239726A1 (en) 2020-07-22 2020-07-22 Method for information transmission, and communication device
CN202080001633.8A CN114342335B (zh) 2020-07-22 2020-07-22 信息传输方法、装置及通信设备
PCT/CN2020/103429 WO2022016406A1 (zh) 2020-07-22 2020-07-22 信息传输方法、装置及通信设备
EP20945998.1A EP4187930A4 (en) 2020-07-22 2020-07-22 INFORMATION TRANSMISSION METHOD AND APPARATUS, AND COMMUNICATION DEVICE

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2020/103429 WO2022016406A1 (zh) 2020-07-22 2020-07-22 信息传输方法、装置及通信设备

Publications (1)

Publication Number Publication Date
WO2022016406A1 true WO2022016406A1 (zh) 2022-01-27

Family

ID=79728409

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/103429 WO2022016406A1 (zh) 2020-07-22 2020-07-22 信息传输方法、装置及通信设备

Country Status (4)

Country Link
US (1) US20230239726A1 (zh)
EP (1) EP4187930A4 (zh)
CN (1) CN114342335B (zh)
WO (1) WO2022016406A1 (zh)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104170337A (zh) * 2012-03-15 2014-11-26 微软公司 无线网络之上的多模态通信优先级
CN106598243A (zh) * 2016-12-08 2017-04-26 广东工业大学 一种多模态自适应光标控制方法及系统
CN107480196A (zh) * 2017-07-14 2017-12-15 中国科学院自动化研究所 一种基于动态融合机制的多模态词汇表示方法
CN107911156A (zh) * 2017-12-29 2018-04-13 深圳市华瑞安科技有限公司 数字波束形成方法及装置

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3302634B2 (ja) * 1997-12-16 2002-07-15 松下電器産業株式会社 データ通信装置及び方法
IL130895A (en) * 1999-07-12 2003-10-31 Ectel Ltd Method and system for controlling quality of service over a telecommunication network
CN101383650B (zh) * 2008-09-03 2012-05-23 中国科学技术大学 基于模糊满意度的多业务多天线广播信道调度方法
CN101685638B (zh) * 2008-09-25 2011-12-21 华为技术有限公司 一种语音信号增强方法及装置
KR102000590B1 (ko) * 2015-12-16 2019-07-16 니폰 덴신 덴와 가부시끼가이샤 오디오 비주얼 품질 추정 장치, 오디오 비주얼 품질 추정 방법, 및 프로그램
CN106797335B (zh) * 2016-11-29 2020-04-07 深圳前海达闼云端智能科技有限公司 数据传输方法、数据传输装置、电子设备和计算机程序产品
CN108966290B (zh) * 2018-07-24 2021-04-16 Oppo广东移动通信有限公司 网络连接方法及相关产品
CN110837758B (zh) * 2018-08-17 2023-06-02 杭州海康威视数字技术股份有限公司 一种关键词输入方法、装置及电子设备
CN111274372A (zh) * 2020-01-15 2020-06-12 上海浦东发展银行股份有限公司 用于人机交互的方法、电子设备和计算机可读存储介质

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104170337A (zh) * 2012-03-15 2014-11-26 微软公司 无线网络之上的多模态通信优先级
CN106598243A (zh) * 2016-12-08 2017-04-26 广东工业大学 一种多模态自适应光标控制方法及系统
CN107480196A (zh) * 2017-07-14 2017-12-15 中国科学院自动化研究所 一种基于动态融合机制的多模态词汇表示方法
CN107911156A (zh) * 2017-12-29 2018-04-13 深圳市华瑞安科技有限公司 数字波束形成方法及装置

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4187930A4 *

Also Published As

Publication number Publication date
US20230239726A1 (en) 2023-07-27
EP4187930A1 (en) 2023-05-31
CN114342335A (zh) 2022-04-12
EP4187930A4 (en) 2024-04-03
CN114342335B (zh) 2024-02-13

Similar Documents

Publication Publication Date Title
US20230034681A1 (en) Positioning processing method and apparatus, base station, terminal device, and storage medium
CN109144705B (zh) 应用程序管理方法、移动终端及计算机可读存储介质
WO2022236639A1 (zh) 资源配置方法、装置、通信设备和存储介质
CN104301308B (zh) 通话控制方法及装置
WO2022052024A1 (zh) 参数配置方法、装置、通信设备和存储介质
US20230134028A1 (en) METHOD AND APPARATUS OF POSITIONING BETWEEN UEs, COMMUNICATION DEVICE AND STORAGE MEDIUM
WO2022094882A1 (zh) 通信处理方法、装置、通信设备及存储介质
WO2022000200A1 (zh) 定位参考信号配置方法及装置、用户设备、存储介质
WO2023245576A1 (zh) Ai模型确定方法、装置、通信设备及存储介质
WO2021243723A1 (zh) 位置确定方法、装置、通信设备及存储介质
WO2023201641A1 (zh) 发送网络能力信息的方法、装置、通信设备及存储介质
WO2022056847A1 (zh) 终端的定位方法、装置、通信设备及存储介质
WO2022141095A1 (zh) 辅助ue的选择方法、装置及存储介质
WO2021120204A1 (zh) 信号测量方法、装置、通信设备及存储介质
WO2022016406A1 (zh) 信息传输方法、装置及通信设备
WO2022036610A1 (zh) 一种通信方法、通信装置及存储介质
WO2022205385A1 (zh) 测量间隔处理方法、装置、通信设备及存储介质
WO2022126642A1 (zh) 信息传输方法、装置、通信设备及存储介质
WO2022016465A1 (zh) 定位测量方法、定位测量装置及存储介质
WO2021217417A1 (zh) 信息传输方法、装置、通信设备及存储介质
US20220295447A1 (en) Network access method, apparatus, communication device, and storage medium
WO2023212951A1 (zh) 测距方法、装置、通信设备和存储介质
WO2024021097A1 (zh) 信道状态信息的测量方法、装置、通信设备及存储介质
WO2023065080A1 (zh) 测距能力开放的方法、装置、通信设备及存储介质
WO2023221025A1 (zh) 波束确定方法、装置、通信设备及存储介质

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20945998

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2020945998

Country of ref document: EP

Effective date: 20230222