WO2023042937A1

WO2023042937A1 - Method and device for transmitting/receiving signal on basis of meta-lens artificial intelligence system in wireless communication system

Info

Publication number: WO2023042937A1
Application number: PCT/KR2021/012704
Authority: WO
Inventors: 오재기; 김성진; 박재용; 장지환; 이명희; 하업성
Original assignee: 엘지전자 주식회사
Priority date: 2021-09-16
Filing date: 2021-09-16
Publication date: 2023-03-23
Also published as: KR20240065063A

Abstract

The present disclosure may provide a method for operating a base station in a wireless communication system. The method for operating a base station may comprise the steps of: acquiring user recognition information from a first terminal; receiving a control value on the basis of meta-lens feature factors by using the acquired user recognition information; and transmitting the acquired control value to the first terminal and a second terminal.

Description

Method and apparatus for transmitting and receiving signals based on the metalens artificial intelligence system in a wireless communication system

The following description relates to a wireless communication system, and relates to a method and apparatus for transmitting and receiving signals based on the Metalens artificial intelligence system in a wireless communication system.

In particular, it relates to a method and apparatus for providing a metalens artificial intelligence system in consideration of a user-based image input/output device.

A wireless access system is widely deployed to provide various types of communication services such as voice and data. In general, a wireless access system is a multiple access system capable of supporting communication with multiple users by sharing available system resources (bandwidth, transmission power, etc.). Examples of the multiple access system include a code division multiple access (CDMA) system, a frequency division multiple access (FDMA) system, a time division multiple access (TDMA) system, an orthogonal frequency division multiple access (OFDMA) system, and a single carrier frequency (SC-FDMA) system. division multiple access) system.

In particular, as many communication devices require large communication capacity, an enhanced mobile broadband (eMBB) communication technology compared to existing radio access technology (RAT) has been proposed. In addition, communication systems considering reliability and latency sensitive services/UEs as well as massive Machine Type Communications (MTC) providing various services anytime and anywhere by connecting multiple devices and objects have been proposed. Various technical configurations for this have been proposed.

The present disclosure relates to a method and apparatus for transmitting and receiving signals based on a metalens artificial intelligence system in a wireless communication system.

The present disclosure relates to a method and apparatus for providing a metalens artificial intelligence system in consideration of a user-based image input/output device in a wireless communication system.

The present disclosure relates to a method of obtaining user-related input information in a wireless communication system and controlling an image input/output device through learning in a metalens artificial intelligence system based on the acquired user-related input information.

The present disclosure relates to a method for controlling a metalens artificial intelligence system based on federated learning in a wireless communication system.

The technical objects to be achieved in the present disclosure are not limited to the above-mentioned matters, and other technical problems not mentioned above are common knowledge in the art to which the technical configuration of the present disclosure is applied from the embodiments of the present disclosure to be described below. can be considered by those who have

As an example of the present disclosure, in a base station operating method in a wireless communication system, receiving user recognition information from a first terminal, obtaining a control value based on a metalens characteristic factor using the obtained user recognition information and transmitting the obtained control value to the first terminal and the second terminal.

In addition, as an example of the present disclosure, a method of operating a terminal in a wireless communication system includes transmitting user recognition information to a base station and receiving a control value derived by the base station based on the transmitted user recognition information. can do.

In addition, as an example of the present disclosure, in a base station of a wireless communication system, including a transceiver and a processor connected to the transceiver, the processor receives user recognition information from a first terminal using the transceiver, and the obtained user recognition information It is possible to obtain a control value based on the metalens characteristic factor using , and transmit the obtained control value to the first terminal and the second terminal using the transceiver.

In addition, as an example of the present disclosure, it includes a transceiver and a processor connected to the transceiver, and the processor transmits user recognition information to a base station using the transceiver, and is transmitted by the base station based on the user recognition information transmitted using the transceiver. The derived control value may be received.

In addition, as an example of the present disclosure, in an apparatus including at least one memory and at least one processor functionally connected to the at least one memory, the at least one processor enables the device to recognize a user from a first terminal. It is possible to receive information, obtain a control value based on the metalens characteristic factor using the obtained user recognition information, and transmit the obtained control value to the first terminal and the second terminal.

In addition, as an example of the present disclosure, in a non-transitory computer-readable medium storing at least one instruction, at least one executable by a processor Includes commands of, wherein at least one command receives user recognition information from the first terminal, obtains a control value based on a metalens characteristic factor using the obtained user recognition information, and converts the obtained control value to It can be controlled to transmit to the first terminal and the second terminal.

The following items may be commonly applied to the above-described base station, terminal, device, and computer recording medium.

Also, as an example of the present disclosure, the obtained control value may be determined based on a codebook based on a metalens characteristic factor.

In addition, as an example of the present disclosure, the metalens characteristic factor may be determined based on at least one of focus position information, focal distance information, and radius of curvature information based on each of the metalens elements.

Also, as an example of the present disclosure, focus position information, focal distance information, and radius of curvature information are expressed as respective indices based on quantization, and a control value may be derived based on each index.

Also, as an example of the present disclosure, user recognition information may include at least one or more of user direction information, user distance information, and user status information.

In addition, as an example of the present disclosure, the first terminal is a terminal having a video input function, and can obtain user recognition information based on the video input function.

In addition, as an example of the present disclosure, a first terminal obtains location information of the first terminal, transmits the location information of the first terminal together with user recognition information to a base station, and the base station further transmits the location information of the first terminal. A control value can be derived by taking this into account.

Also, as an example of the present disclosure, the second terminal may be a terminal having a video output function.

In addition, as an example of the present disclosure, a base station may include at least one function of a server and an access point.

In addition, as an example of the present disclosure, the base station includes a Metalens artificial intelligence system, and the Metalens artificial intelligence system performs learning based on a learning model, but the obtained user recognition information is an input of the learning model and a control value. may be an output of the learning model.

In addition, as an example of the present disclosure, the learning model operates based on reinforcement learning, and the reinforcement learning uses state information and reward information as inputs based on the acquired user recognition information and control value, and corresponds to the control value. Behavioral information can be output.

In addition, as an example of the present disclosure, the learning model operates based on Thomson sampling, and Thomson sampling can perform learning based on compensation information having α and β parameters based on the acquired user recognition information and control value. there is.

The following effects may be obtained by embodiments based on the present disclosure.

In embodiments based on the present disclosure, it is possible to provide a method for transmitting and receiving a signal based on a metalens artificial intelligence system.

In embodiments based on the present disclosure, a metalens artificial intelligence system may be provided in consideration of a user-based image input/output device.

In embodiments based on the present disclosure, user-related input information may be acquired, and based on this, the image input/output device may be controlled through learning in the Metalens artificial intelligence system.

In embodiments based on the present disclosure, a method for controlling a metalens artificial intelligence system based on federated learning may be provided.

Effects obtainable in the embodiments of the present disclosure are not limited to the above-mentioned effects, and other effects not mentioned are technical fields to which the technical configuration of the present disclosure is applied from the description of the following embodiments of the present disclosure. can be clearly derived and understood by those skilled in the art.

That is, unintended effects according to implementing the configuration described in the present disclosure may also be derived by those skilled in the art from the embodiments of the present disclosure.

The accompanying drawings are provided to aid understanding of the present disclosure, and may provide embodiments of the present disclosure together with a detailed description. However, the technical features of the present disclosure are not limited to specific drawings, and features disclosed in each drawing may be combined with each other to form a new embodiment. Reference numerals in each drawing may mean structural elements.

1 is a diagram illustrating an exemplary communication system according to an embodiment of the present disclosure.

2 is a diagram showing an example of a wireless device according to an embodiment of the present disclosure.

3 is a diagram illustrating another example of a wireless device according to an embodiment of the present disclosure.

4 is a diagram illustrating an example of AI (Artificial Intelligence) according to an embodiment of the present disclosure.

5 is a diagram showing federated learning according to an embodiment of the present disclosure.

6 is a diagram illustrating an image input/output device according to an embodiment of the present disclosure.

7 is a diagram illustrating an OIS principle according to an embodiment of the present disclosure.

8 is a diagram showing a metalens structure according to an embodiment of the present disclosure.

9 is a diagram illustrating an artificial intelligence system for controlling a metalens for user-based image input/output according to an embodiment of the present disclosure.

10 is a diagram illustrating a method of controlling a metalens according to an embodiment of the present disclosure.

11 is a diagram illustrating comparison of viewing distances between curved and flat image outputs according to an embodiment of the present disclosure.

12 is a diagram illustrating a comparison of viewing distances between curved and flat image outputs according to an embodiment of the present disclosure.

13 is a diagram illustrating a metalens control method according to an embodiment of the present disclosure.

14 is a diagram illustrating the operation of a distributed learning artificial intelligence system according to an embodiment of the present disclosure.

15 is a diagram illustrating a method of controlling a metalens based on an artificial intelligence system according to an embodiment of the present disclosure.

16 is a flowchart illustrating a method of controlling image input/output based on a metalens according to an embodiment of the present disclosure.

17 is a diagram illustrating a method of detecting an object in a user recognizer according to an embodiment of the present disclosure.

18 is a diagram illustrating a method of detecting an object in a user recognizer according to an embodiment of the present disclosure.

19 is a diagram illustrating final input/output according to an embodiment of the present disclosure.

20 is a diagram illustrating a structure of a metalens control system using reinforcement learning (or MAB) according to an embodiment of the present disclosure.

21 is a block diagram of a metalens control system using reinforcement learning according to an embodiment of the present disclosure.

22 is a diagram illustrating reinforcement learning using DDPG according to an embodiment of the present disclosure.

23 is a diagram illustrating a selection artificial intelligence (MAB AI, 2310) system using Thompson sampling according to an embodiment of the present disclosure.

24 is a diagram showing the structure of a metalens controller according to an embodiment of the present disclosure.

25 is a diagram illustrating a method of generating a metalens control value based on a focus position according to an embodiment of the present disclosure.

26 is a diagram illustrating a method of realizing a zoom lens and a curved lens based on a metalens curvature change according to an embodiment of the present disclosure.

27 is a diagram illustrating a system operating through federated learning of various image input/output devices according to an embodiment of the present disclosure.

28 is a diagram illustrating a method of operating a base station according to an embodiment of the present disclosure.

29 is a flowchart illustrating a method of operating a terminal according to an embodiment of the present disclosure.

The following embodiments are those that combine elements and features of the present disclosure in a predetermined form. Each component or feature may be considered optional unless explicitly stated otherwise. Each component or feature may be implemented in a form not combined with other components or features. In addition, an embodiment of the present disclosure may be configured by combining some elements and/or features. The order of operations described in the embodiments of the present disclosure may be changed. Some components or features of one embodiment may be included in another embodiment, or may be replaced with corresponding components or features of another embodiment.

In the description of the drawings, procedures or steps that may obscure the gist of the present disclosure have not been described, and procedures or steps that can be understood by those skilled in the art have not been described.

Throughout the specification, when a part is said to "comprising" or "including" a certain element, it means that it may further include other elements, not excluding other elements, unless otherwise stated. do. In addition, terms such as “… unit”, “… unit”, and “module” described in the specification mean a unit that processes at least one function or operation, which is hardware or software or a combination of hardware and software. can be implemented as Also, "a or an", "one", "the" and similar related words in the context of describing the present disclosure (particularly in the context of the claims below) Unless indicated or otherwise clearly contradicted by context, both the singular and the plural can be used.

Embodiments of the present disclosure in this specification have been described with a focus on a data transmission/reception relationship between a base station and a mobile station. Here, a base station has meaning as a terminal node of a network that directly communicates with a mobile station. A specific operation described as being performed by a base station in this document may be performed by an upper node of the base station in some cases.

That is, in a network composed of a plurality of network nodes including a base station, various operations performed for communication with a mobile station may be performed by the base station or network nodes other than the base station. At this time, the 'base station' is a term such as a fixed station, Node B, eNode B, gNode B, ng-eNB, advanced base station (ABS), or access point. can be replaced by

In addition, in the embodiments of the present disclosure, a terminal includes a user equipment (UE), a mobile station (MS), a subscriber station (SS), a mobile subscriber station (MSS), It may be replaced with terms such as mobile terminal or advanced mobile station (AMS).

In addition, the transmitting end refers to a fixed and/or mobile node providing data service or voice service, and the receiving end refers to a fixed and/or mobile node receiving data service or voice service. Therefore, in the case of uplink, the mobile station can be a transmitter and the base station can be a receiver. Similarly, in the case of downlink, the mobile station may be a receiving end and the base station may be a transmitting end.

Embodiments of the present disclosure are wireless access systems, such as an IEEE 802.xx system, a 3rd Generation Partnership Project (3GPP) system, a 3GPP Long Term Evolution (LTE) system, a 3GPP 5G (5th generation) NR (New Radio) system, and a 3GPP2 system. It may be supported by at least one disclosed standard document, and in particular, the embodiments of the present disclosure are supported by 3GPP technical specification (TS) 38.211, 3GPP TS 38.212, 3GPP TS 38.213, 3GPP TS 38.321 and 3GPP TS 38.331 documents It can be.

In addition, embodiments of the present disclosure may be applied to other wireless access systems, and are not limited to the above-described systems. For example, it may also be applicable to a system applied after the 3GPP 5G NR system, and is not limited to a specific system.

That is, obvious steps or parts not described in the embodiments of the present disclosure may be described with reference to the above documents. In addition, all terms disclosed in this document can be explained by the standard document.

Hereinafter, preferred embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. The detailed description set forth below in conjunction with the accompanying drawings is intended to describe exemplary embodiments of the present disclosure, and is not intended to represent the only embodiments in which the technical configurations of the present disclosure may be practiced.

In addition, specific terms used in the embodiments of the present disclosure are provided to aid understanding of the present disclosure, and the use of these specific terms may be changed in other forms without departing from the technical spirit of the present disclosure.

The following technologies include code division multiple access (CDMA), frequency division multiple access (FDMA), time division multiple access (TDMA), orthogonal frequency division multiple access (OFDMA), single carrier frequency division multiple access (SC-FDMA), and the like. It can be applied to various wireless access systems.

In the following, in order to clarify the following description, the description is based on the 3GPP communication system (e.g. (eg, LTE, NR, etc.), but the technical spirit of the present invention is not limited thereto. LTE is 3GPP TS 36.xxx Release 8 or later In detail, LTE technology after 3GPP TS 36.xxx Release 10 is referred to as LTE-A, and LTE technology after 3GPP TS 36.xxx Release 13 may be referred to as LTE-A pro. 3GPP NR may mean technology after TS 38.xxx Release 15. 3GPP 6G may mean technology after TS Release 17 and/or Release 18. "xxx" means a standard document detail number. LTE/NR/6G may be collectively referred to as a 3GPP system.

For background art, terms, abbreviations, etc. used in the present disclosure, reference may be made to matters described in standard documents published prior to the present invention. As an example, 36.xxx and 38.xxx standard documents may be referred to.

본 개시에 적용 가능한 통신 시스템Communication systems applicable to the present disclosure

Although not limited thereto, various descriptions, functions, procedures, proposals, methods and / or operational flowcharts of the present disclosure disclosed in this document may be applied to various fields requiring wireless communication / connection (eg, 5G) between devices. there is.

Hereinafter, it will be exemplified in more detail with reference to the drawings. In the following drawings/description, the same reference numerals may represent the same or corresponding hardware blocks, software blocks or functional blocks unless otherwise specified.

1 is a diagram illustrating an example of a communication system applied to the present disclosure.

Referring to FIG. 1 , a communication system 100 applied to the present disclosure includes a wireless device, a base station, and a network. Here, the wireless device means a device that performs communication using a radio access technology (eg, 5G NR, LTE), and may be referred to as a communication/wireless/5G device. Although not limited thereto, the wireless device includes a robot 100a, a vehicle 100b-1 and 100b-2, an extended reality (XR) device 100c, a hand-held device 100d, and a home appliance. appliance) 100e, Internet of Thing (IoT) device 100f, and artificial intelligence (AI) device/server 100g. For example, the vehicle may include a vehicle equipped with a wireless communication function, an autonomous vehicle, a vehicle capable of performing inter-vehicle communication, and the like. Here, the vehicles 100b-1 and 100b-2 may include an unmanned aerial vehicle (UAV) (eg, a drone). The XR device 100c includes augmented reality (AR)/virtual reality (VR)/mixed reality (MR) devices, and includes a head-mounted device (HMD), a head-up display (HUD) installed in a vehicle, a television, It may be implemented in the form of smart phones, computers, wearable devices, home appliances, digital signage, vehicles, robots, and the like. The mobile device 100d may include a smart phone, a smart pad, a wearable device (eg, a smart watch, a smart glass), a computer (eg, a laptop computer), and the like. The home appliance 100e may include a TV, a refrigerator, a washing machine, and the like. The IoT device 100f may include a sensor, a smart meter, and the like. For example, the base station 120 and the network 130 may also be implemented as a wireless device, and a specific wireless device 120a may operate as a base station/network node to other wireless devices.

The wireless devices 100a to 100f may be connected to the network 130 through the base station 120 . AI technology may be applied to the wireless devices 100a to 100f, and the wireless devices 100a to 100f may be connected to the AI server 100g through the network 130. The network 130 may be configured using a 3G network, a 4G (eg LTE) network, or a 5G (eg NR) network. The wireless devices 100a to 100f may communicate with each other through the base station 120/network 130, but communicate directly without going through the base station 120/network 130 (e.g., sidelink communication). You may. For example, the vehicles 100b-1 and 100b-2 may perform direct communication (eg, vehicle to vehicle (V2V)/vehicle to everything (V2X) communication). In addition, the IoT device 100f (eg, sensor) may directly communicate with other IoT devices (eg, sensor) or other wireless devices 100a to 100f.

2 is a diagram illustrating an example of a wireless device applicable to the present disclosure.

Referring to FIG. 2 , a first wireless device 200a and a second wireless device 200b may transmit and receive radio signals through various wireless access technologies (eg, LTE and NR). Here, {the first wireless device 200a, the second wireless device 200b} denotes the {wireless device 100x and the base station 120} of FIG. 1 and/or the {wireless device 100x and the wireless device 100x. } can correspond.

The first wireless device 200a includes one or more processors 202a and one or more memories 204a, and may further include one or more transceivers 206a and/or one or more antennas 208a. The processor 202a controls the memory 204a and/or the transceiver 206a and may be configured to implement the descriptions, functions, procedures, suggestions, methods and/or operational flow diagrams disclosed herein. For example, the processor 202a may process information in the memory 204a to generate first information/signal, and transmit a radio signal including the first information/signal through the transceiver 206a. In addition, the processor 202a may receive a radio signal including the second information/signal through the transceiver 206a and store information obtained from signal processing of the second information/signal in the memory 204a. The memory 204a may be connected to the processor 202a and may store various information related to the operation of the processor 202a. For example, memory 204a may perform some or all of the processes controlled by processor 202a, or instructions for performing the descriptions, functions, procedures, suggestions, methods, and/or flowcharts of operations disclosed herein. It may store software codes including them. Here, the processor 202a and the memory 204a may be part of a communication modem/circuit/chip designed to implement a wireless communication technology (eg, LTE, NR). The transceiver 206a may be coupled to the processor 202a and may transmit and/or receive wireless signals through one or more antennas 208a. The transceiver 206a may include a transmitter and/or a receiver. The transceiver 206a may be used interchangeably with a radio frequency (RF) unit. In the present disclosure, a wireless device may mean a communication modem/circuit/chip.

The second wireless device 200b includes one or more processors 202b, one or more memories 204b, and may further include one or more transceivers 206b and/or one or more antennas 208b. The processor 202b controls the memory 204b and/or the transceiver 206b and may be configured to implement the descriptions, functions, procedures, suggestions, methods and/or operational flow diagrams disclosed herein. For example, the processor 202b may process information in the memory 204b to generate third information/signal, and transmit a radio signal including the third information/signal through the transceiver 206b. In addition, the processor 202b may receive a radio signal including the fourth information/signal through the transceiver 206b and store information obtained from signal processing of the fourth information/signal in the memory 204b. The memory 204b may be connected to the processor 202b and may store various information related to the operation of the processor 202b. For example, memory 204b may perform some or all of the processes controlled by processor 202b, or instructions for performing the descriptions, functions, procedures, suggestions, methods, and/or flowcharts of operations disclosed herein. It may store software codes including them. Here, the processor 202b and the memory 204b may be part of a communication modem/circuit/chip designed to implement a wireless communication technology (eg, LTE, NR). The transceiver 206b may be coupled to the processor 202b and may transmit and/or receive wireless signals through one or more antennas 208b. The transceiver 206b may include a transmitter and/or a receiver. The transceiver 206b may be used interchangeably with an RF unit. In the present disclosure, a wireless device may mean a communication modem/circuit/chip.

Hereinafter, hardware elements of the

wireless devices

200a and 200b will be described in more detail. Although not limited to this, one or more protocol layers may be implemented by one or

more processors

202a, 202b. For example, the one or

more processors

202a and 202b may include one or more layers (eg, PHY (physical), MAC (media access control), RLC (radio link control), PDCP (packet data convergence protocol), RRC (radio resource) control) and functional layers such as service data adaptation protocol (SDAP). One or

more processors

202a, 202b may generate one or more protocol data units (PDUs) and/or one or more service data units (SDUs) according to the descriptions, functions, procedures, proposals, methods, and/or operational flow charts disclosed herein. can create One or

more processors

202a, 202b may generate messages, control information, data or information according to the descriptions, functions, procedures, proposals, methods and/or operational flow diagrams disclosed herein. One or

more processors

202a, 202b generate PDUs, SDUs, messages, control information, data or signals (eg, baseband signals) containing information according to the functions, procedures, proposals and/or methods disclosed herein , may be provided to one or

more transceivers

206a and 206b. One or

more processors

202a, 202b may receive signals (eg, baseband signals) from one or

more transceivers

206a, 206b, and descriptions, functions, procedures, proposals, methods, and/or flowcharts of operations disclosed herein PDUs, SDUs, messages, control information, data or information can be obtained according to these.

One or

more processors

202a, 202b may be referred to as a controller, microcontroller, microprocessor or microcomputer. One or

more processors

202a, 202b may be implemented by hardware, firmware, software, or a combination thereof. For example, one or more application specific integrated circuits (ASICs), one or more digital signal processors (DSPs), one or more digital signal processing devices (DSPDs), one or more programmable logic devices (PLDs), or one or more field programmable gate arrays (FPGAs). may be included in one or

more processors

202a and 202b. The descriptions, functions, procedures, proposals, methods and/or operational flowcharts disclosed in this document may be implemented using firmware or software, and the firmware or software may be implemented to include modules, procedures, functions, and the like. Firmware or software configured to perform the descriptions, functions, procedures, proposals, methods and/or operational flow charts disclosed in this document may be included in one or

more processors

202a or 202b or stored in one or

more memories

204a or 204b. It can be driven by the

above processors

202a and 202b. The descriptions, functions, procedures, suggestions, methods and/or operational flow charts disclosed in this document may be implemented using firmware or software in the form of codes, instructions and/or sets of instructions.

One or

more memories

204a, 204b may be coupled to one or

more processors

202a, 202b and may store various types of data, signals, messages, information, programs, codes, instructions and/or instructions. One or

more memories

204a, 204b may include read only memory (ROM), random access memory (RAM), erasable programmable read only memory (EPROM), flash memory, hard drive, registers, cache memory, computer readable storage media, and/or It may consist of a combination of these. One or

more memories

204a, 204b may be located internally and/or externally to one or

more processors

202a, 202b. In addition, one or

more memories

204a, 204b may be connected to one or

more processors

202a, 202b through various technologies such as wired or wireless connections.

One or

more transceivers

206a, 206b may transmit user data, control information, radio signals/channels, etc. referred to in the methods and/or operational flow charts of this document to one or more other devices. One or

more transceivers

206a, 206b may receive user data, control information, radio signals/channels, etc. referred to in descriptions, functions, procedures, proposals, methods and/or operational flow charts, etc. disclosed herein from one or more other devices. there is. For example, one or

more transceivers

206a and 206b may be connected to one or

more processors

202a and 202b and transmit and receive radio signals. For example, one or

more processors

202a, 202b may control one or

more transceivers

206a, 206b to transmit user data, control information, or radio signals to one or more other devices. In addition, one or

more processors

202a, 202b may control one or

more transceivers

206a, 206b to receive user data, control information, or radio signals from one or more other devices. In addition, one or

more transceivers

206a, 206b may be coupled to one or

more antennas

208a, 208b, and one or

more transceivers

206a, 206b may be connected to one or

more antennas

208a, 208b to achieve the descriptions, functions disclosed in this document. , procedures, proposals, methods and / or operation flowcharts, etc. can be set to transmit and receive user data, control information, radio signals / channels, etc. In this document, one or more antennas may be a plurality of physical antennas or a plurality of logical antennas (eg, antenna ports). One or more transceivers (206a, 206b) in order to process the received user data, control information, radio signal / channel, etc. using one or more processors (202a, 202b), the received radio signal / channel, etc. in the RF band signal It can be converted into a baseband signal. One or

more transceivers

206a and 206b may convert user data, control information, and radio signals/channels processed by one or

more processors

202a and 202b from baseband signals to RF band signals. To this end, one or

more transceivers

206a, 206b may include (analog) oscillators and/or filters.

본 개시에 적용 가능한 무선 기기 구조Wireless device structure applicable to the present disclosure

3 is a diagram illustrating another example of a wireless device applied to the present disclosure.

Referring to FIG. 3, a wireless device 300 corresponds to the

wireless devices

200a and 200b of FIG. 2, and includes various elements, components, units/units, and/or modules. ) can be configured. For example, the wireless device 300 may include a communication unit 310, a control unit 320, a memory unit 330, and an additional element 340. The communication unit may include communication circuitry 312 and transceiver(s) 314 . For example, communication circuitry 312 may include one or

more processors

202a, 202b of FIG. 2 and/or one or

more memories

204a, 204b. For example, transceiver(s) 314 may include one or

more transceivers

206a, 206b of FIG. 2 and/or one or

more antennas

208a, 208b. The control unit 320 is electrically connected to the communication unit 310, the memory unit 330, and the additional element 340 and controls overall operations of the wireless device. For example, the control unit 320 may control electrical/mechanical operations of the wireless device based on programs/codes/commands/information stored in the memory unit 330. In addition, the control unit 320 transmits the information stored in the memory unit 330 to the outside (eg, another communication device) through the communication unit 310 through a wireless/wired interface, or transmits the information stored in the memory unit 330 to the outside (eg, another communication device) through the communication unit 310. Information received through a wireless/wired interface from other communication devices) may be stored in the memory unit 330 .

The additional element 340 may be configured in various ways according to the type of wireless device. For example, the additional element 340 may include at least one of a power unit/battery, an input/output unit, a driving unit, and a computing unit. Although not limited thereto, the wireless device 300 may be a robot (FIG. 1, 100a), a vehicle (FIG. 1, 100b-1, 100b-2), an XR device (FIG. 1, 100c), a mobile device (FIG. 1, 100d) ), home appliances (FIG. 1, 100e), IoT devices (FIG. 1, 100f), digital broadcasting terminals, hologram devices, public safety devices, MTC devices, medical devices, fintech devices (or financial devices), security devices, climate/ It may be implemented in the form of an environment device, an AI server/device (FIG. 1, 140), a base station (FIG. 1, 120), a network node, and the like. Wireless devices can be mobile or used in a fixed location depending on the use-case/service.

In FIG. 3 , various elements, components, units/units, and/or modules in the wireless device 300 may be entirely interconnected through a wired interface or at least partially connected wirelessly through the communication unit 310 . For example, in the wireless device 300, the control unit 320 and the communication unit 310 are connected by wire, and the control unit 320 and the first units (eg, 130 and 140) are connected wirelessly through the communication unit 310. can be connected Additionally, each element, component, unit/unit, and/or module within wireless device 300 may further include one or more elements. For example, the control unit 320 may be composed of one or more processor sets. For example, the control unit 320 may include a set of a communication control processor, an application processor, an electronic control unit (ECU), a graphic processing processor, a memory control processor, and the like. As another example, the memory unit 330 may include RAM, dynamic RAM (DRAM), ROM, flash memory, volatile memory, non-volatile memory, and/or combinations thereof. can be configured.

4 is a diagram illustrating an example of an AI device applied to the present disclosure. As an example, AI devices include TVs, projectors, smartphones, PCs, laptops, digital broadcasting terminals, tablet PCs, wearable devices, set-top boxes (STBs), radios, washing machines, refrigerators, digital signage, robots, vehicles, etc. It may be implemented as a device or a movable device.

Referring to FIG. 4, the AI device 600 includes a communication unit 610, a control unit 620, a memory unit 630, an input/output unit 640a/640b, a running processor unit 640c, and a sensor unit 640d. can include Blocks 910 to 930/940a to 940d may respectively correspond to blocks 310 to 330/340 of FIG. 3 .

The communication unit 610 communicates wired and wireless signals (eg, sensor information, user data) with external devices such as other AI devices (eg, FIG. 1, 100x, 120, and 140) or AI servers (Fig. input, learning model, control signal, etc.) can be transmitted and received. To this end, the communication unit 610 may transmit information in the memory unit 630 to an external device or transmit a signal received from the external device to the memory unit 630 .

The controller 620 may determine at least one executable operation of the AI device 600 based on information determined or generated using a data analysis algorithm or a machine learning algorithm. And, the controller 620 may perform the determined operation by controlling components of the AI device 600 . For example, the control unit 620 may request, retrieve, receive, or utilize data from the learning processor unit 640c or the memory unit 630, and may perform a predicted operation among at least one feasible operation or one determined to be desirable. Components of the AI device 600 may be controlled to execute an operation. In addition, the control unit 920 collects history information including user feedback on the operation contents or operation of the AI device 600 and stores it in the memory unit 630 or the running processor unit 640c, or the AI server ( 1, 140) can be transmitted to an external device. The collected history information can be used to update the learning model.

The memory unit 630 may store data supporting various functions of the AI device 600 . For example, the memory unit 630 may store data obtained from the input unit 640a, data obtained from the communication unit 610, output data of the learning processor unit 640c, and data obtained from the sensing unit 640. Also, the memory unit 930 may store control information and/or software codes required for operation/execution of the control unit 620 .

The input unit 640a may obtain various types of data from the outside of the AI device 600. For example, the input unit 620 may obtain learning data for model learning and input data to which the learning model is to be applied. The input unit 640a may include a camera, a microphone, and/or a user input unit. The output unit 640b may generate an output related to sight, hearing, or touch. The output unit 640b may include a display unit, a speaker, and/or a haptic module. The sensing unit 640 may obtain at least one of internal information of the AI device 600, surrounding environment information of the AI device 600, and user information by using various sensors. The sensing unit 640 may include a proximity sensor, an illuminance sensor, an acceleration sensor, a magnetic sensor, a gyro sensor, an inertial sensor, an RGB sensor, an IR sensor, a fingerprint recognition sensor, an ultrasonic sensor, an optical sensor, a microphone, and/or a radar. there is.

The learning processor unit 640c may learn a model composed of an artificial neural network using learning data. The running processor unit 640c may perform AI processing together with the running processor unit of the AI server (FIG. 1, 140). The learning processor unit 640c may process information received from an external device through the communication unit 610 and/or information stored in the memory unit 630 . In addition, the output value of the learning processor unit 940c may be transmitted to an external device through the communication unit 610 and/or stored in the memory unit 630.

6G 통신 시스템 6G communication system

6G (radio communications) systems are characterized by (i) very high data rates per device, (ii) very large number of connected devices, (iii) global connectivity, (iv) very low latency, (v) battery- It aims to lower energy consumption of battery-free IoT devices, (vi) ultra-reliable connectivity, and (vii) connected intelligence with machine learning capabilities. The vision of the 6G system can be four aspects such as “intelligent connectivity”, “deep connectivity”, “holographic connectivity”, and “ubiquitous connectivity”, and the 6G system can satisfy the requirements shown in Table 1 below. That is, Table 1 is a table showing the requirements of the 6G system.

At this time, the 6G system is enhanced mobile broadband (eMBB), ultra-reliable low latency communications (URLLC), mMTC (massive machine type communications), AI integrated communication, tactile Internet (tactile internet), high throughput, high network capacity, high energy efficiency, low backhaul and access network congestion and improved data security ( can have key factors such as enhanced data security.

인공 지능(artificial Intelligence, AI)Artificial Intelligence (AI)

The most important and newly introduced technology for the 6G system is AI. AI was not involved in the 4G system. 5G systems will support partial or very limited AI. However, the 6G system will be AI-enabled for full automation. Advances in machine learning will create more intelligent networks for real-time communication in 6G. Introducing AI in communications can simplify and enhance real-time data transmission. AI can use a plethora of analytics to determine how complex target tasks are performed. In other words, AI can increase efficiency and reduce processing delays.

Time-consuming tasks such as handover, network selection, and resource scheduling can be performed instantly by using AI. AI can also play an important role in machine-to-machine, machine-to-human and human-to-machine communications. In addition, AI can be a rapid communication in the brain computer interface (BCI). AI-based communication systems can be supported by metamaterials, intelligent structures, intelligent networks, intelligent devices, intelligent cognitive radios, self-sustaining wireless networks, and machine learning.

Recently, there have been attempts to integrate AI with wireless communication systems, but these are focused on the application layer, network layer, and especially deep learning, wireless resource management and allocation. come. However, such research is gradually developing into the MAC layer and the physical layer, and in particular, attempts to combine deep learning with wireless transmission are appearing in the physical layer. AI-based physical layer transmission means applying a signal processing and communication mechanism based on an AI driver rather than a traditional communication framework in fundamental signal processing and communication mechanisms. For example, deep learning-based channel coding and decoding, deep learning-based signal estimation and detection, deep learning-based multiple input multiple output (MIMO) mechanism, It may include AI-based resource scheduling and allocation.

In addition, machine learning may be used for channel estimation and channel tracking, and may be used for power allocation, interference cancellation, and the like in a downlink (DL) physical layer. Machine learning can also be used for antenna selection, power control, symbol detection, and the like in a MIMO system.

However, the application of DNN for transmission in the physical layer may have the following problems.

AI algorithms based on deep learning require a lot of training data to optimize training parameters. However, due to limitations in acquiring data in a specific channel environment as training data, a lot of training data is used offline. This is because static training on training data in a specific channel environment may cause a contradiction between dynamic characteristics and diversity of a radio channel.

In addition, current deep learning mainly targets real signals. However, the signals of the physical layer of wireless communication are complex signals. In order to match the characteristics of a wireless communication signal, further research is needed on a neural network that detects a complex domain signal.

Hereinafter, machine learning will be described in more detail.

Machine learning refers to a set of actions that train a machine to create a machine that can do tasks that humans can or cannot do. Machine learning requires data and a running model. In machine learning, data learning methods can be largely classified into three types: supervised learning, unsupervised learning, and reinforcement learning.

Neural network training is aimed at minimizing errors in the output. Neural network learning repeatedly inputs training data to the neural network, calculates the output of the neural network for the training data and the error of the target, and backpropagates the error of the neural network from the output layer of the neural network to the input layer in a direction to reduce the error. ) to update the weight of each node in the neural network.

Supervised learning uses training data in which correct answers are labeled in the learning data, and unsupervised learning may not have correct answers labeled in the learning data. That is, for example, learning data in the case of supervised learning related to data classification may be data in which each learning data is labeled with a category. Labeled training data is input to the neural network, and an error may be calculated by comparing the output (category) of the neural network and the label of the training data. The calculated error is back-propagated in a reverse direction (ie, from the output layer to the input layer) in the neural network, and the connection weight of each node of each layer of the neural network may be updated according to the back-propagation. The amount of change in the connection weight of each updated node may be determined according to a learning rate. The neural network's computation of input data and backpropagation of errors can constitute a learning cycle (epoch). The learning rate may be applied differently according to the number of iterations of the learning cycle of the neural network. For example, a high learning rate is used in the early stages of neural network learning to increase efficiency by allowing the neural network to quickly achieve a certain level of performance, and a low learning rate can be used in the late stage to increase accuracy.

The learning method may vary depending on the characteristics of the data. For example, in a case where the purpose of the receiver is to accurately predict data transmitted by the transmitter in a communication system, it is preferable to perform learning using supervised learning rather than unsupervised learning or reinforcement learning.

The learning model corresponds to the human brain, and the most basic linear model can be considered. ) is called

The neural network cord used as a learning method is largely divided into deep neural networks (DNN), convolutional deep neural networks (CNN), and recurrent boltzmann machine (RNN). and this learning model can be applied.

In the following, a method for organically operating an image input device (e.g. camera) and an image output device based on artificial intelligence or connected intelligence based on the above description will be described. For example, an image input device and an image output device can effectively apply an artificial intelligence system through federated learning in various environments, and this will be described below. For example, as a term to be described below, OIS (Optical Image Stabilizer) may mean a video image stabilization device. As an example, the OIS may be a camera shake prevention function, and is not limited to the above-described embodiment. Also, as an example, federated learning may refer to a learning method in which an algorithm is trained on multiple devices or servers having local data samples without exchanging data samples. In this case, the federated learning method can build a common powerful machine learning model without sharing data between multiple devices. Thus, federated learning can address important issues such as data privacy, data security, data access rights, and heterogeneous data access.

Also, as an example, a plurality of devices may learn a local model for a local data sample as an operation for federated learning. After that, each of the plurality of devices may transfer parameters of the local model (e.g., weights and information of the deep neural network) to the base station or the server based on the learned local model. In this case, the base station or the server may update the global model and transmit the updated global model to each of the plurality of devices again. In this case, as an example, an AirComp (Over the Air Computing) method may be a method in which parameters of local models are transmitted to a base station or a server based on the same radio resource. That is, a global model may exist in a base station or a server, and local models may exist in each of a plurality of devices. At this time, a plurality of devices and a base station or server may operate organically by being connected by various communication methods (including wired/wireless) for transmitting the above-described models. Also, as an example, the above method may be equally applied to an image input device and an image output device, which will be described below.

As an example, FIG. 5 is a diagram illustrating associative learning according to an embodiment of the present disclosure. Referring to FIG. 5, a method for efficient distributed artificial intelligence learning in a mobile communication system can be provided. When data for a plurality of terminals exist in a distributed manner, the centralized learning method may be a method in which terminals transfer each data to a base station and learning is performed at the base station. However, centralized learning that transmits data of terminals to a base station or server may have limitations in data security. Therefore, a wireless federated learning method may be required as a distributed learning method that does not send user data. In this case, the wireless joint learning method may be a method in which each terminal transmits a local model update to the base station while performing individual learning instead of transmitting each data of the terminals to the base station. In this case, the base station may transmit a combined value of the local model updates to each terminal based on the local model update received from the plurality of terminals. At this time, the above-described procedure may be continuously repeated, and distributed learning may be performed by association of terminals.

Here, since the size of the local model is often large, when terminals participating in learning transmit information on the local model through an uplink channel using independent radio resources, radio resource loss may increase. Therefore, a radio calculation (Aircomp) scheme in which terminals send local instantaneous models through uplink using the same radio resource and automatically merge them over radio may be used.

For example, the wireless calculation (Aircomp) method may be a method of applying and transmitting a weight proportional to an inverse of a radio channel in order to combine local models sent by respective terminals into the same size.

As a specific example, model parameters of federated learning may be applied to a new communication system. Federated learning may be applied to any one of cases of protecting individual privacy, reducing load of a base station through distributed processing, and reducing traffic between a base station and a terminal. However, it may not be limited thereto. At this time, as an example, the traffic of local model parameters (e.g., weights and information of deep neural networks) can impose a heavy burden in a wireless communication environment. Air Computing)) can reduce traffic.

However, a wireless communication environment in a communication system may vary. In addition, the number of terminals requiring learning in the communication system may be set in various ways. Here, the communication system may require a flexible operating method and system, rather than a specific fixed technology, in consideration of the above-mentioned environment. Through this, resource efficiency of the communication system can be increased. For example, a federated learning method through Aircomp may be a method of combining terminal model parameters. When transmission is performed based on the air comp method, since signal transmission is performed based on the superposition property of a wireless communication channel, transmission efficiency can be increased and the load of a base station can be reduced. Also, terminals can share the same communication channel. Therefore, when there are a large number of terminals, transmission efficiency can be increased.

Considering the above points, the federated learning method through terminal model parameter compression may be a method in which each terminal performs compression on data in consideration of characteristics of parameters and transmits the compressed data to the base station. Therefore, when the base station receives a signal based on the joint learning method, the base station needs to decompress based on the received signal and perform an operation of summing the collected parameters, and the load of the base station may increase . Also, for example, since a communication channel needs to be allocated for each number of terminals, communication traffic may increase in proportion to the number of terminals in use. Therefore, when there are a large number of terminals, the method through compression may reduce efficiency.

For example, when a weight signaling method between a terminal and a base station is fixedly used in a federated learning scheme, efficiency may vary based on a wireless environment. For example, efficiency may be high in a specific environment, but in the opposite case, efficiency may be deteriorated. Since the wireless environment can change flexibly, there is a need to recognize the dynamically changing wireless environment and to select a technology based on the recognized wireless environment. In the following, an operation based on the foregoing will be described in order to increase the efficiency of a wireless environment.

For example, each terminal may transmit parameters (eg, weights and information of a deep neural network) of a model learned based on a federated learning method to the base station. Each terminal transmits the compressed parameters, and the base station may update the global model based on Equation 1 below. Here, c may be information compression and modulation processing, and d may be demodulation and information restoration processing. After that, the base station can deliver the updated global model to each terminal.

[Equation 1]

More specifically, each terminal may perform compression based on a method of minimizing the amount of model parameters. For example, compression may be performed based on at least one of weight pruning, quantization, and weight sharing. Also, as an example, compression may be performed based on another method, and is not limited to the above-described embodiment. Here, when compression is performed based on the existing neural network, values necessary for actual inference among weights may have resistance to small values. That is, the weight value required for actual inference may have a small effect on small values. Considering the above points, weight pruning may set all small weight values to 0. Through this, the neural network can reduce the size of the network model. Also, as an example, quantization may be a method of calculating by reducing data to a specific number of bits. That is, data can be expressed only with specific quantized values. Also, as an example, weight sharing may be a method of adjusting weight values based on approximate values (e.g. codebooks) and sharing them. Here, when a signal is transmitted in a network, only codebooks and indexes for corresponding information may be shared.

Based on any one of the above methods, each terminal may perform compression on data and transmit compressed information to the base station. At this time, the base station is compressed

may be received from each terminal, and the parameters of the global model may be calculated and updated by decompressing the received information.

Here, each terminal may set local model parameters having individual characteristics. Therefore, when each terminal performs compression, compression efficiency may be different for each terminal. Also, as an example, each terminal may have different hardware resources. Here, compression efficiency may be affected by hardware resources. Therefore, compression efficiency may be different for each terminal.

As a specific example, when a terminal performs 8-bit quantization, a terminal equipped with a 64-bit arithmetic processing function can obtain high compression efficiency. On the other hand, a terminal equipped with a 16-bit arithmetic processing function may have low compression efficiency. Also, for example, when the terminal has low-end hardware, the terminal may receive a large compression load. Therefore, it may be advantageous for the terminal described above to use a simple compression method. For example, since Internet of Thing (IoT) terminals or low-power terminals may have relatively low-end hardware, a simple compression technique may be used. On the other hand, since a terminal operating based on AI or a terminal processing large amounts of data may have high-end hardware, compression efficiency may be increased by using a complex compression technique. That is, different compression methods may be used for each terminal, and it may be necessary to use a compression method suitable for each terminal.

Considering the above points, each terminal may use a compression method suitable for individual characteristics of local model parameters and hardware resources. At this time, the terminals need to transmit information about the compression method to the base station. The base station may restore compressed data and model parameters received from each terminal based on information received from the terminal.

As an example, as a case where the base station and the terminal communicate through a reflector, a case in which the above-described aircom-type combined learning is performed may be considered. When air comp is applied in a smart communication environment in which a plurality of reflectors exist, optimization may be complicated because all signals of a plurality of terminals are transmitted to the reflector. Thus, signaling processing may become difficult. In addition, a method for efficiently performing Aircom combined learning through a reflector based on an efficient protocol and optimization method may be required, and a method for this will be described below.

For example, referring to FIG. 5 , when a plurality of

terminals

520, 530, and 540 transmit local model update information to a base station 510 in order to efficiently use uplink resources, the

terminals

520 and 530 , 540) may transmit local model update information to the base station 510 through a method in which the global model is calculated over the air using the same radio resource, as described above. As an example, the plurality of

terminals

520, 530, and 540 in FIG. 5 may be vehicles, smart devices, or other devices that perform autonomous driving based on AI, and are not limited to the above-described embodiment.

For example, associative learning can be divided into the following three steps. At this time, the following three steps may be sequentially repeated until the update value converges to a certain value.

Here, in the first step, each of the

terminals

520, 530, and 540 uses their own data to receive a previously received global model (

), update the local model (

) can be created. For example, the initial global model may be an initial untrained neural network, but may not be limited thereto. In this case, an update may be performed based on the local model. As an example, Equation 2 below may be a case where an update is performed based on a stochastic gradient method, but the corresponding method may not be limited.

[Equation 2]

Next, in the second step, each of the

terminals

520, 530, and 540 has an updated local model (

) may be transmitted to the base station 510 through the same radio resource. Here, as an example, the base station 510 may be connected to a server or a cloud server. For example, for convenience of explanation, the base station 510 is described based on a case in which the server serves, but may not be limited thereto. For example, the base station 510 may transmit the global model to the server in order to combine local models received from a plurality of terminals, but is not limited to the above-described embodiment.

Finally, in the third step, based on Equation 3 below, the base station 510 may receive a global model in which local models are combined over the air, and may perform an update. After that, the base station 510 may transmit the global model to the plurality of

terminals

520, 530, and 540 at time k based on the updated information. For example, the above-described step 3 may be repeatedly performed based on the convergence of values of the global model or a given number of times.

[Equation 3]

As described above, data processing may be performed based on associative learning. In this case, as an example, FIG. 6 is a diagram illustrating an image input/output device according to an embodiment of the present disclosure. Referring to FIG. 6 , the video input/output device may acquire video input. At this time, the image input device (e.g. camera) may change the characteristics of the image through the lens. For example, at least one of functions such as zoom, wide angle, auto focus, and OIS can be performed through mechanical movement of a solid lens (e.g. glass, plastic), and the image input device As shown in FIG. 6 , image characteristics may be changed based on motions of the 3-group lenses operating in the telephoto and wide-angle directions. As a specific example, the group 3 zoom lens of FIG. 6 may be a multi-zoom group lens system. At this time, if the first lens (L_1) and the second lens (L_2) are close and the distance between the third lens (L_3) is far, the focal length of the optical system can be shortened (wide-angle). If the lens is far away and the second and third lenses are close together, the focal length of the optical system can be long (telephoto). Therefore, when changing the position between the first lens (L_1) and the third lens (L_3), the focal length is proportional to the position. can be continuously changed. As an example, FIG. 6 may display a wide-angle view and a telephoto view sequentially from the top. Also, as an example, FIG. 7 is a diagram illustrating an OIS principle according to an embodiment of the present disclosure. Referring to FIG. 7 , a tilt phenomenon of a camera caused by hand shake can be compensated for through a mechanical movement of a lens, and based on the foregoing description, the image input device can change the characteristics of an image.

Also, as an example, FIG. 8 is a diagram illustrating a metalens structure according to an embodiment of the present disclosure. For example, the lenses may be made of glass or a material similar to glass. At this time, the lenses can be configured based on the above-described material because a more accurate and clear image can be obtained only when the glass is accurately polished to a curved surface and the light is clearly collected. However, the cost of polishing the curved surface may be high, and since the characteristics of the curved surface are fixed during lens manufacturing, many restrictions may exist in space. At this time, as an example, the meta lens may be a technology for implementing characteristics similar to those of a glass lens using a meta surface. At this time, the focal length of the image may be variously adjusted through the meta lens. As an example, referring to FIG. 8(a), the metalens makes the surface of a material into an artificial small and precisely patterned structure, and adjusts the refractive index (permittivity and permeability equation) of each element to focus. can change its distance or location. As an example, FIG. 8(a) may be a fixed structure metalens implemented with PGMS (Phase Gradient MetaSurface), but the metalens structure may not be limited thereto. For example, the metalens can control the directionality of light by changing a spherical wave into a plane wave.

More specifically, referring to FIG. 8( b ), a spherical wave may be changed into a plane wave through a metal lens to give directionality of light. At this time, referring to FIG. 8(b), the wavefront transmitted by combining the wavefront of the spherical wave and the phase shift of the metalens may have a shape of a plane wave, but may not be limited thereto.

That is, the meta lens can change the focal length and other image characteristics by controlling the parameters of each element differently from conventional fixed lenses, which will be described below.

For example, in the following, a method of applying a metalens to an imaging device having fixed characteristics (e.g. size, shape, use, angle of view) and an output device capable of changing characteristics through mechanical lens adjustment will be described. At this time, as an example, the metalens may give variable characteristics to the image input/output device, and input/output may be controlled integrally or organically through a controller. Also, as an example, an imaging device whose characteristics are variable as described above may be controlled based on the location of a user terminal, and artificial intelligence learning (eg, federated learning) may be performed using information about this.

That is, each terminal communicates with the base station or the server to change the characteristics of the imaging apparatus through associative learning based on location information of the terminals, thereby controlling the imaging apparatus in consideration of the environment. Here, each terminal may be a device used by a user. For example, the location information of the user may be recognized as the location information of the terminal, and based on this, the metalens artificial system may be controlled. In addition, as an example, the server may be a device that operates in connection with a base station or access point (AP), receives location-based parameter information from a terminal, and updates a global model through federated learning based thereon. can be That is, a device that updates a global model in consideration of federated learning may refer to a device that communicates with a terminal such as a server, base station, or access point, and may not be limited to a specific form. However, in the following description, the base station is mainly described for convenience of description.

As an example, FIG. 9 is a diagram illustrating an artificial intelligence system for controlling a metalens for user-based image input/output according to an embodiment of the present disclosure. Referring to FIG. 9 , the location and distance of the user 930 may be recognized through a Time of Flight (TOF) camera. At this time, since the TOF camera has a small angle of view, there may be limitations in detecting all areas. Accordingly, a wide-angle effect may be obtained by sequentially adjusting the direction of the angle of view (focus direction) through the metal lens 940 . At this time, the metalens controller 920 sequentially selects a direction vector to obtain information about the user 930, and the Agent (artificial intelligence, 910) recognizes the direction or distance of the user 830 through the direction vector and the TOF image. can do. For example, the artificial intelligence 910 may be implemented in a base station or may be a device connected to the base station or a cloud, and may not be limited to a specific form. At this time, the artificial intelligence 910 may transfer a value for controlling the metalens of the image output device to the metalens controller 920 based on the user's perceived image. Here, the control value may be implemented in the form of a codebook having a focal length, a focal direction, and a radius of curvature according to the location of the user, and may not be limited to a specific form.

As another example, the artificial intelligence 910 or the base station may operate by receiving location information from a terminal equipped by the user 930 instead of a TOF camera, and may not be limited to a specific form.

When the metalens are controlled based on the above, the phase of each element of the metalens may be adjusted. More specifically, FIG. 10 is a diagram illustrating a method of controlling a metalens according to an embodiment of the present disclosure. Referring to FIG. 10 (a), the meta lens can adjust the steering of the light source. At this time, by adjusting the phase of each meta-atom (or element) of the meta-lens, the spherical wave can be adjusted so that the wavefront has steering properties. For example, referring to FIG. 10 (a), as the phase change of the metalens is biased to the right, the passing wavefront may be tilted to the right, and may be controlled through the above-described method. In addition, FIG. 10 (b) is a diagram showing how the meta lens adjusts the curvature of an image. Referring to FIG. 10(b), by adjusting the curvature of an image, the user can have a three-dimensional effect like a curved monitor. For example, in order to have a curved monitor shape of an image through a metal lens, it is necessary to set the distance to the image and the position of the user's eyes the same by enlarging the image at the edge of the field of view as well as the direction of light.

As a specific example, FIGS. 11 and 12 are diagrams illustrating comparison of viewing distances between curved and flat image outputs according to an embodiment of the present disclosure. At this time, referring to FIGS. 11 and 12, since the enlargement or reduction area of the metalens varies for each position of the screen, not one but a plurality of overlapping may be required, and it may be controlled based on the curved radius of curvature. For example, as the radius of curvature increases, the amount of bending may decrease. As a more specific example, the radius of curvature 1800R may indicate a degree of curvature having a radius of 180 cm, but this is only one example and is not limited to the above-described embodiment.

13 is a diagram illustrating a metalens control method according to an embodiment of the present disclosure. Referring to FIG. 13 , the metalens artificial intelligence system 1310 may include a user recognition unit 1311 and a metalens characteristic prediction unit 1312. In addition, the metalens artificial intelligence system 1310 may further include other components, and is not limited to the above-described embodiment. At this time, the metalens artificial intelligence system 1310 may recognize a user through an image from a camera based on a currently set metalens characteristic value. For example, the user recognition unit 1311 of the metalens artificial intelligence system 1310 may extract user-related information and transmit it to the metalens characteristic prediction unit 1312 . In this case, the user-related information may include at least one of user direction information, distance information, and state information, and is not limited to a specific information type.

In addition, as an example, the metalens artificial intelligence system 1310 may further obtain user-related additional information from a terminal equipped by the user and transmit it to the metalens controller 1320. For example, the terminal may be a terminal used by the user after completing authentication, and may be possessed by the user. As another example, the terminal may be a device (e.g. remote controller) related to the video input/output device, and may not be limited to a specific form. Here, the terminal may transmit location information of the terminal and other user-related additional information to the artificial intelligence system 1310, and may utilize them.

After that, the metalens characteristic prediction unit 1312 may select a metalens control value stored as a codebook based on user recognition information, and transmit the selected control value to the metalens controller 1320 . At this time, the image input device 1321 and the image output device 1322 in the metalens controller 1320 may control the metalens based on the obtained control value corresponding to the codebook.

At this time, the metalens artificial intelligence system 1310 may operate based on communication, and data burden may exist. Considering the above points, as an example, the user recognition unit 1311 of the artificial intelligence system 1310 is implemented in a device (e.g. camera) that acquires an actual image, and the metalens characteristic prediction unit 1312 is a base station or server. It can be implemented in, and learning and prediction can be performed in a distributed manner.

As an example, FIG. 14 is a diagram illustrating the operation of a distributed learning artificial intelligence system according to an embodiment of the present disclosure. Referring to FIG. 14 , a base station (or server) 1410, an image input device 1420, and an image output device 1430 may exchange data through mutual communication. For example, as an image input device 1420, the metal lens controller of the camera can recognize user-related information through the input image by changing the steering of the metal lens. For example, the camera may control the received image by changing the steering of the meta lens like the lens tilt function of the OIS, but may not be limited thereto. In this case, the user-related information may include at least one of the user's direction, distance, and state (viewing angle) information, and is not limited to the above-described embodiment. Here, as described above, the image input device 1420 includes an artificial intelligence user recognition unit, and based on this, user-related information may be derived. After that, the video input device 1420 may transmit the derived user-related information (or user recognition information) to the base station (or server, 1410). At this time, as an example, the base station (or server, 1410) uses the artificial intelligence The artificial intelligence metalens characteristic prediction unit of the system may be provided, and through this, the metalens characteristic factor (or metalens parameter) may be derived (or predicted).At this time, as an example, the metalens characteristic factor is the user's direction, It can be defined as an index of a codebook having various patterns of metalens according to location information such as distance, etc. As another example, the base station 1410 further obtains location information and user-related additional information from a terminal owned by the user. At this time, the artificial intelligence metalens characteristic predictor may derive a metalens characteristic factor by further using information acquired from the terminal, and may not be limited to a specific form.

At this time, as an example, the characteristic factor of the metalens in the form of an index of the codebook is

,

and

At least one of them can be used. At this time,

and

may be the x-coordinate and y-coordinate of a focal point, respectively. also,

is the focal length,

may be the radius of curvature. Also, as an example, the characteristic factor of the metalens may further include other parameters, and may not be limited to the above parameters. At this time, as an example, the user's direction and distance may be expressed based on the characteristic factor of the metalens, and each of the above-described values may be x, y, l, and c, and the corresponding values may be quantized as an index. At this time, the base station (or server, 1410) may generate a related metalens control value based on the artificial intelligence metalens characteristic prediction unit. At this time, the generated metalens control value may be transmitted to the image input device 1420 and the image output device 1430, respectively. Through this, user-based image input/output can be implemented through the metalens. In addition, as an example, the image input device 1420 transmits related information to the metalens characteristic predictor while recognizing and tracking the user's motion, and continuously sets the metalens control value based on whether the metalens characteristic predictor is the metalens characteristic. can be updated At this time, the image input device 1420 may recognize the entire situation of a preset space while sequentially searching in all directions based on a predetermined period. That is, the image input device 1420 periodically obtains image information, extracts user-related information, transmits it to the base station 1410, and periodically updates control values related to the metalens characteristic factor, thereby performing user perception-based control. can do.

As a specific example, in an artificial intelligence system through control of a video input device (e.g. camera) and a metal lens, a case where the video output device is an indoor TV may be considered. At this time, the image input device may acquire user-related information, as described above. Also, as an example, the TV remote control may be the above-described terminal, and the location of the user may be indirectly measured through the location of the remote control. For example, when using a remote control through Bluetooth, WIFI, or other local area networks, the user's distance and direction are determined by measuring the TV reception power from the remote control or the Arrival of Angle (AoA) of the TV array antenna to determine the distance and direction from the TV to the remote control. It can be measured by recognizing distance and direction. As another example, in the case of a remote control using infrared rays, the user's distance and direction can be detected by performing object detection through WIFI sensing to recognize the relative position of the remote control to the TV, and through this, the user's distance and direction can be recognized.

As another example, in the case of a remote controller having location information in advance, the location information may be transmitted to a TV by applying power or additional signaling, and is not limited to a specific embodiment.

Here, as an example, the user recognition unit may be replaced by transmitting user distance and direction information to the base station (or server) through the above-described remote controller or terminal, but may not be limited thereto.

As a specific example, FIG. 15 is a diagram illustrating a method of controlling a metalens based on an artificial intelligence system according to an embodiment of the present disclosure. Referring to FIG. 15 , a remote controller 1510 and a video output device 1520 may be provided. In this case, the remote controller 1510 may be a device that communicates with a server or an access point (AP). As another example, the remote controller 1510 may be a user terminal. For example, the terminal may replace the function of the remote controller 1510 and may be a video output device 1520 or a device that operates through authentication with a server or AP. That is, for convenience of description, the description is based on the remote controller 1510, but is not limited thereto, and may be performed through a terminal or other device, and is not limited to a specific form. Also, as an example, the image output device 1520 may be a TV or other device that outputs images. Also, as an example, the video output device 1520 may be a device having a server or an AP. That is, the video output device 1520 may be combined with a server or AP, and may not be limited to a specific form. As another example, there is a separate base station (or server) that communicates with the above-described remote controller 1510 or terminal, and the video output device may be a device controlled based on this, and is not limited to a specific form. can

For example, referring to FIG. 15 , when the remote controller 1510 is turned on, the remote controller 1510 may transmit user interference recognition information to the image output device 1520. In this case, the user interference recognition information may be direction and distance information with the image output device 1520. Also, as an example, other user-related recognition information may be further included, and is not limited to a specific form.

After that, the image output device 1520 recognizes the user based on the user interference recognition information, and transfers the information about this to the artificial intelligence metalens characteristic prediction unit to obtain the index (or control value) of the metalens characteristic factor in codebook format. can be derived. Then, the control value is transferred to the metalens controller, and based on this, the image output device 1520 may output an image, which may be the same as in FIG. 14 . Also, as an example, the above-described operation may be repeated based on signaling of the remote controller 1510. For example, when the remote controller 1510 is turned on, the above-described operation may be performed. Also, as an example, the remote controller 1510 may transmit signaling to the video output device 1520 based on a preset period and periodically perform the above-described operation. As another example, the remote control 1510 may perform the above-described operation based on event triggering (e.g. user's remote control input), and may not be limited to a specific form.

Also, as an example, FIG. 16 is a flowchart illustrating a method of controlling image input/output based on a metalens according to an embodiment of the present disclosure. Referring to FIG. 16 , the operation of the Metalens artificial intelligence system for user-based image input/output may include a user recognition step, a user tracking repetition step, and a periodic sequential step. For example, the user recognition step may be a step of recognizing at least one information of the user's direction, distance, and state (viewing angle, movement) through the input image by sequentially changing the steering of the metalens. (S1610 ) At this time, as an example, a camera (particularly an infrared camera) having a limited angle of view can produce a wide-angle effect without the help of a mechanical device, and may not be limited to a specific shape. The artificial intelligence user recognition unit may recognize user recognition information through at least one of object detection and feature extraction, and is not limited to a specific form.

Next, the repeating user tracking step may be a step of continuously tracking the user's position and changing the metal lens control of the image input/output device. For example, the video input device may transmit user recognition information to the base station (or server) (S1620), and the base station (or server) may transmit characteristic factors predicted based on the user recognition information to the video input/output device (S1630). ) After that, the metalens controller of the video input/output device may generate a control value corresponding to the received metalens characteristic factor or perform setting by referring to the codebook, and may derive the setting value. (S1640) At this time, Based on whether the setting value converges, if it does not converge, the user tracking repetition step may be repeated. On the other hand, when the setting values converge, a periodic sequential search step may be performed. As another example, the repeating user tracking step may be performed N times, and the periodic sequential search step may be performed after performing N times. At this time, as an example, since the optimal value of the metalens codebook is derived based on user recognition information, MAB AI or reinforcement learning may be used, and implementation may be possible through supervised learning. For example, when the above-described learning is performed, the location tracking of the user may be impossible or a certain amount of time may be required, and may not be limited to a specific form. Thereafter, the periodic sequential search may be performed at regular intervals to re-search the user or the environment as a whole. For example, a periodic sequential search step may be performed in order to sequentially search in all directions after a certain number of times when it is impossible to track the user's location (S1650).

Also, as an example, FIGS. 17 and 18 are diagrams illustrating a method of detecting an object in a user recognizer according to an embodiment of the present disclosure. The artificial intelligence system for controlling the metalens may include an artificial intelligence user recognition unit and a metalens characteristic prediction unit. At this time, the artificial intelligence user recognition unit may need to secure accuracy and real-time in the form of object detection. Therefore, the artificial intelligence user recognition unit may be suitable for a 1-Stage Detector, and the 1-Stage Detector is divided into a convolution layer for extracting features and a full connected layer for region and classification, which may be shown in FIG. 17.

Referring to FIG. 17 , features of an image obtained from a camera (TOF) may be extracted through a convolution layer with an object detection structure used by a user recognition unit. Then, the extracted features can be composed of Feature Maps, and Feature Maps are composed of a Bounding Box Regression part that predicts the position and size coordinates (x, y, w, h) of objects and multi-class classification that classifies objects. can be derived separately. However, this is only one example and is not limited to the above-described embodiment. In addition, as an example, referring to FIG. 18, in the feature map structure of Object Detection (YOLO), Objectness Score and Class Score can be derived for each individual Grid number S * S and B Box coordinates, each of which is an object (object ) and the probability that the target belongs to the class. Here, as an example, based on the above, the loss function may be equal to Equation 4 below.

[Equation 4]

At this time,

is a balancing parameter for balancing the loss for coordinates (x, y, w, h) with other losses,

may be a balancing parameter for balancing between a box with an object and a box without an object. For example, there may be more cells without objects than cells with objects in an image, and the above-described parameters may be used in consideration of this.

For example, the first and second lines in Equation 4 may represent losses of x, y, w, and h for the predictor bounding box j of grid cell i. Here, the coordinates of the center point and the horizontal and vertical sizes may be respectively indicated.

In the second line, we can take the square root to compensate for the large deviation caused by the large box. In addition, the third line may be the calculation of the loss of the confidence score for the predictor bounding box j of grid cell i where the target exists. (

= 1) On the other hand, the fourth line calculates the loss of the confidence score for the bounding box j of grid cell i where the target does not exist (

= 0) may have been. Finally, the fifth line may be a loss calculation of the conditional class probability for grid cell i where the target exists. For example, for Correct class c

= 1, otherwise

= 0. Also, as an example, in addition to object detection, a user's viewing angle may be additionally extracted and used as user recognition information, and may not be limited to a specific form.

Also, as an example, FIG. 19 illustrates a final input/output according to an embodiment of the present disclosure. For example, referring to FIG. 19(a), when an image and an image are extracted from a camera, the user's cognitive information (direction, distance, gaze direction, mobility) can be finally extracted through the metalens control value, which is described above. same as bar In addition, the metalens characteristic prediction unit can be implemented by supervised learning like the user recognition unit, but can also be implemented by reinforcement learning.

In more detail, referring to FIG. 19(b), the structure of the metalens control system using supervised learning is based on the user's cognitive information (direction, distance, gaze direction, mobility), which is a characteristic function of the metalens, such as focus position and distance. And the radius of curvature can be predicted, which can be as shown in FIG. 19 (b).

20 is a diagram illustrating a structure of a metalens control system using reinforcement learning (or MAB) according to an embodiment of the present disclosure. For example, reinforcement learning may consist of two inputs and one output, but may not be limited thereto. For example, in reinforcement learning, a state and a reward are used as inputs, and an agent can select an action as an output. At this time, the state may not be used in MAB. In addition, the action may be an action of providing a user-based optimal image by controlling the metal lens. The metalens control artificial intelligence system can derive the optimal control value as described above by obtaining reward values and changed state information for actions from the environment, using them for learning, and repeating the action selection process. there is.

21 is a block diagram of a metalens control system using reinforcement learning, according to an embodiment of the present disclosure. Referring to FIG. 21 , the reinforcement learning-based artificial intelligence 2110 may include a reinforcement learning model 2120. At this time, the reinforcement learning model 2120 consists of a learning unit 2121 and a prediction unit 2122, and can predict the next action at the same time as learning. Here, the user's recognition information (direction, distance, viewing direction, mobility), etc. may be used as the state factor, and may be expressed as Equation 5 below.

[Equation 5]

At this time, Action is a characteristic factor for generating a metalens control value selected by artificial intelligence and may be a discrete value or a continuous value. For example, a continuous value may be used as an input value of a metalens control value generator, a discrete value may be used as an index of a pre-shared codebook, and an operation may be as shown in Equation 6 below.

[Equation 6]

Also, for example, in reinforcement learning, reward may be an essential factor for learning. For example, a reaction can be predicted through user cognitive information, and a user's liking or concentration can be digitized and used as a reward. That is, user feedback information may be compensation information. However, the compensation information is not limited to a specific form and may be of various forms. For example, compensation information may be derived as shown in Equation 7 below.

[Equation 7]

Also, as an example, deep deterministic policy gradient (DDPG) may be used as reinforcement learning for predicting the above-described continuous value. DDPG can have the advantages of both policy gradient and DQN. That is, DDPG is stable and can be used in continuous space. Also, as an example, since a deterministic policy is used, convergence is possible quickly and it may be a lightweight algorithm.

As an example, FIG. 22 illustrates reinforcement learning using DDPG according to an embodiment of the present disclosure. Referring to FIG. 22, reinforcement learning using DDPG contains the characteristics of both the Actor-Critic algorithm and the DQN algorithm. Q (Value Action Function) and action can be predicted through Actor network and Critic network, respectively. In addition, the Critic network can be trained in the direction of reducing TD-Error (Temporal Difference) using Experience Replay Memory, and the learned Critic network can be used for Policy Gradient calculation and learning.

In addition, since the target policy is deterministic, the Bellman equation can be expressed through μ instead of π, and it can be the same as Equation 8.

[Equation 8]

At this time, State and Action are respectively

and

, and ~ E may be a value extracted as an environment. here,

is an action-value function for policy μ, γ is a discount factor, and r may be a reward value.

In addition, the weight of the Critic Network

Expressed as , and when the action policy with various probability distributions is β, the loss of the Critic Network can be as shown in Equation 9 below.

[Equation 9]

In Equation 9, the expected value is Transitions stored in Relay Buffer R (

,

) can be expressed by minibatching and taking the average value, which can be shown in Equation 10.

[Equation 10]

also,

becomes TD (Temporal Difference), and the Critic Network can be learned in the direction of reducing TD Error. At this time, the Actor Network may update the policy in the direction of maximizing the expected reward value (Expected Return), and may be as shown in Equation 11.

[Equation 11]

In addition, the expected value can be approximated using a Relay Buffer as shown in Equation 12 below.

[Equation 12]

Also, as an example, it may be easy to add Exploration as an off-policy model in DDPG. For example, the Exploration policy may be generated by adding a noise distribution and may be expressed as Equation 13.

[Equation 13]

Here, the variance of the normal distribution

Exploration rate can be adjusted using , and it can be adjusted adaptively using the above-mentioned decaying e-greedy or UCB.

As another example, prediction of discrete values may be implemented through selective artificial intelligence. As an example, FIG. 23 shows a selection artificial intelligence (MAB AI, 2310) system using Thompson sampling according to an embodiment of the present disclosure. For example, selection artificial intelligence (MAB AI, 2310) using Thomson sampling can be largely divided into a learning part and a selection part. For example, the learning unit may update the Thomson sampling parameters (α, β) for the previous action i based on the reward value (Reward), and the sampling parameters (α, β) according to the reward may be as shown in Equation 14 there is.

[Equation 14]

At this time,

is a reference compensation value, which may be different according to the compensation form of the IRS performance measurer in the terminal, and may not be limited to a specific form. In the Thomson sampling selection unit, the metalens control parameter index having the largest value among the sampled values can be selected by applying the accumulated α and β to the beta distribution. At this time, the selected {

,

} can be passed to the metalens control value generator. In addition, the index representing the characteristics of the lens {

,

} can be individually selected. also,

= (

,

), a new index that is unified, such as

A method of selecting using may also be used, and may not be limited to a specific form. As an example, the beta distribution may be as shown in Equation 15 below.

[Equation 15]

In addition, as an example, the metalens control value generator is a characteristic factor of the lens

,

You can create the control value of the metalens through .

,

are factors representing the position and distance of the focal point, respectively,

may be a factor for supporting a curved image output device.

For example, the metalens control value generator in the metalens controller generates the metalens control value through each characteristic factor, and the size of the metalens, the number of layers, and other settings based on the related factors are based on a pre-calculated codebook. can be controlled to operate.

In more detail, FIG. 24 is a diagram showing the structure of a metalens controller according to an embodiment of the present disclosure. The metalens controller 2410 is a parameter for generating a metalens control value from the artificial intelligence metalens characteristic factor prediction unit (

,

) and standard (size, number of control elements, metalens layer) information of the currently installed metalens of the image input/output device can be obtained. Also, as an example, the metalens controller 2410 may further obtain additional information, and may not be limited to a specific form. After that, the metalens control value generation unit 2411 may generate a control value by referring to the codebook of the metalens control value. As another example, the metalens control value generating unit 2411 may generate control values required for the N-layered metalens drivers by utilizing artificial intelligence, and based on this, the drivers control the metalens for each metalens layer. can do. In this case, a value output from the L-th metalens driver may be the same as Equation 16, and θ in Equation 16 may be a phase shift value for each device.

[Equation 16]

At this time, for example, indicating the focus position

,

May be a factor indicating the tilt direction and tilt angle of the lens, similar to OIS and the principle. For example, referring to FIG. 25 (a), the distance of AA' in FIG. 25 (a) is

can be Also, as an example, referring to FIG. 25 (b), the directivity of the lens may use the principle of changing the steering property according to the metal lens pattern. More specifically, the curve that becomes the phase shift of the meta lens is the focus position (

,

) can be changed in proportion to the control value.

In addition, for example, representing the focal length

represents wide-angle or telephoto, and may be a characteristic factor related to the role of zoom. For example, a zoom configuration principle may be applied to the metalens, and the same effect as mechanically changing the position of the metalens itself may be obtained by adjusting the curvature (phase shift slope) of the metalens.

As a specific example, FIG. 26 (a) is a diagram illustrating a method of producing wide-angle and telephoto effects by using the curvature of a metal lens in a 3-group zoom lens method. Referring to FIG. 26 (a), the focal length

A control value of the metalens can be generated to adjust the curvature of the intermediate concave lens in inverse proportion to .

In addition, as an example, the radius of curvature to represent the curved image output effect

can be used. At this time, the radius of curvature

is a factor indicating the degree of curved shape, and as the radius of curvature increases, a more gentle curved image can be output. For example, in order to generate a metalens control value for curved image output, it is necessary to apply a zoom degree in proportion to a distance from the center point (center) to the horizontal (x-axis coordinate). Therefore, the focal length (

) uses a uniform zoom effect on the entire screen, but curved image output needs to apply a different zoom effect in the x-axis horizontal direction from the center line including the origin. As an example, FIG. 26(b) is a diagram showing a curved image output device according to a change in the curvature of a metal lens and magnification for each position. Referring to FIG. 26 (b), it may be necessary to use a lens application value applied according to the position of the meta lens zoom magnification. Therefore, an additional metal lens layer may be required in addition to the basic 3-group zoom lens.

As another example, FIG. 27 is a diagram illustrating a system operating through federated learning of various image input/output devices according to an embodiment of the present disclosure. As an example, referring to FIG. 27 , federated learning may be operated by AirComp through a base station. In this case, the video input/output devices may exist indoors or outdoors. For example, the video input/output device may transmit its own local model information to a base station at various locations and distances, and the base station may update global model information based on the received local model information and provide the updated global model information to respective terminals. In this case, each of the terminals may be a video input/output device. For example, federated learning can be used to check whether control of video input/output devices is properly operated through video input devices (cameras) at various locations and distances, and through this, new services can be created.

28 is a diagram illustrating a method of operating a base station according to an embodiment of the present disclosure. Referring to FIG. 28, the base station may receive user recognition information from the first terminal (S2810). At this time, the base station may be a device that communicates with the terminal. As another example, the base station may be the aforementioned server or access point, and may not be limited to a specific form. Also, as an example, the first terminal may be an image input device. As another example, the first terminal may be a device having an image input function, and is not limited to a specific form. Also, as an example, the second terminal may be an image output device. As another example, the second terminal may be a device having a video output function, and is not limited to a specific form. Also, as an example, the first terminal and the second terminal may be implemented in one device, and are not limited to a specific form.

When the base station receives the user recognition information from the first terminal, the base station may obtain a control value based on the metalens characteristic factor using the user recognition information (S2820). It may be transmitted to the first terminal and the second terminal. (S2830) At this time, as an example, the control value may be determined based on a codebook based on a metalens characteristic factor. Here, the metalens characteristic factor may be determined based on at least one of focus position information, focal distance information, and curvature radius information based on each of the metalens elements. Also, the focal position information, the focal distance information, and the radius of curvature information are expressed as respective indices based on quantization, and a control value may be derived based on each index, as described above. Also, as an example, the user recognition information may include at least one or more of user direction information, user distance information, and user status information, as described above. At this time, as an example, the first terminal may obtain location information of the first terminal and transmit the location information of the first terminal to the base station together with user recognition information. The base station may derive a control value by further considering the location information of the first terminal, as described above.

Also, as an example, the base station may include a Metalens artificial intelligence system, and the Metalens artificial intelligence system may perform learning based on a learning model. In this case, the acquired user recognition information may be an input of the learning model, and the control value may be an output of the learning model. As a specific example, the learning model operates based on reinforcement learning, and the reinforcement learning may use state information and reward information as inputs based on the acquired user recognition information and control value. In addition, behavioral information corresponding to the control value may be output, as described above.

As another example, the learning model operates based on Thomson sampling, and the Thomson sampling may perform learning based on compensation information having α and β parameters based on the acquired user recognition information and control value, which is described above. It's like a bar.

29 is a flowchart illustrating a method of operating a terminal according to an embodiment of the present disclosure. Referring to FIG. 29 , the terminal may transmit user recognition information to the base station (S2910). At this time, the terminal may be the first terminal of FIG. 28 . Also, as an example, the terminal may be a device in which the first terminal and the second terminal are implemented together, and may not be limited to a specific form. Thereafter, the terminal may receive a control value derived by the base station based on the transmitted user recognition information. (S2920) At this time, as an example, the control value may be determined based on a codebook based on a metalens characteristic factor. . Here, the metalens characteristic factor may be determined based on at least one of focus position information, focal distance information, and curvature radius information based on each of the metalens elements. Also, the focal position information, the focal distance information, and the radius of curvature information are expressed as respective indices based on quantization, and a control value may be derived based on each index, as described above. Also, as an example, the user recognition information may include at least one or more of user direction information, user distance information, and user status information, as described above. At this time, as an example, the first terminal may obtain location information of the first terminal and transmit the location information of the first terminal to the base station together with user recognition information. The base station may derive a control value by further considering the location information of the first terminal, as described above.

It is obvious that examples of the proposed schemes described above may also be included as one of the implementation methods of the present disclosure, and thus may be regarded as a kind of proposed schemes. In addition, the above-described proposed schemes may be implemented independently, but may also be implemented in a combination (or merged) form of some proposed schemes. Information on whether the proposed methods are applied (or information on the rules of the proposed methods) may be defined so that the base station informs the terminal through a predefined signal (eg, a physical layer signal or a higher layer signal). there is.

The present disclosure may be embodied in other specific forms without departing from the technical ideas and essential characteristics described in the present disclosure. Accordingly, the above detailed description should not be construed as limiting in all respects and should be considered illustrative. The scope of the present disclosure should be determined by reasonable interpretation of the appended claims, and all changes within the equivalent range of the present disclosure are included in the scope of the present disclosure. In addition, claims that do not have an explicit citation relationship in the claims may be combined to form an embodiment or may be included as new claims by amendment after filing.

Embodiments of the present disclosure may be applied to various wireless access systems. As an example of various wireless access systems, there is a 3rd Generation Partnership Project (3GPP) or 3GPP2 system.

Embodiments of the present disclosure may be applied not only to the various wireless access systems, but also to all technical fields to which the various wireless access systems are applied. Furthermore, the proposed method can be applied to mmWave and THz communication systems using ultra-high frequency bands.

Additionally, embodiments of the present disclosure may be applied to various applications such as free-running vehicles and drones.

Claims

In the method of operating a base station in a wireless communication system,

Receiving user identification information from a first terminal;

obtaining a control value based on a metalens characteristic factor using the obtained user recognition information; and

Transmitting the obtained control value to the first terminal and the second terminal; including, base station operating method.
According to claim 1,

The obtained control value is determined based on a codebook based on the metalens characteristic factor.
According to claim 2,

The metalens characteristic factor is determined based on at least one of focus position information, focal distance information, and curvature radius information based on each of the metalens elements, base station operating method.
According to claim 3,

The focal position information, the focal distance information, and the radius of curvature information are expressed as respective indices based on quantization,

The control value is derived based on the respective index, the base station operating method.
According to claim 1,

The user recognition information includes at least one or more of user direction information, user distance information, and user status information.
According to claim 5,

Wherein the first terminal is a terminal having a video input function, and obtains the user recognition information based on the video input function.
According to claim 6,

The first terminal obtains location information of the first terminal, transmits the location information of the first terminal together with the user recognition information to the base station,

wherein the base station derives the control value by further considering location information of the first terminal.
According to claim 1,

The second terminal is a terminal having a video output function, the base station operating method.
According to claim 1,

The base station includes at least one function of a server and an access point (access point), base station operating method.
According to claim 1,

The base station is equipped with a metalens artificial intelligence system,

The metalens artificial intelligence system performs learning based on a learning model, the obtained user recognition information is an input of the learning model, and the control value is an output of the learning model.
According to claim 10,

The learning model operates based on reinforcement learning,

Wherein the reinforcement learning uses state information and reward information as inputs based on the acquired user recognition information and the control value, and outputs behavioral information corresponding to the control value.
According to claim 10,

The learning model operates based on Thomson sampling,

The Thomson sampling performs learning based on compensation information having α and β parameters based on the acquired user recognition information and the control value.
In a terminal operating method in a wireless communication system,

Transmitting user identification information to a base station; and

Receiving a control value derived by the base station based on the transmitted user recognition information; including, a terminal operating method.
In a base station of a wireless communication system,

transceiver; and

A processor connected to the transceiver;

the processor,

Receiving user recognition information from a first terminal using the transceiver;

Obtaining a control value based on a metalens characteristic factor using the obtained user recognition information, and

Transmitting the obtained control value to the first terminal and the second terminal using the transceiver, the base station.
In the terminal of the wireless communication system,

transceiver; and

A processor connected to the transceiver;

the processor,

Transmitting user recognition information to a base station using the transceiver,

Receiving a control value derived by the base station based on the transmitted user recognition information using the transceiver.
An apparatus comprising at least one memory and at least one processor functionally connected to the at least one memory, comprising:

The at least one processor is the device,

Receiving user recognition information from a first terminal;

Obtaining a control value based on a metalens characteristic factor using the obtained user recognition information, and

An apparatus for controlling transmission of the obtained control value to the first terminal and the second terminal.
In a non-transitory computer-readable medium storing at least one instruction,

comprising the at least one instruction executable by a processor;

The at least one command,

Receiving user recognition information from a first terminal;

Obtaining a control value based on a metalens characteristic factor using the obtained user recognition information, and

A computer readable medium for controlling transmission of the obtained control value to the first terminal and the second terminal.