CN113595608B - Millimeter wave/terahertz communication method, device and system based on visual perception - Google Patents

Millimeter wave/terahertz communication method, device and system based on visual perception Download PDF

Info

Publication number
CN113595608B
CN113595608B CN202110698104.4A CN202110698104A CN113595608B CN 113595608 B CN113595608 B CN 113595608B CN 202110698104 A CN202110698104 A CN 202110698104A CN 113595608 B CN113595608 B CN 113595608B
Authority
CN
China
Prior art keywords
mobile terminal
base station
camera
reflecting surface
intelligent reflecting
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110698104.4A
Other languages
Chinese (zh)
Other versions
CN113595608A (en
Inventor
高飞飞
张腾宇
张�浩
王鸣锦
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tsinghua University
Original Assignee
Tsinghua University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tsinghua University filed Critical Tsinghua University
Priority to CN202110698104.4A priority Critical patent/CN113595608B/en
Publication of CN113595608A publication Critical patent/CN113595608A/en
Application granted granted Critical
Publication of CN113595608B publication Critical patent/CN113595608B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04B—TRANSMISSION
    • H04B7/00—Radio transmission systems, i.e. using radiation field
    • H04B7/02—Diversity systems; Multi-antenna system, i.e. transmission or reception using multiple antennas
    • H04B7/04—Diversity systems; Multi-antenna system, i.e. transmission or reception using multiple antennas using two or more spaced independent antennas
    • H04B7/06—Diversity systems; Multi-antenna system, i.e. transmission or reception using multiple antennas using two or more spaced independent antennas at the transmitting station
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04B—TRANSMISSION
    • H04B7/00—Radio transmission systems, i.e. using radiation field
    • H04B7/02—Diversity systems; Multi-antenna system, i.e. transmission or reception using multiple antennas
    • H04B7/04—Diversity systems; Multi-antenna system, i.e. transmission or reception using multiple antennas using two or more spaced independent antennas
    • H04B7/04013—Intelligent reflective surfaces
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/02—Systems using reflection of radio waves, e.g. primary radar systems; Analogous systems
    • G01S13/06—Systems determining position data of a target
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S5/00—Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations
    • G01S5/02—Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations using radio waves
    • G01S5/08—Position of single direction-finder fixed by determining direction of a plurality of spaced sources of known location
    • H—ELECTRICITY
    • H01—ELECTRIC ELEMENTS
    • H01Q—ANTENNAS, i.e. RADIO AERIALS
    • H01Q3/00—Arrangements for changing or varying the orientation or the shape of the directional pattern of the waves radiated from an antenna or antenna system
    • H01Q3/26—Arrangements for changing or varying the orientation or the shape of the directional pattern of the waves radiated from an antenna or antenna system varying the relative phase or relative amplitude of energisation between two or more active radiating elements; varying the distribution of energy across a radiating aperture
    • H01Q3/30—Arrangements for changing or varying the orientation or the shape of the directional pattern of the waves radiated from an antenna or antenna system varying the relative phase or relative amplitude of energisation between two or more active radiating elements; varying the distribution of energy across a radiating aperture varying the relative phase between the radiating elements of an array
    • H—ELECTRICITY
    • H01—ELECTRIC ELEMENTS
    • H01Q—ANTENNAS, i.e. RADIO AERIALS
    • H01Q3/00—Arrangements for changing or varying the orientation or the shape of the directional pattern of the waves radiated from an antenna or antenna system
    • H01Q3/26—Arrangements for changing or varying the orientation or the shape of the directional pattern of the waves radiated from an antenna or antenna system varying the relative phase or relative amplitude of energisation between two or more active radiating elements; varying the distribution of energy across a radiating aperture
    • H01Q3/30—Arrangements for changing or varying the orientation or the shape of the directional pattern of the waves radiated from an antenna or antenna system varying the relative phase or relative amplitude of energisation between two or more active radiating elements; varying the distribution of energy across a radiating aperture varying the relative phase between the radiating elements of an array
    • H01Q3/32—Arrangements for changing or varying the orientation or the shape of the directional pattern of the waves radiated from an antenna or antenna system varying the relative phase or relative amplitude of energisation between two or more active radiating elements; varying the distribution of energy across a radiating aperture varying the relative phase between the radiating elements of an array by mechanical means
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04B—TRANSMISSION
    • H04B7/00—Radio transmission systems, i.e. using radiation field
    • H04B7/02—Diversity systems; Multi-antenna system, i.e. transmission or reception using multiple antennas
    • H04B7/04—Diversity systems; Multi-antenna system, i.e. transmission or reception using multiple antennas using two or more spaced independent antennas
    • H04B7/06—Diversity systems; Multi-antenna system, i.e. transmission or reception using multiple antennas using two or more spaced independent antennas at the transmitting station
    • H04B7/0613—Diversity systems; Multi-antenna system, i.e. transmission or reception using multiple antennas using two or more spaced independent antennas at the transmitting station using simultaneous transmission
    • H04B7/0615—Diversity systems; Multi-antenna system, i.e. transmission or reception using multiple antennas using two or more spaced independent antennas at the transmitting station using simultaneous transmission of weighted versions of same signal
    • H04B7/0617—Diversity systems; Multi-antenna system, i.e. transmission or reception using multiple antennas using two or more spaced independent antennas at the transmitting station using simultaneous transmission of weighted versions of same signal for beam forming

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Mobile Radio Communication Systems (AREA)
  • Radar Systems Or Details Thereof (AREA)

Abstract

本发明提供一种基于视觉感知的毫米波/太赫兹通信方法、装置和系统，其中通信方法包括：利用相机实时采集图像信息以及移动端的深度信息；利用所述智能反射面根据所述图像信息识别出移动端在图像中的位置；基于所述移动端的深度信息，将移动端在图像中的位置转化为移动端的三维坐标；基于所述移动端的三维坐标所计算出的移动端所在实时位置的俯仰角和方位角，控制智能反射面的天线将信号波束指向移动端方向。本发明可以快速识别移动端并确定目标位置，并控制智能反射面的相控阵天线产生对应方向的俯仰角和方位角，可以使波束更快指向对应目标，提高识别移动端位置的能力，降低时延，实现实时波束跟踪。

Figure 202110698104

The present invention provides a millimeter wave/terahertz communication method, device and system based on visual perception, wherein the communication method includes: using a camera to collect image information and depth information of a mobile terminal in real time; using the intelligent reflective surface to identify according to the image information The position of the mobile terminal in the image is obtained; based on the depth information of the mobile terminal, the position of the mobile terminal in the image is converted into the three-dimensional coordinates of the mobile terminal; the pitch of the real-time position of the mobile terminal is calculated based on the three-dimensional coordinates of the mobile terminal Angle and azimuth, control the antenna of the smart reflector to point the signal beam in the direction of the mobile terminal. The invention can quickly identify the mobile terminal and determine the target position, and control the phased array antenna of the intelligent reflective surface to generate the pitch angle and azimuth angle of the corresponding direction, which can make the beam point to the corresponding target faster, improve the ability to identify the position of the mobile terminal, and reduce the time delay to achieve real-time beam tracking.

Figure 202110698104

Description

基于视觉感知的毫米波/太赫兹通信方法、装置和系统Millimeter wave/terahertz communication method, device and system based on visual perception

技术领域technical field

本发明涉及通信技术领域，尤其涉及一种基于视觉感知的毫米波/太赫兹通信方法、装置和系统。The present invention relates to the field of communication technologies, and in particular, to a millimeter wave/terahertz communication method, device and system based on visual perception.

背景技术Background technique

通信技术行业自诞生以来一直遵循着“十年一换代”的“摩尔定律”，从1980年开始的第一代移动通信(1G)的模拟通信方式，到1990左右以TDMA为核心技术的2G，再到本世纪初以CDMA为核心技术的3G时代，再到2010年左右以OFDM技术为核心的4G移动通信网络的大规模商用，移动通信整体在朝着越来越快、越来越稳定、越来越安全的方向发展。2020年因为第五代移动通信技术(5G)的问世而被称为“5G元年”，其在增强的移动带宽通信(eMBB)、超可靠低时延通信(URLLC)、大规模机器类型通信(mMTC)等场景有着很好的表现。Since its birth, the communication technology industry has been following the "Moore's Law" of "every ten years", from the analog communication method of the first generation of mobile communication (1G) started in 1980, to the 2G with TDMA as the core technology around 1990, In the 3G era with CDMA as the core technology at the beginning of this century, and the large-scale commercial use of 4G mobile communication networks with OFDM technology as the core around 2010, the overall mobile communication is moving towards faster, more stable, more and more secure. 2020 is called "the first year of 5G" because of the advent of the fifth-generation mobile communication technology (5G), which is in the field of enhanced mobile bandwidth communication (eMBB), ultra-reliable and low-latency communication (URLLC), large-scale machine type communication. (mMTC) and other scenarios have good performance.

目前，传统的高频通信领域的智能反射面的波束赋形与跟踪基本采用如下方式：先使用导频扫描和遍历智能反射面方向上的所有相位，以确定移动端的方向。当移动端移动时，传统方法需要不断发送导频扫描移动端方向附近的信噪比，移动端需要通过上行信道将信噪比的值反馈给智能反射面，随后智能反射面选取最大的方向的信噪比为反射波束的方向；在雷达相控阵领域，主要采用的是将信道分为通信信道和信标信道，依赖差功分网络对信标信道的信道值进行波达算法分析，由此判断信号来源方向。At present, the beamforming and tracking of the smart reflector in the traditional high-frequency communication field basically adopts the following method: first use the pilot frequency to scan and traverse all the phases in the direction of the smart reflector to determine the direction of the mobile terminal. When the mobile terminal moves, the traditional method needs to continuously send the pilot frequency to scan the signal-to-noise ratio near the mobile terminal. The mobile terminal needs to feed back the value of the signal-to-noise ratio to the smart reflector through the uplink channel, and then the smart reflector selects the maximum direction. The signal-to-noise ratio is the direction of the reflected beam; in the field of radar phased array, it is mainly used to divide the channel into communication channel and beacon channel, and rely on the differential power division network to analyze the channel value of the beacon channel by the wave arrival algorithm. Determine the signal source direction.

上述系统对智能反射面的硬件依赖程度较大，当扫描信号能量较弱或信标信号能量难以检测时，不便确定移动端方向。并且导频占用了较多的信道资源。当移动端数较多的时候，计算复杂度也急剧上升，时延较大。The above system relies heavily on the hardware of the smart reflective surface, and when the energy of the scanning signal is weak or the energy of the beacon signal is difficult to detect, it is inconvenient to determine the direction of the mobile terminal. And the pilot frequency occupies more channel resources. When the number of mobile terminals is large, the computational complexity also increases sharply, and the delay is large.

发明内容SUMMARY OF THE INVENTION

本发明提供一种基于视觉感知的毫米波/太赫兹通信方法、装置和系统，用以解决现有技术中存在的技术缺陷，以实现使用光谱判断目标位置，并跟踪目标，节约了导频所占用的信道资源；当移动端数目较多时，计算复杂度并未改变，时延较小。The present invention provides a millimeter wave/terahertz communication method, device and system based on visual perception, which are used to solve the technical defects in the prior art, so as to use the spectrum to determine the target position and track the target, and save the pilot frequency. Occupied channel resources; when the number of mobile terminals is large, the computational complexity does not change, and the delay is small.

本发明提供一种基于视觉感知的毫米波/太赫兹通信方法，包括：The present invention provides a millimeter wave/terahertz communication method based on visual perception, including:

利用相机实时采集图像信息以及移动端的深度信息；所述相机设置在智能反射面上；The camera is used to collect image information and depth information of the mobile terminal in real time; the camera is arranged on the intelligent reflection surface;

利用所述智能反射面根据所述图像信息识别出移动端在图像中的位置；Using the smart reflective surface to identify the position of the mobile terminal in the image according to the image information;

基于所述移动端的深度信息，将移动端在图像中的位置转化为移动端的三维坐标；Based on the depth information of the mobile terminal, convert the position of the mobile terminal in the image into the three-dimensional coordinates of the mobile terminal;

基于所述移动端的三维坐标所计算出的移动端所在实时位置的俯仰角和方位角，控制智能反射面的天线将信号波束指向移动端方向。Based on the pitch angle and azimuth angle of the real-time position of the mobile terminal calculated based on the three-dimensional coordinates of the mobile terminal, the antenna of the intelligent reflecting surface is controlled to point the signal beam in the direction of the mobile terminal.

优选的，所述的基于视觉感知的毫米波/太赫兹通信方法，其中，所述相机坐标原点在相机表面的几何中心处，所述智能反射面的坐标原点近似地和相机坐标原点相同；Preferably, in the millimeter wave/terahertz communication method based on visual perception, the camera coordinate origin is at the geometric center of the camera surface, and the coordinate origin of the smart reflective surface is approximately the same as the camera coordinate origin;

所述移动端在相机坐标系中的坐标为：The coordinates of the mobile terminal in the camera coordinate system are:

(XOQ,YOQ,ZOQ)(X OQ ,Y OQ ,Z OQ )

所述移动端在智能反射面坐标系中的坐标为：The coordinates of the mobile terminal in the smart reflective surface coordinate system are:

Figure BDA0003129350770000021
Figure BDA0003129350770000021

所述移动端在相机坐标系中以及在智能反射面坐标系中的坐标的转换公式为：The conversion formula of the coordinates of the mobile terminal in the camera coordinate system and in the smart reflective surface coordinate system is:

Figure BDA0003129350770000031
Figure BDA0003129350770000031

所述移动端的俯仰角θ由下式求得：The pitch angle θ of the mobile terminal is obtained by the following formula:

Figure BDA0003129350770000032
Figure BDA0003129350770000032

所述移动端的方位角

Figure BDA0003129350770000033
由下式求得：The azimuth of the mobile terminal
Figure BDA0003129350770000033
It is obtained by the following formula:

Figure BDA0003129350770000034
Figure BDA0003129350770000034

所述方法还包括：按照下式计算出所述智能反射面的每根天线对应的反射波束的相角

Figure BDA0003129350770000035
The method further includes: calculating the phase angle of the reflected beam corresponding to each antenna of the smart reflective surface according to the following formula
Figure BDA0003129350770000035

Figure BDA0003129350770000036
Figure BDA0003129350770000036

上式中，j为虚数单位，e为自然对数的底，M为矩阵行数，N为矩阵列数，λ为电磁波波长，d为天线间距。In the above formula, j is the imaginary unit, e is the base of the natural logarithm, M is the number of matrix rows, N is the number of matrix columns, λ is the wavelength of the electromagnetic wave, and d is the antenna spacing.

优选的，所述的基于视觉感知的毫米波/太赫兹通信方法，其中，所述方法还包括：Preferably, in the millimeter wave/terahertz communication method based on visual perception, the method further includes:

利用所述智能反射面根据所述图像信息识别出基站端在图像中的位置；Using the intelligent reflective surface to identify the position of the base station in the image according to the image information;

基于所述基站端的深度信息，将基站端在图像中的位置转化为基站端的三维坐标；Based on the depth information of the base station, convert the position of the base station in the image into the three-dimensional coordinates of the base station;

基于所述基站端的三维坐标所计算出的基站端所在实时位置的俯仰角和方位角，对所述智能反射面的天线进行入射角补偿。Based on the elevation angle and the azimuth angle of the real-time position of the base station end calculated based on the three-dimensional coordinates of the base station end, the incident angle compensation is performed on the antenna of the smart reflector.

优选的，所述的基于视觉感知的毫米波/太赫兹通信方法，其中，所述基于所述基站端的三维坐标所计算出的基站端所在实时位置的俯仰角和方位角，包括：Preferably, in the millimeter wave/terahertz communication method based on visual perception, the pitch angle and azimuth angle of the real-time location of the base station calculated based on the three-dimensional coordinates of the base station include:

按照下式计算出所述智能反射面的每根天线对应的入射角：Calculate the incident angle corresponding to each antenna of the smart reflector according to the following formula:

Figure BDA0003129350770000041
Figure BDA0003129350770000041

上式中，

Figure BDA0003129350770000042
为基站端俯仰角，θ0为基站端的方位角。In the above formula,
Figure BDA0003129350770000042
is the pitch angle of the base station, and θ 0 is the azimuth of the base station.

优选的，所述的基于视觉感知的毫米波/太赫兹通信方法，其中，所述控制智能反射面的天线将信号波束指向移动端方向之前，通过下式得到所述智能反射面的天线的移相相位

Figure BDA0003129350770000043
Preferably, in the millimeter wave/terahertz communication method based on visual perception, before the antenna that controls the smart reflective surface points the signal beam in the direction of the mobile terminal, the displacement of the antenna of the smart reflective surface is obtained by the following formula phase
Figure BDA0003129350770000043

Figure BDA0003129350770000044
Figure BDA0003129350770000044

其中，·表示点乘，即两个维度相同的矩阵对应位置的元素相乘。Among them, · represents the point multiplication, that is, the elements of the corresponding positions of the two matrices with the same dimension are multiplied.

优选的，所述的基于视觉感知的毫米波/太赫兹通信方法，其中，所述方法还包括：Preferably, in the millimeter wave/terahertz communication method based on visual perception, the method further includes:

利用相机所采集的图像识别移动端与智能反射面的位置；当基站端与移动端之间的直射径可达的情况下，基站端根据相机采集的图像判断移动端的位置，再控制阵列天线将波束指向移动端的位置，所输入的相角矩阵为

Figure BDA0003129350770000045
当直射径被遮挡时，基站端根据相机采集的图像判断智能反射面的位置，再控制阵列天线将波束指向智能反射面的位置，所输入的相角矩阵为
Figure BDA0003129350770000046
Use the image collected by the camera to identify the position of the mobile terminal and the smart reflector; when the direct radiation between the base station and the mobile terminal is reachable, the base station determines the position of the mobile terminal according to the image collected by the camera, and then controls the array antenna to The position where the beam points to the mobile terminal, the input phase angle matrix is
Figure BDA0003129350770000045
When the direct beam is blocked, the base station determines the position of the smart reflector according to the image collected by the camera, and then controls the array antenna to point the beam to the position of the smart reflector. The input phase angle matrix is:
Figure BDA0003129350770000046

优选的，所述的基于视觉感知的毫米波/太赫兹通信方法，其中，所述方法还包括：Preferably, in the millimeter wave/terahertz communication method based on visual perception, the method further includes:

利用相机采集的图像识别基站端与智能反射面的位置，当基站端与移动端之间的直射径可达的情况下，移动端根据其相机采集的图像判断基站端位置，再控制阵列天线将波束指向基站端的位置，所输入的相角矩阵为

Figure BDA0003129350770000047
当直射径被遮挡时，移动端根据其相机采集的图像判断智能反射面的位置，再控制阵列天线将波束指向智能反射面的位置，所输入的相角矩阵为
Figure BDA0003129350770000051
Use the image collected by the camera to identify the position of the base station and the smart reflector. When the direct radiation between the base station and the mobile terminal is reachable, the mobile terminal determines the position of the base station according to the image collected by the camera, and then controls the array antenna to The position where the beam points to the base station, and the input phase angle matrix is
Figure BDA0003129350770000047
When the direct beam is blocked, the mobile terminal judges the position of the smart reflector according to the image collected by its camera, and then controls the array antenna to point the beam to the position of the smart reflector. The input phase angle matrix is:
Figure BDA0003129350770000051

本发明还提供了一种基于视觉感知的毫米波/太赫兹通信装置，包括：相机、基站端、智能反射面、移动端；The present invention also provides a millimeter wave/terahertz communication device based on visual perception, comprising: a camera, a base station, an intelligent reflective surface, and a mobile terminal;

所述相机用于实时采集图像信息以及移动端的深度信息；所述相机设置在智能反射面上；The camera is used to collect image information and depth information of the mobile terminal in real time; the camera is arranged on the intelligent reflection surface;

所述智能反射面根据所述图像信息识别出移动端在图像中的位置；所述相机还用于基于所述移动端的深度信息，将移动端在图像中的位置转化为移动端的三维坐标；The intelligent reflective surface identifies the position of the mobile terminal in the image according to the image information; the camera is also used to convert the position of the mobile terminal in the image into three-dimensional coordinates of the mobile terminal based on the depth information of the mobile terminal;

所述智能反射面还用于基于所述移动端的三维坐标所计算出的移动端所在实时位置的俯仰角和方位角，控制智能反射面的天线将信号波束指向移动端方向。The smart reflective surface is also used to control the antenna of the smart reflective surface to point the signal beam in the direction of the mobile terminal based on the pitch angle and azimuth angle of the real-time position of the mobile terminal calculated based on the three-dimensional coordinates of the mobile terminal.

优选的，所述的基于视觉感知的毫米波/太赫兹通信装置，其中，所述智能反射面根据所述图像信息识别出基站端在图像中的位置；所述相机还用于基于所述基站端的深度信息，将基站端在图像中的位置转化为基站端的三维坐标；Preferably, in the millimeter wave/terahertz communication device based on visual perception, wherein the intelligent reflective surface identifies the position of the base station in the image according to the image information; the camera is also used for The depth information of the terminal is converted into the position of the base station in the image into the three-dimensional coordinates of the base station;

所述智能反射面还用于基于所述基站端的三维坐标所计算出的基站端所在实时位置的俯仰角和方位角，对所述智能反射面的天线进行入射角补偿。The smart reflective surface is also used for performing incident angle compensation on the antenna of the smart reflective surface based on the elevation angle and the azimuth angle of the real-time position of the base station end calculated based on the three-dimensional coordinates of the base station end.

优选的，所述的基于视觉感知的毫米波/太赫兹通信装置，其中，所述智能反射面的天线采用无源相控阵天线，和/或，所述相机具有计算物体左右距离及深度的功能，和/或，所述相机为双目相机。Preferably, in the millimeter wave/terahertz communication device based on visual perception, the antenna of the smart reflective surface adopts a passive phased array antenna, and/or the camera has a function of calculating the left and right distance and depth of an object. function, and/or, the camera is a binocular camera.

优选的，所述的基于视觉感知的毫米波/太赫兹通信装置，其中，所述相机坐标原点在相机表面的几何中心处，所述智能反射面的坐标原点近似地和相机坐标原点相同。Preferably, in the millimeter wave/terahertz communication device based on visual perception, the camera coordinate origin is at the geometric center of the camera surface, and the coordinate origin of the smart reflective surface is approximately the same as the camera coordinate origin.

本发明还提供了一种基于视觉感知的毫米波/太赫兹通信系统，包括所述的基于视觉感知的毫米波/太赫兹通信装置。The present invention also provides a millimeter wave/terahertz communication system based on visual perception, including the millimeter wave/terahertz communication device based on visual perception.

本发明提供的基于视觉感知的毫米波/太赫兹通信方法、装置和系统，使用相机作为视觉辅助，可以快速识别移动端并确定目标位置，并控制智能反射面的相控阵天线产生对应方向的俯仰角和方位角，可以使波束更快指向对应目标，提高识别移动端位置的能力，降低时延。当移动端运动时，能够计算出移动端实时位置，控制智能反射面随之改变反射方向，实现实时波束跟踪。The millimeter wave/terahertz communication method, device and system based on visual perception provided by the present invention, using the camera as a visual aid, can quickly identify the mobile terminal and determine the target position, and control the phased array antenna of the intelligent reflective surface to generate the corresponding direction. The pitch angle and azimuth angle can make the beam point to the corresponding target faster, improve the ability to identify the location of the mobile terminal, and reduce the delay. When the mobile terminal moves, the real-time position of the mobile terminal can be calculated, and the intelligent reflective surface can be controlled to change the reflection direction accordingly to realize real-time beam tracking.

附图说明Description of drawings

为了更清楚地说明本发明或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作一简单地介绍，显而易见地，下面描述中的附图是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。In order to explain the present invention or the technical solutions in the prior art more clearly, the following will briefly introduce the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are the For some embodiments of the invention, for those of ordinary skill in the art, other drawings can also be obtained according to these drawings without any creative effort.

图1是本发明提供的基于视觉感知的毫米波/太赫兹通信方法的流程示意图；1 is a schematic flowchart of a visual perception-based millimeter wave/terahertz communication method provided by the present invention;

图2是本发明提供的相机获取的图片转化成的空间三维坐标的示意图；Fig. 2 is the schematic diagram of the spatial three-dimensional coordinates converted into the picture obtained by the camera provided by the present invention;

图3是本发明提供的智能反射面设定俯仰角与方位角时参考的坐标系的示意图。FIG. 3 is a schematic diagram of a coordinate system for reference when the intelligent reflective surface provided by the present invention sets the pitch angle and the azimuth angle.

图4是本发明提供的基于视觉感知的毫米波/太赫兹通信装置之三的结构示意图。FIG. 4 is a schematic structural diagram of the third millimeter wave/terahertz communication device based on visual perception provided by the present invention.

具体实施方式Detailed ways

为使本发明的目的、技术方案和优点更加清楚，下面将结合本发明中的附图，对本发明中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。In order to make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention. , not all examples. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

本发明提供了一种基于视觉感知的毫米波/太赫兹通信方法，参见图1，该方法包括：The present invention provides a millimeter wave/terahertz communication method based on visual perception, see FIG. 1 , the method includes:

S1、利用相机10实时采集图像信息以及移动端的深度信息；S1, use the camera 10 to collect image information and depth information of the mobile terminal in real time;

所述相机10可以设置在基站端20、智能反射面30或移动端40上，也可以是两个相机10分别设置在基站端20、智能反射面30、移动端40中的其中两端，或是三个相机10分别设置在基站端20、智能反射面30以及移动端40上，以上几种情况皆可实现，所述相机10用于实时采集图像信息；The camera 10 may be arranged on the base station 20, the smart reflection surface 30 or the mobile terminal 40, or two cameras 10 may be arranged at the two ends of the base station 20, the smart reflection surface 30, and the mobile terminal 40, respectively, or Three cameras 10 are respectively arranged on the base station end 20, the smart reflection surface 30 and the mobile end 40, the above situations can be realized, and the cameras 10 are used to collect image information in real time;

S2、利用所述智能反射面30根据所述图像信息识别出移动端40在图像中的位置；S2, using the intelligent reflective surface 30 to identify the position of the mobile terminal 40 in the image according to the image information;

根据图片识别移动端40在图像中的位置既可以通过基于HSV成像原理的传统图像处理技术，也可以通过深度神经网络的人工智能技术。此时，相机10设置在智能反射面30上。Recognizing the position of the mobile terminal 40 in the image according to the picture can be performed either through the traditional image processing technology based on the HSV imaging principle, or through the artificial intelligence technology of the deep neural network. At this time, the camera 10 is arranged on the smart reflection surface 30 .

S3、基于所述移动端的深度信息，将移动端40在图像中的位置转化为移动端40的三维坐标；S3, based on the depth information of the mobile terminal, convert the position of the mobile terminal 40 in the image into the three-dimensional coordinates of the mobile terminal 40;

S4、基于所述移动端40的三维坐标所计算出的移动端40所在实时位置的俯仰角和方位角，控制智能反射面30的天线将信号波束指向移动端40方向。S4 , controlling the antenna of the smart reflector 30 to point the signal beam in the direction of the mobile terminal 40 based on the pitch angle and azimuth angle of the real-time position of the mobile terminal 40 calculated based on the three-dimensional coordinates of the mobile terminal 40 .

所述智能反射面30的天线采用无源相控阵天线。The antenna of the smart reflecting surface 30 adopts a passive phased array antenna.

所述基于视觉感知的毫米波/太赫兹通信方法中需要使用的硬件部分包括相机10、多核服务器、由PXIe机箱、FPGA协处理器、时钟处理模块和软件定义无线电等设备构成的基站端20、连接基站的智能反射面30、智能反射面30和由无人小车、PC机以及连接PC机的软件定义无线电模块等设备构成的移动端40。所述相机10是指相机10及其上位机，相机10位于智能反射面30上，会实时采集图像信息。相机10连接的上位机根据图像，使用传统图像处理算法或深度学习算法识别出移动端40(或基站端20)在图像中的位置，再根据相机10获取移动端40的深度信息，使用相机10的内参矩阵将图像中的二维位置转化为空间中的三维坐标。相机10的上位机根据三维坐标计算出移动端40所在位置的俯仰角和方位角，将角度信息发送给智能反射面30，使波束指向该方向。当移动端40运动时，相机10及其上位机器能够计算出移动端40实时位置，控制智能反射面30随之改变反射方向，实现实时波束跟踪。The hardware parts that need to be used in the millimeter wave/terahertz communication method based on visual perception include a camera 10, a multi-core server, a base station 20 composed of a PXIe chassis, an FPGA co-processor, a clock processing module, a software-defined radio, and other devices. The smart reflective surface 30 connected to the base station, the smart reflective surface 30, and the mobile terminal 40 composed of devices such as an unmanned car, a PC, and a software-defined radio module connected to the PC. The camera 10 refers to the camera 10 and its host computer. The camera 10 is located on the smart reflective surface 30 and collects image information in real time. The host computer connected to the camera 10 identifies the position of the mobile terminal 40 (or the base station terminal 20 ) in the image according to the image, using a traditional image processing algorithm or a deep learning algorithm, and then obtains the depth information of the mobile terminal 40 according to the camera 10 , using the camera 10 The internal parameter matrix of transforms 2D positions in the image into 3D coordinates in space. The host computer of the camera 10 calculates the pitch angle and the azimuth angle of the position of the mobile terminal 40 according to the three-dimensional coordinates, and sends the angle information to the smart reflection surface 30 to direct the beam to this direction. When the mobile terminal 40 moves, the camera 10 and its host computer can calculate the real-time position of the mobile terminal 40 and control the intelligent reflecting surface 30 to change the reflection direction accordingly, so as to realize real-time beam tracking.

上述基站与移动端40的通信可以遵循4G-LTE、5G-NR协议或其它，软件可以使用LabVIEW、LabVIEW NXG或其它，频段可以选择任意毫米波和太赫兹等高频频段。相机10可以为双目相机10或其它，基站端20与移动端40的硬件组成可以上述硬件或其它，本方案不做限制。可以根据实际情况或需要进行合理配置。The communication between the base station and the mobile terminal 40 may follow the 4G-LTE, 5G-NR protocol or others, the software may use LabVIEW, LabVIEW NXG or others, and the frequency band may be any high frequency frequency band such as millimeter wave and terahertz. The camera 10 may be a binocular camera 10 or others, and the hardware components of the base station 20 and the mobile terminal 40 may be the above-mentioned hardware or others, which are not limited in this solution. It can be reasonably configured according to the actual situation or needs.

本发明提供的基于视觉感知的毫米波/太赫兹通信方法，使用相机10作为视觉辅助，可以快速识别移动端40并确定目标位置，并控制智能反射面30的相控阵天线产生对应方向的俯仰角和方位角，可以使波束更快指向对应目标，提高识别移动端40位置的能力，降低时延。当移动端40运动时，能够计算出移动端40实时位置，控制智能反射面30随之改变反射方向，实现实时波束跟踪。The millimeter wave/terahertz communication method based on visual perception provided by the present invention, using the camera 10 as a visual aid, can quickly identify the mobile terminal 40 and determine the target position, and control the phased array antenna of the smart reflector 30 to generate the pitch in the corresponding direction The angle and azimuth angle can make the beam point to the corresponding target faster, improve the ability to identify the position of the mobile terminal 40, and reduce the delay. When the mobile terminal 40 moves, the real-time position of the mobile terminal 40 can be calculated, and the intelligent reflecting surface 30 can be controlled to change the reflection direction accordingly, so as to realize real-time beam tracking.

上述智能反射面30设定俯仰角与方位角时参考的坐标系如图3所示。当站在智能反射面30后方看去，原点在智能反射面30的右上角顶点所在的位置，X轴向下，Y轴向左，Z轴向前。The coordinate system referenced by the smart reflective surface 30 for setting the pitch angle and the azimuth angle is shown in FIG. 3 . When looking from behind the smart reflective surface 30, the origin is at the position of the top right corner of the smart reflective surface 30, the X axis is downward, the Y axis is left, and the Z axis is forward.

在此坐标系下，θ为移动端40与原点的连线OQ与Z轴正向的夹角，

Figure BDA0003129350770000081
为移动端40在XOY平面的投影点与原点的连线OS与X轴正向的夹角。其中
Figure BDA0003129350770000082
在X轴左半边为正，在右半边为负。为便于描述与计算，规定以下几个主要的问题：In this coordinate system, θ is the angle between the connection line OQ between the moving end 40 and the origin and the positive direction of the Z axis,
Figure BDA0003129350770000081
is the angle between the line OS connecting the projection point of the mobile terminal 40 on the XOY plane and the origin and the positive X axis. in
Figure BDA0003129350770000082
It is positive on the left half of the x-axis and negative on the right half. For the convenience of description and calculation, the following main problems are specified:

相机10坐标原点在相机10表面的几何中心处；The origin of the coordinates of the camera 10 is at the geometric center of the surface of the camera 10;

智能反射面30的坐标原点近似地和相机10坐标原点相同；The coordinate origin of the smart reflective surface 30 is approximately the same as the coordinate origin of the camera 10;

坐标系都是右手系，且X/Y/Z坐标都是自带正负性的。移动端40不论在哪个坐标系中，Z坐标都大于0，而X坐标与Y坐标的正负性没有限制。The coordinate systems are all right-handed systems, and the X/Y/Z coordinates are all positive and negative. No matter which coordinate system the mobile terminal 40 is in, the Z coordinate is greater than 0, and the positive or negative of the X coordinate and the Y coordinate is not limited.

上述上位机将相机10拍摄的图片转化成的空间三维坐标如图2。当站在相机10后方看去，原点在相机10的右眼所在的位置，X轴向右，Y轴向下，Z轴向前。这是由相机10原始设定的，在此坐标系下得到移动端40位置的三维坐标。The above-mentioned upper computer converts the picture captured by the camera 10 into three-dimensional coordinates in space as shown in FIG. 2 . When looking from behind the camera 10, the origin is at the position of the right eye of the camera 10, the X axis is rightward, the Y axis is downward, and the Z axis is forward. This is originally set by the camera 10, and the three-dimensional coordinates of the position of the mobile terminal 40 are obtained in this coordinate system.

优选的，所述的基于视觉感知的毫米波/太赫兹通信方法，其中，所述相机10坐标原点在相机10表面的几何中心处，所述智能反射面30的坐标原点近似地和相机10坐标原点相同；Preferably, in the millimeter wave/terahertz communication method based on visual perception, the origin of the coordinates of the camera 10 is at the geometric center of the surface of the camera 10 , and the origin of the coordinates of the smart reflective surface 30 is approximately the coordinates of the camera 10 the origin is the same;

所述移动端40在相机10坐标系中的坐标为：The coordinates of the mobile terminal 40 in the coordinate system of the camera 10 are:

(XOQ,YOQ,ZOQ)(X OQ ,Y OQ ,Z OQ )

所述移动端40在智能反射面30坐标系中的坐标为：The coordinates of the mobile terminal 40 in the coordinate system of the smart reflective surface 30 are:

Figure BDA0003129350770000091
Figure BDA0003129350770000091

所述移动端40在相机10坐标系中以及在智能反射面30坐标系中的坐标的转换公式为：The conversion formula of the coordinates of the mobile terminal 40 in the coordinate system of the camera 10 and the coordinate system of the smart reflective surface 30 is:

Figure BDA0003129350770000092
Figure BDA0003129350770000092

为了避免不必要的计算，简化算法，下面所有的公式都只使用(XOQ,YOQ,ZOQ)作为已知量进行表示。In order to avoid unnecessary calculations and simplify the algorithm, all the following formulas are expressed only using (X OQ , Y OQ , Z OQ ) as known quantities.

所述移动端40的俯仰角θ由下式求得：The pitch angle θ of the mobile terminal 40 is obtained by the following formula:

Figure BDA0003129350770000093
Figure BDA0003129350770000093

所述移动端40的方位角

Figure BDA0003129350770000094
由下式求得：The azimuth of the mobile terminal 40
Figure BDA0003129350770000094
It is obtained by the following formula:

Figure BDA0003129350770000101
Figure BDA0003129350770000101

相机10的上位机按照以上公式求出θ与

Figure BDA0003129350770000102
然后，所述方法还包括：按照下式计算出所述智能反射面30的每根天线对应的反射波束的相角
Figure BDA0003129350770000103
The upper computer of the camera 10 calculates θ and
Figure BDA0003129350770000102
Then, the method further includes: calculating the phase angle of the reflected beam corresponding to each antenna of the smart reflective surface 30 according to the following formula
Figure BDA0003129350770000103

Figure BDA0003129350770000104
Figure BDA0003129350770000104

上式中，j为虚数单位，e为自然对数的底，M为矩阵行数，N为矩阵列数，λ为电磁波波长，d为天线间距。In the above formula, j is the imaginary unit, e is the base of the natural logarithm, M is the number of matrix rows, N is the number of matrix columns, λ is the wavelength of the electromagnetic wave, and d is the antenna spacing.

优选的，所述的基于视觉感知的毫米波/太赫兹通信方法，其中，所述方法还包括：Preferably, in the millimeter wave/terahertz communication method based on visual perception, the method further includes:

利用所述智能反射面30根据所述图像信息识别出基站端20在图像中的位置；Using the smart reflective surface 30 to identify the position of the base station 20 in the image according to the image information;

基于所述基站端的深度信息，将基站端20在图像中的位置转化为基站端20的三维坐标；Converting the position of the base station 20 in the image into the three-dimensional coordinates of the base station 20 based on the depth information of the base station;

基于所述基站端20的三维坐标所计算出的基站端20所在实时位置的俯仰角和方位角，对所述智能反射面30的天线进行入射角补偿。Based on the elevation angle and the azimuth angle of the real-time position of the base station terminal 20 calculated based on the three-dimensional coordinates of the base station terminal 20 , the incident angle compensation is performed on the antenna of the smart reflector 30 .

优选的，所述的基于视觉感知的毫米波/太赫兹通信方法，其中，所述基于所述基站端20的三维坐标所计算出的基站端20所在实时位置的俯仰角和方位角，包括：Preferably, in the millimeter wave/terahertz communication method based on visual perception, the pitch angle and azimuth angle of the real-time location of the base station terminal 20 calculated based on the three-dimensional coordinates of the base station terminal 20 include:

按照下式计算出所述智能反射面30的每根天线对应的入射角：Calculate the incident angle corresponding to each antenna of the smart reflective surface 30 according to the following formula:

Figure BDA0003129350770000105
Figure BDA0003129350770000105

上式中，

Figure BDA0003129350770000106
为基站端20俯仰角，θ0为基站端20的方位角，同上所述，M为矩阵行数，N为矩阵列数，λ为电磁波波长，d为天线间距。In the above formula,
Figure BDA0003129350770000106
is the pitch angle of the base station end 20, θ 0 is the azimuth angle of the base station end 20, as described above, M is the number of matrix rows, N is the number of matrix columns, λ is the wavelength of electromagnetic waves, and d is the antenna spacing.

在本实施例中，M＝N＝8，设中心频段为f＝30GHz，则λ＝10mm,d＝40mm。In this embodiment, M=N=8, and if the center frequency band is f=30GHz, then λ=10mm, d=40mm.

优选的，所述的基于视觉感知的毫米波/太赫兹通信方法，其中，所述控制智能反射面30的天线将信号波束指向移动端40方向之前，通过下式得到所述智能反射面30的天线的移相相位

Figure BDA0003129350770000111
Preferably, in the millimeter wave/terahertz communication method based on visual perception, before the antenna that controls the smart reflective surface 30 points the signal beam in the direction of the mobile terminal 40, the following formula is used to obtain the value of the smart reflective surface 30 phase shift of the antenna
Figure BDA0003129350770000111

Figure BDA0003129350770000112
Figure BDA0003129350770000112

其中，·表示点乘，即两个维度相同的矩阵对应位置的元素相乘，结果仍然为相同维度的矩阵。移相相位

Figure BDA0003129350770000113
的每个元素即为天线阵列上对应位置天线需要相位移相的相位。Among them, · represents point multiplication, that is, the elements of the corresponding positions of two matrices with the same dimension are multiplied, and the result is still a matrix of the same dimension. phase shift
Figure BDA0003129350770000113
Each element of is the phase of the antenna at the corresponding position on the antenna array that needs to be phase-shifted.

优选的，所述的基于视觉感知的毫米波/太赫兹通信方法，其中，所述方法还包括：Preferably, in the millimeter wave/terahertz communication method based on visual perception, the method further includes:

利用相机10所采集的图像识别移动端与智能反射面30的位置，此时相机10设置在基站端20；当基站端20与移动端40之间的直射径可达的情况下，基站端20根据相机10采集的图像判断移动端40的位置，再控制阵列天线将波束指向移动端40的位置，所输入的相角矩阵为

Figure BDA0003129350770000114
当直射径被遮挡时，基站端20根据相机10采集的图像判断智能反射面30的位置，再控制阵列天线将波束指向智能反射面30的位置，所输入的相角矩阵为
Figure BDA0003129350770000115
Use the images collected by the camera 10 to identify the positions of the mobile terminal and the smart reflective surface 30. At this time, the camera 10 is set at the base station terminal 20; when the direct radiation path between the base station terminal 20 and the mobile terminal 40 is reachable, the base station terminal 20 Determine the position of the mobile terminal 40 according to the image collected by the camera 10, and then control the array antenna to point the beam to the position of the mobile terminal 40. The input phase angle matrix is:
Figure BDA0003129350770000114
When the direct beam is blocked, the base station 20 determines the position of the smart reflective surface 30 according to the image collected by the camera 10, and then controls the array antenna to point the beam to the position of the smart reflective surface 30. The input phase angle matrix is:
Figure BDA0003129350770000115

优选的，所述的基于视觉感知的毫米波/太赫兹通信方法，其中，所述方法还包括：Preferably, in the millimeter wave/terahertz communication method based on visual perception, the method further includes:

利用相机10采集的图像识别基站端20与智能反射面30的位置，此时相机10设置在移动端40，当基站端20与移动端40之间的直射径可达的情况下，移动端40根据其相机10采集的图像判断基站端20位置，再控制阵列天线将波束指向基站端20的位置，所输入的相角矩阵为

Figure BDA0003129350770000116
当直射径被遮挡时，移动端40根据其相机采集的图像判断智能反射面30的位置，再控制阵列天线将波束指向智能反射面30的位置，所输入的相角矩阵为
Figure BDA0003129350770000121
The positions of the base station 20 and the smart reflective surface 30 are identified by the images collected by the camera 10. At this time, the camera 10 is set on the mobile terminal 40. When the direct radiation between the base station 20 and the mobile terminal 40 is reachable, the mobile terminal 40 Determine the position of the base station 20 according to the image collected by the camera 10, and then control the array antenna to point the beam to the position of the base station 20. The input phase angle matrix is:
Figure BDA0003129350770000116
When the direct beam is blocked, the mobile terminal 40 determines the position of the smart reflective surface 30 according to the image collected by its camera, and then controls the array antenna to point the beam to the position of the smart reflective surface 30. The input phase angle matrix is:
Figure BDA0003129350770000121

本发明还提供了一种基于视觉感知的毫米波/太赫兹通信装置，包括：相机10、基站端20、智能反射面30、移动端40；The present invention also provides a millimeter wave/terahertz communication device based on visual perception, comprising: a camera 10, a base station end 20, an intelligent reflection surface 30, and a mobile end 40;

所述相机10用于实时采集图像信息以及移动端的深度信息；The camera 10 is used to collect image information and depth information of the mobile terminal in real time;

此时，所述相机10设置在智能反射面30上；At this time, the camera 10 is arranged on the smart reflection surface 30;

所述智能反射面30根据所述图像信息识别出移动端40在图像中的位置；所述相机10还用于基于所述移动端的深度信息，将移动端40在图像中的位置转化为移动端40的三维坐标；The smart reflective surface 30 identifies the position of the mobile terminal 40 in the image according to the image information; the camera 10 is further configured to convert the position of the mobile terminal 40 in the image into the mobile terminal based on the depth information of the mobile terminal 3D coordinates of 40;

所述智能反射面30还用于基于所述移动端40的三维坐标所计算出的移动端40所在实时位置的俯仰角和方位角，控制智能反射面30的天线将信号波束指向移动端40方向。The smart reflective surface 30 is also used to control the antenna of the smart reflective surface 30 to point the signal beam to the direction of the mobile terminal 40 based on the elevation angle and azimuth angle of the real-time position of the mobile terminal 40 calculated based on the three-dimensional coordinates of the mobile terminal 40. .

参见图4，当所述相机10位于智能反射面30上时，所述智能反射面30根据所述图像信息识别出移动端40的实时位置、控制智能反射面30的天线将第二信号波束指向移动端40方向。Referring to FIG. 4 , when the camera 10 is located on the smart reflective surface 30, the smart reflective surface 30 recognizes the real-time position of the mobile terminal 40 according to the image information, and controls the antenna of the smart reflective surface 30 to point the second signal beam. Mobile terminal 40 direction.

本发明提供的基于视觉感知的毫米波/太赫兹通信装置，使用相机10作为视觉辅助，可以快速识别移动端40并确定目标位置，并控制智能反射面30的相控阵天线产生对应方向的俯仰角和方位角，可以使波束更快指向对应目标，提高识别移动端40位置的能力，降低时延。整个方案是感知通信一体化的具体实现和创新型应用。The millimeter wave/terahertz communication device based on visual perception provided by the present invention, using the camera 10 as a visual aid, can quickly identify the mobile terminal 40 and determine the target position, and control the phased array antenna of the smart reflector 30 to generate the pitch in the corresponding direction The angle and azimuth angle can make the beam point to the corresponding target faster, improve the ability to identify the position of the mobile terminal 40, and reduce the delay. The whole scheme is a concrete realization and innovative application of the integration of perception and communication.

优选的，所述的基于视觉感知的毫米波/太赫兹通信装置，其中，所述智能反射面30根据所述图像信息识别出基站端20在图像中的位置；所述相机10还用于基于所述基站端的深度信息，将基站端20在图像中的位置转化为基站端20的三维坐标；Preferably, in the millimeter wave/terahertz communication device based on visual perception, the intelligent reflective surface 30 identifies the position of the base station 20 in the image according to the image information; the camera 10 is also used for The depth information of the base station converts the position of the base station 20 in the image into the three-dimensional coordinates of the base station 20;

所述智能反射面30还用于基于所述基站端20的三维坐标所计算出的基站端20所在实时位置的俯仰角和方位角，对所述智能反射面30的天线进行入射角补偿。The smart reflective surface 30 is further configured to perform incident angle compensation on the antenna of the smart reflective surface 30 based on the elevation angle and azimuth angle of the real-time position of the base station 20 calculated based on the three-dimensional coordinates of the base station 20 .

需要指出，智能反射面30的入射角

Figure BDA0003129350770000122
一般情况下是固定的，也可以通过手动测量后输入至智能反射面30，或者使用相机10识别基站端20并判断其位置，进而计算出入射角。参见图4，当所述相机10位于智能反射面30上时，所述智能反射面30根据所述图像信息识别出基站端20的实时位置，对所述智能反射面30的天线进行入射角补偿。It should be pointed out that the incident angle of the smart reflective surface 30
Figure BDA0003129350770000122
Generally, it is fixed, and it can also be manually measured and then input to the smart reflective surface 30, or the camera 10 can be used to identify the base station 20 and determine its position, and then calculate the incident angle. Referring to FIG. 4 , when the camera 10 is located on the smart reflective surface 30 , the smart reflective surface 30 recognizes the real-time position of the base station 20 according to the image information, and performs incident angle compensation on the antenna of the smart reflective surface 30 .

相机10需要将基站端20发射至智能反射面30的波束的入射角补偿后，再将反射至移动端40的反射角度写入。相机10的上位机按照

Figure BDA0003129350770000131
写入智能反射面30。The camera 10 needs to write the reflection angle reflected to the mobile terminal 40 after compensating the incident angle of the beam emitted by the base station terminal 20 to the smart reflecting surface 30 . The host computer of the camera 10 follows the
Figure BDA0003129350770000131
Write to the smart reflective surface 30.

本方案在通信领域的波束成形与跟踪方向上比传统方法的应用场景更加广泛、时延更低，是感知通信一体化的具体实现与应用。Compared with the traditional method, this scheme has wider application scenarios and lower delay in the beamforming and tracking direction in the communication field, and is a specific realization and application of the integration of perception and communication.

优选的，所述的基于视觉感知的毫米波/太赫兹通信装置，其中，所述智能反射面30的天线采用无源相控阵天线，和/或，所述相机10具有计算物体左右距离及深度的功能，和/或，所述相机10为双目相机10。相机10也可以是其他具有计算物体左右距离及深度的功能的相机，作为优选，相机10拍照不受频段限制，因此该装置适用于任意毫米波和太赫兹等高频频段。Preferably, in the millimeter wave/terahertz communication device based on visual perception, the antenna of the smart reflector 30 adopts a passive phased array antenna, and/or the camera 10 has the ability to calculate the left and right distance of the object and Depth function, and/or, the camera 10 is a binocular camera 10 . The camera 10 can also be other cameras with the function of calculating the left and right distance and depth of the object. Preferably, the camera 10 is not limited by the frequency band for taking pictures, so the device is suitable for any high frequency frequency bands such as millimeter waves and terahertz.

优选的，所述的基于视觉感知的毫米波/太赫兹通信装置，其中，所述相机10坐标原点在相机10表面的几何中心处，所述智能反射面30的坐标原点近似地和相机10坐标原点相同。近似地和相机10坐标原点相同表示基本接近相同状态，因为很难达到完全相同，因此达到基本相同的程度即可。Preferably, in the millimeter wave/terahertz communication device based on visual perception, the origin of the coordinates of the camera 10 is at the geometric center of the surface of the camera 10 , and the origin of the coordinates of the smart reflective surface 30 is approximately the coordinates of the camera 10 The origin is the same. Approximately the same as the origin of the coordinates of the camera 10 means that it is basically close to the same state, because it is difficult to achieve the exact same state, so it is sufficient to achieve the same degree.

优选的，所述的基于视觉感知的毫米波/太赫兹通信装置，其中，所述移动端40的天线为全向天线或喇叭天线。Preferably, in the millimeter wave/terahertz communication device based on visual perception, the antenna of the mobile terminal 40 is an omnidirectional antenna or a horn antenna.

本发明还提供了一种基于视觉感知的毫米波/太赫兹通信系统，包括所述的基于视觉感知的毫米波/太赫兹通信装置，所述基于视觉感知的毫米波/太赫兹通信装置，包括：相机10、基站端20、智能反射面30、移动端40；The present invention also provides a millimeter wave/terahertz communication system based on visual perception, including the millimeter wave/terahertz communication device based on visual perception, the millimeter wave/terahertz communication device based on visual perception, including : camera 10, base station 20, smart reflective surface 30, mobile terminal 40;

所述相机10用于实时采集图像信息以及移动端的深度信息；The camera 10 is used to collect image information and depth information of the mobile terminal in real time;

此时，所述相机10设置在智能反射面30上；At this time, the camera 10 is arranged on the smart reflection surface 30;

所述智能反射面30根据所述图像信息识别出移动端40在图像中的位置；所述相机10还用于基于所述移动端的深度信息，将移动端40在图像中的位置转化为移动端40的三维坐标；The smart reflective surface 30 identifies the position of the mobile terminal 40 in the image according to the image information; the camera 10 is further configured to convert the position of the mobile terminal 40 in the image into the mobile terminal based on the depth information of the mobile terminal 3D coordinates of 40;

所述智能反射面30还用于基于所述移动端40的三维坐标所计算出的移动端40所在实时位置的俯仰角和方位角，控制智能反射面30的天线将信号波束指向移动端40方向。The smart reflective surface 30 is also used to control the antenna of the smart reflective surface 30 to point the signal beam to the direction of the mobile terminal 40 based on the elevation angle and azimuth angle of the real-time position of the mobile terminal 40 calculated based on the three-dimensional coordinates of the mobile terminal 40. .

以上所描述的装置实施例仅仅是示意性的，其中所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案的目的。本领域普通技术人员在不付出创造性的劳动的情况下，即可以理解并实施。The device embodiments described above are only illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in One place, or it can be distributed over multiple network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution in this embodiment. Those of ordinary skill in the art can understand and implement it without creative effort.

最后应说明的是：以上实施例仅用以说明本发明的技术方案，而非对其限制；尽管参照前述实施例对本发明进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本发明各实施例技术方案的精神和范围。Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, but not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that it can still be The technical solutions described in the foregoing embodiments are modified, or some technical features thereof are equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims (8)

1. A millimeter wave/terahertz communication method based on visual perception is characterized by comprising the following steps:
acquiring image information and depth information of a mobile terminal in real time by using a camera;
recognizing the position of the mobile terminal in an image according to the image information by using an intelligent reflecting surface, wherein the camera is arranged on the intelligent reflecting surface;
converting the position of the mobile terminal in the image into a three-dimensional coordinate of the mobile terminal based on the depth information of the mobile terminal;
controlling an antenna of the intelligent reflecting surface to point a signal beam to the direction of the moving end based on the pitch angle and the azimuth angle of the real-time position of the moving end calculated by the three-dimensional coordinates of the moving end, wherein,
the camera coordinate origin is at the geometric center of the camera surface, and the coordinate origin of the intelligent reflecting surface is approximately the same as the camera coordinate origin;
the coordinates of the mobile terminal in the camera coordinate system are as follows:
(XOQ,YOQ,ZOQ)
the coordinate of the mobile terminal in the intelligent reflecting surface coordinate system is as follows:
Figure FDA0003510439370000011
the conversion formula of the coordinates of the mobile terminal in the camera coordinate system and the intelligent reflecting surface coordinate system is as follows:
Figure FDA0003510439370000012
the pitch angle θ of the moving end is obtained by the following formula:
Figure FDA0003510439370000013
azimuth angle of the moving end
Figure FDA0003510439370000024
The following equation is obtained:
Figure FDA0003510439370000021
the method further comprises the following steps: calculating the phase angle of the reflected wave beam corresponding to each antenna of the intelligent reflecting surface according to the following formula
Figure FDA0003510439370000022
Figure FDA0003510439370000023
In the above formula, j is an imaginary unit, e is the base of the natural logarithm, M is the number of rows of the matrix, N is the number of columns of the matrix, λ is the wavelength of the electromagnetic wave, and d is the antenna pitch.
2. The millimeter wave/terahertz communication method based on visual perception according to claim 1, further comprising:
acquiring depth information of a base station end in real time by using a camera;
identifying the position of the base station end in the image according to the image information by using the intelligent reflecting surface;
converting the position of the base station end in the image into a three-dimensional coordinate of the base station end based on the depth information of the base station end;
and performing incident angle compensation on the antenna of the intelligent reflecting surface based on the pitch angle and the azimuth angle of the real-time position of the base station end calculated by the three-dimensional coordinates of the base station end.
3. The millimeter wave/terahertz communication method based on visual perception according to claim 2, wherein the pitch angle and the azimuth angle of the real-time position of the base station end calculated based on the three-dimensional coordinates of the base station end comprise:
calculating the incident angle corresponding to each antenna of the intelligent reflecting surface according to the following formula:
Figure FDA0003510439370000031
in the above formula, the first and second carbon atoms are,
Figure FDA0003510439370000032
at base station end pitch angle, θ0Is the azimuth of the base station side.
4. The millimeter wave/terahertz communication method based on visual perception according to claim 3, wherein the antenna for controlling the intelligent reflecting surface is controlled to direct the signal beam to the direction of the moving end by the following formulaObtaining the phase shift of the antenna of the intelligent reflecting surface
Figure FDA0003510439370000033
Figure FDA0003510439370000034
Where, is, the dot multiplication, i.e. the multiplication of elements at corresponding positions of two matrices with the same dimension.
5. The millimeter wave/terahertz communication method based on visual perception according to claim 4, wherein the method further comprises:
recognizing the positions of the moving end and the intelligent reflecting surface by using the image acquired by the camera; when the direct beam path between the base station end and the mobile end can be reached, the base station end judges the position of the mobile end according to the image collected by the camera, and then controls the array antenna to point the beam to the position of the mobile end, and the input phase angle matrix is
Figure FDA0003510439370000035
When the direct radiation path is blocked, the base station end judges the position of the intelligent reflecting surface according to the image collected by the camera, and then controls the array antenna to point the wave beam to the position of the intelligent reflecting surface, and the input phase angle matrix is
Figure FDA0003510439370000036
6. The millimeter wave/terahertz communication method based on visual perception according to claim 4 or 5, wherein the method further comprises:
the position of the base station end and the position of the intelligent reflecting surface are identified by using the image acquired by the camera, when the direct beam path between the base station end and the mobile end can be reached, the mobile end judges the position of the base station end according to the image acquired by the camera, then the array antenna is controlled to point the wave beam to the position of the base station end, and the input wave beam points to the position of the base station endIs a phase angle matrix of
Figure FDA0003510439370000037
When the direct radiation path is blocked, the mobile terminal judges the position of the intelligent reflecting surface according to the image collected by the camera, and then controls the array antenna to point the wave beam to the position of the intelligent reflecting surface, and the input phase angle matrix is
Figure FDA0003510439370000038
7. A millimeter wave/terahertz communication device based on visual perception is characterized by comprising: the system comprises a camera, a base station end, an intelligent reflecting surface and a moving end, wherein the camera is arranged on the intelligent reflecting surface;
the camera is used for acquiring image information and depth information of the mobile terminal in real time;
the intelligent reflecting surface identifies the position of the mobile terminal in the image according to the image information; the camera is also used for converting the position of the mobile terminal in the image into a three-dimensional coordinate of the mobile terminal based on the depth information of the mobile terminal;
the intelligent reflecting surface is also used for controlling an antenna of the intelligent reflecting surface to point a signal beam to the direction of the mobile terminal based on a pitch angle and an azimuth angle of the real-time position of the mobile terminal calculated by the three-dimensional coordinate of the mobile terminal;
the intelligent reflecting surface identifies the position of the base station end in the image according to the image information; the camera is also used for converting the position of the base station end in the image into a three-dimensional coordinate of the base station end based on the depth information of the base station end;
and the intelligent reflecting surface is also used for compensating the incident angle of the antenna of the intelligent reflecting surface based on the pitch angle and the azimuth angle of the real-time position of the base station end calculated by the three-dimensional coordinates of the base station end.
8. A millimeter wave/terahertz communication system based on visual perception, characterized by comprising the millimeter wave/terahertz communication device based on visual perception according to claim 7.
CN202110698104.4A 2021-06-23 2021-06-23 Millimeter wave/terahertz communication method, device and system based on visual perception Active CN113595608B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110698104.4A CN113595608B (en) 2021-06-23 2021-06-23 Millimeter wave/terahertz communication method, device and system based on visual perception

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110698104.4A CN113595608B (en) 2021-06-23 2021-06-23 Millimeter wave/terahertz communication method, device and system based on visual perception

Publications (2)

Publication Number Publication Date
CN113595608A CN113595608A (en) 2021-11-02
CN113595608B true CN113595608B (en) 2022-04-12

Family

ID=78244366

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110698104.4A Active CN113595608B (en) 2021-06-23 2021-06-23 Millimeter wave/terahertz communication method, device and system based on visual perception

Country Status (1)

Country Link
CN (1) CN113595608B (en)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114520980B (en) * 2022-01-12 2024-11-05 山西鑫鹏益电子科技有限公司 A coal mine communication network model optimization method and device
CN118489262A (en) * 2022-01-13 2024-08-13 华为技术有限公司 Beam control method and device
CN114828128B (en) * 2022-02-25 2023-08-22 北京邮电大学 Multi-cell communication coverage method and related equipment for high-speed mobile train
CN114845332A (en) * 2022-05-05 2022-08-02 东南大学 Millimeter wave communication link blocking prediction method based on visual information fusion
CN116488740B (en) * 2023-04-17 2023-11-07 军事科学院系统工程研究院网络信息研究所 Space terahertz communication load system

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102374860A (en) * 2011-09-23 2012-03-14 奇瑞汽车股份有限公司 Three-dimensional visual positioning method and system
CN107579759A (en) * 2017-09-19 2018-01-12 清华大学 Method and device for stabilizing antenna beam in UAV satellite communication system
CN111245494A (en) * 2020-01-13 2020-06-05 东南大学 Positioning Information Assisted Beam Steering Method Based on Smart Reflector
CN112994765A (en) * 2021-03-04 2021-06-18 电子科技大学 Wave beam alignment method of intelligent reflecting surface auxiliary millimeter wave communication system

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104567758B (en) * 2013-10-29 2017-11-17 同方威视技术股份有限公司 Stereo imaging system and its method
CN110411375B (en) * 2019-08-05 2021-05-11 上海亨临光电科技有限公司 Three-dimensional imaging method based on passive millimeter wave/terahertz imaging technology

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102374860A (en) * 2011-09-23 2012-03-14 奇瑞汽车股份有限公司 Three-dimensional visual positioning method and system
CN107579759A (en) * 2017-09-19 2018-01-12 清华大学 Method and device for stabilizing antenna beam in UAV satellite communication system
CN111245494A (en) * 2020-01-13 2020-06-05 东南大学 Positioning Information Assisted Beam Steering Method Based on Smart Reflector
CN112994765A (en) * 2021-03-04 2021-06-18 电子科技大学 Wave beam alignment method of intelligent reflecting surface auxiliary millimeter wave communication system

Also Published As

Publication number Publication date
CN113595608A (en) 2021-11-02

Similar Documents

Publication Publication Date Title
CN113595608B (en) Millimeter wave/terahertz communication method, device and system based on visual perception
WO2022117011A1 (en) Control method and apparatus for laser radar, storage medium, and electronic apparatus
CN113298947A (en) Multi-source data fusion-based three-dimensional modeling method medium and system for transformer substation
CN114079866B (en) Signal transmission method, equipment and device
CN104038707B (en) A Portable Terahertz Passive Color Camera
CN110297213A (en) Radiation source positioning device and method based on the unmanned aerial vehicle platform for loading relatively prime linear array
CN108924408B (en) Depth imaging method and system
CN110632670B (en) A fast imaging method for multi-frequency reconfigurable electromagnetic surfaces
CN109946748A (en) Thinned array antenna design method applied to cylinder scanning safety check imaging system
CN114740472B (en) A forward-looking three-dimensional imaging method and system for non-scanning single-channel terahertz radar
CN110794471A (en) Millimeter wave sparse array remote monitoring imaging method and system
CN106452541A (en) Light-wireless signal mutually assisting beamforming method and apparatus
CN115278526A (en) Terminal positioning method and device, electronic equipment and storage medium
EP3394633B1 (en) Device in a car for communicating with at least one neighboring device, corresponding method and computer program product.
CN109597076B (en) Data processing method and device for ground-based synthetic aperture radar
CN110286351A (en) A two-dimensional DOA estimation method and device based on L-shaped nested matrix
WO2024109161A1 (en) Unmanned aerial vehicle applied to power grid inspection, method, and power grid inspection system
CN110736986A (en) Intelligent Wi-Fi imaging method and system based on field programmable metamaterial
CN104242996B (en) A kind of method, system and device for establishing communication link
CN215835399U (en) Millimeter wave/terahertz communication device and system based on visual perception
EP3482562B1 (en) An apparatus and a method for generating data representative of a pixel beam
CN109884622A (en) Three-dimensional imaging method of cylindrical array antenna
CN116528172A (en) Anti-jamming method for civil aviation VHF ground-to-air communication by fusing secondary radar information
CN114499615A (en) A unified transmit beamforming method for near and far fields in terahertz communication systems
CN110873859A (en) Method, device, medium and equipment for determining transmitting and receiving end positions based on directional antenna

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant