CN113595608A - Millimeter wave/terahertz communication method, device and system based on visual perception - Google Patents

Millimeter wave/terahertz communication method, device and system based on visual perception Download PDF

Info

Publication number
CN113595608A
CN113595608A CN202110698104.4A CN202110698104A CN113595608A CN 113595608 A CN113595608 A CN 113595608A CN 202110698104 A CN202110698104 A CN 202110698104A CN 113595608 A CN113595608 A CN 113595608A
Authority
CN
China
Prior art keywords
mobile terminal
base station
reflecting surface
camera
intelligent reflecting
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110698104.4A
Other languages
Chinese (zh)
Other versions
CN113595608B (en
Inventor
高飞飞
张腾宇
张�浩
王鸣锦
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tsinghua University
Original Assignee
Tsinghua University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tsinghua University filed Critical Tsinghua University
Priority to CN202110698104.4A priority Critical patent/CN113595608B/en
Publication of CN113595608A publication Critical patent/CN113595608A/en
Application granted granted Critical
Publication of CN113595608B publication Critical patent/CN113595608B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04B—TRANSMISSION
    • H04B7/00—Radio transmission systems, i.e. using radiation field
    • H04B7/02—Diversity systems; Multi-antenna system, i.e. transmission or reception using multiple antennas
    • H04B7/04—Diversity systems; Multi-antenna system, i.e. transmission or reception using multiple antennas using two or more spaced independent antennas
    • H04B7/06—Diversity systems; Multi-antenna system, i.e. transmission or reception using multiple antennas using two or more spaced independent antennas at the transmitting station
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04B—TRANSMISSION
    • H04B7/00—Radio transmission systems, i.e. using radiation field
    • H04B7/02—Diversity systems; Multi-antenna system, i.e. transmission or reception using multiple antennas
    • H04B7/04—Diversity systems; Multi-antenna system, i.e. transmission or reception using multiple antennas using two or more spaced independent antennas
    • H04B7/04013—Intelligent reflective surfaces
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/02—Systems using reflection of radio waves, e.g. primary radar systems; Analogous systems
    • G01S13/06—Systems determining position data of a target
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S5/00—Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations
    • G01S5/02—Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations using radio waves
    • G01S5/08—Position of single direction-finder fixed by determining direction of a plurality of spaced sources of known location
    • H—ELECTRICITY
    • H01—ELECTRIC ELEMENTS
    • H01Q—ANTENNAS, i.e. RADIO AERIALS
    • H01Q3/00—Arrangements for changing or varying the orientation or the shape of the directional pattern of the waves radiated from an antenna or antenna system
    • H01Q3/26—Arrangements for changing or varying the orientation or the shape of the directional pattern of the waves radiated from an antenna or antenna system varying the relative phase or relative amplitude of energisation between two or more active radiating elements; varying the distribution of energy across a radiating aperture
    • H01Q3/30—Arrangements for changing or varying the orientation or the shape of the directional pattern of the waves radiated from an antenna or antenna system varying the relative phase or relative amplitude of energisation between two or more active radiating elements; varying the distribution of energy across a radiating aperture varying the relative phase between the radiating elements of an array
    • H—ELECTRICITY
    • H01—ELECTRIC ELEMENTS
    • H01Q—ANTENNAS, i.e. RADIO AERIALS
    • H01Q3/00—Arrangements for changing or varying the orientation or the shape of the directional pattern of the waves radiated from an antenna or antenna system
    • H01Q3/26—Arrangements for changing or varying the orientation or the shape of the directional pattern of the waves radiated from an antenna or antenna system varying the relative phase or relative amplitude of energisation between two or more active radiating elements; varying the distribution of energy across a radiating aperture
    • H01Q3/30—Arrangements for changing or varying the orientation or the shape of the directional pattern of the waves radiated from an antenna or antenna system varying the relative phase or relative amplitude of energisation between two or more active radiating elements; varying the distribution of energy across a radiating aperture varying the relative phase between the radiating elements of an array
    • H01Q3/32—Arrangements for changing or varying the orientation or the shape of the directional pattern of the waves radiated from an antenna or antenna system varying the relative phase or relative amplitude of energisation between two or more active radiating elements; varying the distribution of energy across a radiating aperture varying the relative phase between the radiating elements of an array by mechanical means
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04B—TRANSMISSION
    • H04B7/00—Radio transmission systems, i.e. using radiation field
    • H04B7/02—Diversity systems; Multi-antenna system, i.e. transmission or reception using multiple antennas
    • H04B7/04—Diversity systems; Multi-antenna system, i.e. transmission or reception using multiple antennas using two or more spaced independent antennas
    • H04B7/06—Diversity systems; Multi-antenna system, i.e. transmission or reception using multiple antennas using two or more spaced independent antennas at the transmitting station
    • H04B7/0613—Diversity systems; Multi-antenna system, i.e. transmission or reception using multiple antennas using two or more spaced independent antennas at the transmitting station using simultaneous transmission
    • H04B7/0615—Diversity systems; Multi-antenna system, i.e. transmission or reception using multiple antennas using two or more spaced independent antennas at the transmitting station using simultaneous transmission of weighted versions of same signal
    • H04B7/0617—Diversity systems; Multi-antenna system, i.e. transmission or reception using multiple antennas using two or more spaced independent antennas at the transmitting station using simultaneous transmission of weighted versions of same signal for beam forming

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Mobile Radio Communication Systems (AREA)
  • Radar Systems Or Details Thereof (AREA)

Abstract

The invention provides a millimeter wave/terahertz communication method, device and system based on visual perception, wherein the communication method comprises the following steps: acquiring image information and depth information of a mobile terminal in real time by using a camera; identifying the position of the mobile terminal in the image according to the image information by using the intelligent reflecting surface; converting the position of the mobile terminal in the image into a three-dimensional coordinate of the mobile terminal based on the depth information of the mobile terminal; and controlling an antenna of the intelligent reflecting surface to point the signal beam to the direction of the mobile terminal based on the pitch angle and the azimuth angle of the real-time position of the mobile terminal calculated by the three-dimensional coordinate of the mobile terminal. The invention can quickly identify the moving end and determine the position of the target, and control the phased array antenna of the intelligent reflecting surface to generate the pitch angle and the azimuth angle in the corresponding direction, so that the beam can be more quickly pointed to the corresponding target, the capacity of identifying the position of the moving end is improved, the time delay is reduced, and the real-time beam tracking is realized.

Description

Millimeter wave/terahertz communication method, device and system based on visual perception
Technical Field
The invention relates to the technical field of communication, in particular to a millimeter wave/terahertz communication method, device and system based on visual perception.
Background
Since birth, the communication technology industry has been following the moore's law of ' ten years and one generation ', and the whole mobile communication is developing in a direction of becoming faster, more stable and safer from the analog communication mode of the first generation mobile communication (1G) beginning in 1980, to the 2G taking TDMA as the core technology in about 1990, to the 3G age taking CDMA as the core technology in the beginning of the century, to the large-scale commercial use of the 4G mobile communication network taking OFDM as the core technology in about 2010. In 2020, the fifth generation mobile communication technology (5G) is called "5G yuan year" because of its advent, and has a good performance in scenarios such as enhanced mobile bandwidth communication (eMBB), ultra-reliable low-latency communication (URLLC), and large-scale machine type communication (mtc).
At present, the beam forming and tracking of the intelligent reflecting surface in the traditional high-frequency communication field basically adopts the following modes: pilot scanning and traversing all phases in the direction of the intelligent reflecting surface are used firstly to determine the direction of the moving end. When the mobile terminal moves, the traditional method needs to continuously send the signal-to-noise ratio near the direction of the pilot frequency scanning mobile terminal, the mobile terminal needs to feed back the value of the signal-to-noise ratio to the intelligent reflecting surface through an uplink channel, and then the intelligent reflecting surface selects the signal-to-noise ratio in the largest direction as the direction of the reflected wave beam; in the field of radar phased arrays, channels are mainly divided into communication channels and beacon channels, and the channel values of the beacon channels are analyzed by a difference power division network through a wave arrival algorithm, so that the signal source direction is judged.
The system has a large degree of dependence on the hardware of the intelligent reflecting surface, and when the energy of a scanning signal is weak or the energy of a beacon signal is difficult to detect, the direction of a mobile terminal is inconvenient to determine. And the pilot occupies more channel resources. When the number of mobile terminals is large, the calculation complexity also rises sharply, and the time delay is large.
Disclosure of Invention
The invention provides a millimeter wave/terahertz communication method, a device and a system based on visual perception, which are used for solving the technical defects in the prior art, realizing the purpose of judging the position of a target by using a spectrum, tracking the target and saving channel resources occupied by pilot frequency; when the number of the mobile terminals is large, the calculation complexity is not changed, and the time delay is small.
The invention provides a millimeter wave/terahertz communication method based on visual perception, which comprises the following steps:
acquiring image information and depth information of a mobile terminal in real time by using a camera; the camera is arranged on the intelligent reflecting surface;
identifying the position of the mobile terminal in the image according to the image information by using the intelligent reflecting surface;
converting the position of the mobile terminal in the image into a three-dimensional coordinate of the mobile terminal based on the depth information of the mobile terminal;
and controlling an antenna of the intelligent reflecting surface to point the signal beam to the direction of the mobile terminal based on the pitch angle and the azimuth angle of the real-time position of the mobile terminal calculated by the three-dimensional coordinate of the mobile terminal.
Preferably, the millimeter wave/terahertz communication method based on visual perception is characterized in that the origin of coordinates of the camera is at the geometric center of the camera surface, and the origin of coordinates of the intelligent reflecting surface is approximately the same as the origin of coordinates of the camera;
the coordinates of the mobile terminal in the camera coordinate system are as follows:
(XOQ,YOQ,ZOQ)
the coordinate of the mobile terminal in the intelligent reflecting surface coordinate system is as follows:
Figure BDA0003129350770000021
the conversion formula of the coordinates of the mobile terminal in the camera coordinate system and the intelligent reflecting surface coordinate system is as follows:
Figure BDA0003129350770000031
the pitch angle θ of the moving end is obtained by the following formula:
Figure BDA0003129350770000032
azimuth angle of the moving end
Figure BDA0003129350770000033
The following equation is obtained:
Figure BDA0003129350770000034
the method further comprises the following steps: calculating the phase angle of the reflected wave beam corresponding to each antenna of the intelligent reflecting surface according to the following formula
Figure BDA0003129350770000035
Figure BDA0003129350770000036
In the above formula, j is an imaginary unit, e is the base of the natural logarithm, M is the number of rows of the matrix, N is the number of columns of the matrix, λ is the wavelength of the electromagnetic wave, and d is the antenna pitch.
Preferably, the millimeter wave/terahertz communication method based on visual perception further includes:
identifying the position of the base station end in the image according to the image information by using the intelligent reflecting surface;
converting the position of the base station end in the image into a three-dimensional coordinate of the base station end based on the depth information of the base station end;
and performing incident angle compensation on the antenna of the intelligent reflecting surface based on the pitch angle and the azimuth angle of the real-time position of the base station end calculated by the three-dimensional coordinates of the base station end.
Preferably, the millimeter wave/terahertz communication method based on visual perception, wherein the pitch angle and the azimuth angle of the real-time position of the base station end, which are calculated based on the three-dimensional coordinates of the base station end, include:
calculating the incident angle corresponding to each antenna of the intelligent reflecting surface according to the following formula:
Figure BDA0003129350770000041
in the above formula, the first and second carbon atoms are,
Figure BDA0003129350770000042
at base station end pitch angle, θ0Is the azimuth of the base station side.
Preferably, in the millimeter wave/terahertz communication method based on visual perception, before the antenna controlling the intelligent reflecting surface directs the signal beam to the direction of the moving end, the phase shift phase of the antenna of the intelligent reflecting surface is obtained by the following formula
Figure BDA0003129350770000043
Figure BDA0003129350770000044
Where, is, the dot multiplication, i.e. the multiplication of elements at corresponding positions of two matrices with the same dimension.
Preferably, the millimeter wave/terahertz communication method based on visual perception further includes:
recognizing the positions of the moving end and the intelligent reflecting surface by using the image acquired by the camera; when the direct beam path between the base station end and the mobile end can be reached, the base station end judges the position of the mobile end according to the image collected by the camera, and then controls the array antenna to point the beam to the position of the mobile end, and the input phase angle matrix is
Figure BDA0003129350770000045
When the direct radiation path is blocked, the base station end judges the position of the intelligent reflecting surface according to the image collected by the camera, and then controls the array antenna to point the wave beam to the position of the intelligent reflecting surface, and the input phase angle matrix is
Figure BDA0003129350770000046
Preferably, the millimeter wave/terahertz communication method based on visual perception further includes:
the position of the base station end and the position of the intelligent reflecting surface are identified by using the image acquired by the camera, when the direct beam path between the base station end and the mobile end can be reached, the mobile end judges the position of the base station end according to the image acquired by the camera, then the array antenna is controlled to point the wave beam to the position of the base station end, and the input phase angle matrix is
Figure BDA0003129350770000047
When the direct radiation path is blocked, the mobile terminal judges the position of the intelligent reflecting surface according to the image collected by the camera of the mobile terminal, and then controls the array antenna to point the wave beam to the position of the intelligent reflecting surface, and the input phase angle matrix is
Figure BDA0003129350770000051
The invention also provides a millimeter wave/terahertz communication device based on visual perception, which comprises: the system comprises a camera, a base station end, an intelligent reflecting surface and a mobile end;
the camera is used for acquiring image information and depth information of the mobile terminal in real time; the camera is arranged on the intelligent reflecting surface;
the intelligent reflecting surface identifies the position of the mobile terminal in the image according to the image information; the camera is also used for converting the position of the mobile terminal in the image into a three-dimensional coordinate of the mobile terminal based on the depth information of the mobile terminal;
the intelligent reflecting surface is also used for controlling an antenna of the intelligent reflecting surface to point the signal beam to the direction of the moving end based on the pitch angle and the azimuth angle of the real-time position of the moving end calculated by the three-dimensional coordinate of the moving end.
Preferably, the millimeter wave/terahertz communication device based on visual perception is characterized in that the intelligent reflecting surface identifies the position of the base station end in the image according to the image information; the camera is also used for converting the position of the base station end in the image into a three-dimensional coordinate of the base station end based on the depth information of the base station end;
and the intelligent reflecting surface is also used for compensating the incident angle of the antenna of the intelligent reflecting surface based on the pitch angle and the azimuth angle of the real-time position of the base station end calculated by the three-dimensional coordinates of the base station end.
Preferably, the millimeter wave/terahertz communication device based on visual perception is characterized in that the antenna of the intelligent reflecting surface is a passive phased array antenna, and/or the camera has a function of calculating the left-right distance and the depth of an object, and/or the camera is a binocular camera.
Preferably, the millimeter wave/terahertz communication device based on visual perception is characterized in that the camera coordinate origin is at the geometric center of the camera surface, and the coordinate origin of the intelligent reflecting surface is approximately the same as the camera coordinate origin.
The invention also provides a millimeter wave/terahertz communication system based on visual perception, which comprises the millimeter wave/terahertz communication device based on visual perception.
According to the millimeter wave/terahertz communication method, device and system based on visual perception, provided by the invention, the camera is used as visual assistance, the mobile terminal can be rapidly identified, the position of the target can be determined, the phased array antenna of the intelligent reflecting surface is controlled to generate the pitch angle and the azimuth angle in the corresponding direction, the wave beam can be enabled to be more rapidly directed to the corresponding target, the capability of identifying the position of the mobile terminal is improved, and the time delay is reduced. When the mobile terminal moves, the real-time position of the mobile terminal can be calculated, the intelligent reflecting surface is controlled to change the reflecting direction, and real-time beam tracking is realized.
Drawings
In order to more clearly illustrate the technical solutions of the present invention or the prior art, the drawings needed for the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.
FIG. 1 is a schematic flow chart of a millimeter wave/terahertz communication method based on visual perception provided by the invention;
FIG. 2 is a schematic diagram of spatial three-dimensional coordinates into which a picture taken by a camera provided by the present invention is translated;
fig. 3 is a schematic diagram of a coordinate system for reference when the intelligent reflecting surface is set to the pitch angle and the azimuth angle.
Fig. 4 is a schematic structural diagram of a third millimeter wave/terahertz communication device based on visual perception provided by the invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The invention provides a millimeter wave/terahertz communication method based on visual perception, and referring to fig. 1, the method comprises the following steps:
s1, acquiring image information and depth information of the mobile terminal in real time by using the camera 10;
the cameras 10 may be disposed on the base station end 20, the intelligent reflective surface 30 or the mobile end 40, or two cameras 10 may be disposed on two ends of the base station end 20, the intelligent reflective surface 30 or the mobile end 40, or three cameras 10 may be disposed on the base station end 20, the intelligent reflective surface 30 or the mobile end 40, respectively, which may be implemented, where the cameras 10 are configured to collect image information in real time;
s2, recognizing the position of the mobile terminal 40 in the image according to the image information by using the intelligent reflecting surface 30;
the position of the mobile terminal 40 in the image can be identified according to the picture by using a traditional image processing technology based on the HSV imaging principle and an artificial intelligence technology of a deep neural network. At this time, the camera 10 is disposed on the smart reflective surface 30.
S3, converting the position of the mobile terminal 40 in the image into the three-dimensional coordinate of the mobile terminal 40 based on the depth information of the mobile terminal;
and S4, controlling the antenna of the intelligent reflecting surface 30 to point the signal beam to the direction of the mobile terminal 40 according to the pitch angle and the azimuth angle of the real-time position of the mobile terminal 40 calculated based on the three-dimensional coordinates of the mobile terminal 40.
The antenna of the intelligent reflecting surface 30 is a passive phased array antenna.
The hardware part required to be used in the millimeter wave/terahertz communication method based on visual perception comprises a camera 10, a multi-core server, a base station end 20 composed of a PXIe case, an FPGA coprocessor, a clock processing module, software defined radio and other equipment, an intelligent reflecting surface 30 and an intelligent reflecting surface 30 connected with the base station, and a mobile end 40 composed of unmanned vehicles, PCs, software defined radio modules connected with the PCs and other equipment. The camera 10 refers to the camera 10 and an upper computer thereof, and the camera 10 is located on the intelligent reflecting surface 30 and can acquire image information in real time. The upper computer connected to the camera 10 recognizes the position of the mobile terminal 40 (or the base station terminal 20) in the image according to the image by using a conventional image processing algorithm or a depth learning algorithm, obtains depth information of the mobile terminal 40 according to the camera 10, and converts a two-dimensional position in the image into a three-dimensional coordinate in a space by using an internal reference matrix of the camera 10. The upper computer of the camera 10 calculates the pitch angle and the azimuth angle of the position of the moving end 40 according to the three-dimensional coordinates, and sends the angle information to the intelligent reflection surface 30, so that the beam points to the direction. When the mobile terminal 40 moves, the camera 10 and the host machine thereof can calculate the real-time position of the mobile terminal 40, control the intelligent reflective surface 30 to change the reflective direction, and realize real-time beam tracking.
The communication between the base station and the mobile terminal 40 may follow 4G-LTE, 5G-NR protocols or others, the software may use LabVIEW, LabVIEW NXG or others, and the frequency band may be any high-frequency band such as millimeter wave and terahertz. The camera 10 may be a binocular camera 10 or others, and the hardware composition of the base station side 20 and the mobile terminal 40 may be the above hardware or others, and the present solution is not limited. Can be configured reasonably according to actual conditions or needs.
According to the millimeter wave/terahertz communication method based on visual perception, provided by the invention, the camera 10 is used as visual assistance, the mobile terminal 40 can be rapidly identified and the target position can be determined, and the phased array antenna of the intelligent reflecting surface 30 is controlled to generate the pitch angle and the azimuth angle in the corresponding direction, so that the wave beam can be more rapidly directed to the corresponding target, the capacity of identifying the position of the mobile terminal 40 is improved, and the time delay is reduced. When the mobile terminal 40 moves, the real-time position of the mobile terminal 40 can be calculated, and the intelligent reflecting surface 30 is controlled to change the reflecting direction, so that real-time beam tracking is realized.
The coordinate system referred to when the intelligent reflection surface 30 sets the pitch angle and the azimuth angle is shown in fig. 3. When looking after the intelligent reflecting surface 30, the origin is at the top right vertex of the intelligent reflecting surface 30, the X axis is downward, the Y axis is leftward, and the Z axis is forward.
In the coordinate system, θ is the included angle between the line OQ between the moving end 40 and the origin and the positive direction of the Z axis,
Figure BDA0003129350770000081
is the included angle between the connecting line OS of the projection point of the mobile terminal 40 on the XOY plane and the origin and the positive direction of the X axis. Wherein
Figure BDA0003129350770000082
Positive in the left half and negative in the right half of the X-axis. For ease of description and calculation, the following major issues are specified:
the camera 10 origin of coordinates is at the geometric center of the camera 10 surface;
the origin of coordinates of the intelligent reflecting surface 30 is approximately the same as the origin of coordinates of the camera 10;
the coordinate systems are all right-handed systems, and the X/Y/Z coordinates are all positive and negative. The moving tip 40 has a Z coordinate greater than 0 in any coordinate system, and the positive and negative of the X coordinate and the Y coordinate are not limited.
The upper computer converts the pictures taken by the camera 10 into three-dimensional space coordinates as shown in fig. 2. When viewed from behind the camera 10, the origin is at the location of the right eye of the camera 10, with the X-axis to the right, the Y-axis down, and the Z-axis forward. This is originally set by the camera 10, and three-dimensional coordinates of the position of the moving end 40 are obtained in this coordinate system.
Preferably, the millimeter wave/terahertz communication method based on visual perception is characterized in that the origin of coordinates of the camera 10 is at the geometric center of the surface of the camera 10, and the origin of coordinates of the intelligent reflecting surface 30 is approximately the same as the origin of coordinates of the camera 10;
the coordinates of the moving end 40 in the coordinate system of the camera 10 are:
(XOQ,YOQ,ZOQ)
the coordinates of the moving end 40 in the coordinate system of the intelligent reflecting surface 30 are as follows:
Figure BDA0003129350770000091
the transformation formula of the coordinates of the moving end 40 in the coordinate system of the camera 10 and the coordinate system of the intelligent reflection surface 30 is as follows:
Figure BDA0003129350770000092
to avoid unnecessary calculations, the algorithm is simplified, and all of the following equations are used only for (X)OQ,YOQ,ZOQ) Expressed as a known quantity.
The pitch angle θ of the movable end 40 is obtained by the following equation:
Figure BDA0003129350770000093
azimuth angle of the moving end 40
Figure BDA0003129350770000094
The following equation is obtained:
Figure BDA0003129350770000101
the upper computer of the camera 10 calculates θ and
Figure BDA0003129350770000102
then, the method further comprises: calculating the phase angle of the reflected beam corresponding to each antenna of the intelligent reflecting surface 30 according to the following formula
Figure BDA0003129350770000103
Figure BDA0003129350770000104
In the above formula, j is an imaginary unit, e is the base of the natural logarithm, M is the number of rows of the matrix, N is the number of columns of the matrix, λ is the wavelength of the electromagnetic wave, and d is the antenna pitch.
Preferably, the millimeter wave/terahertz communication method based on visual perception further includes:
identifying the position of the base station end 20 in the image according to the image information by using the intelligent reflecting surface 30;
converting the position of the base station end 20 in the image into a three-dimensional coordinate of the base station end 20 based on the depth information of the base station end;
and performing angle of incidence compensation on the antenna of the intelligent reflecting surface 30 based on the pitch angle and the azimuth angle of the real-time position of the base station end 20 calculated by the three-dimensional coordinates of the base station end 20.
Preferably, the millimeter wave/terahertz communication method based on visual perception, wherein the pitch angle and the azimuth angle of the real-time position of the base station end 20, which are calculated based on the three-dimensional coordinates of the base station end 20, include:
the incident angle corresponding to each antenna of the intelligent reflection surface 30 is calculated according to the following formula:
Figure BDA0003129350770000105
in the above formula, the first and second carbon atoms are,
Figure BDA0003129350770000106
at a base station end of 20 pitch angles theta0As the azimuth angle of the base station 20, M is the number of matrix rows, N is the number of matrix columns, λ is the wavelength of the electromagnetic wave, and d is the antenna spacing, as described above.
In this embodiment, when M is 8, and f is 30GHz, λ is 10mm, and d is 40 mm.
Preferably, in the millimeter wave/terahertz communication method based on visual perception, before the antenna controlling the intelligent reflection surface 30 directs the signal beam to the direction of the mobile terminal 40, the phase shift phase of the antenna of the intelligent reflection surface 30 is obtained by the following formula
Figure BDA0003129350770000111
Figure BDA0003129350770000112
Wherein, the expression dot multiplication is that the elements of the corresponding positions of the two matrixes with the same dimension are multiplied, and the result is still the matrix with the same dimension. Phase shifting phase
Figure BDA0003129350770000113
Each element of (a) is the phase of the antenna that needs phase shift at the corresponding position on the antenna array.
Preferably, the millimeter wave/terahertz communication method based on visual perception further includes:
recognizing the positions of the mobile terminal and the intelligent reflecting surface 30 by using the image acquired by the camera 10, wherein the camera 10 is arranged at the base station terminal 20; when the direct path between the base station 20 and the mobile station 40 is accessibleIn this case, the base station 20 determines the position of the mobile terminal 40 according to the image collected by the camera 10, and then controls the array antenna to direct the beam to the position of the mobile terminal 40, where the input phase angle matrix is
Figure BDA0003129350770000114
When the direct beam path is blocked, the base station 20 determines the position of the intelligent reflection surface 30 according to the image collected by the camera 10, and then controls the array antenna to direct the beam to the position of the intelligent reflection surface 30, wherein the input phase angle matrix is
Figure BDA0003129350770000115
Preferably, the millimeter wave/terahertz communication method based on visual perception further includes:
the positions of the base station end 20 and the intelligent reflecting surface 30 are identified by using the image acquired by the camera 10, the camera 10 is arranged at the mobile end 40 at the moment, when the direct beam path between the base station end 20 and the mobile end 40 can be reached, the mobile end 40 judges the position of the base station end 20 according to the image acquired by the camera 10, then the array antenna is controlled to point the wave beam to the position of the base station end 20, and the input phase angle matrix is
Figure BDA0003129350770000116
When the direct beam path is blocked, the mobile terminal 40 determines the position of the intelligent reflection surface 30 according to the image collected by the camera, and then controls the array antenna to direct the beam to the position of the intelligent reflection surface 30, wherein the input phase angle matrix is
Figure BDA0003129350770000121
The invention also provides a millimeter wave/terahertz communication device based on visual perception, which comprises: the system comprises a camera 10, a base station end 20, an intelligent reflecting surface 30 and a mobile end 40;
the camera 10 is used for acquiring image information and depth information of a mobile terminal in real time;
at this time, the camera 10 is disposed on the intelligent reflection surface 30;
the intelligent reflecting surface 30 identifies the position of the mobile terminal 40 in the image according to the image information; the camera 10 is further configured to convert the position of the mobile terminal 40 in the image into a three-dimensional coordinate of the mobile terminal 40 based on the depth information of the mobile terminal;
the intelligent reflecting surface 30 is further configured to control the antenna of the intelligent reflecting surface 30 to direct the signal beam to the direction of the moving end 40 based on the pitch angle and the azimuth angle of the real-time position of the moving end 40 calculated by the three-dimensional coordinates of the moving end 40.
Referring to fig. 4, when the camera 10 is located on the intelligent reflective surface 30, the intelligent reflective surface 30 identifies the real-time position of the mobile terminal 40 according to the image information, and controls the antenna of the intelligent reflective surface 30 to direct the second signal beam toward the mobile terminal 40.
The millimeter wave/terahertz communication device based on visual perception provided by the invention uses the camera 10 as visual assistance, can quickly identify the moving end 40 and determine the position of a target, and controls the phased array antenna of the intelligent reflecting surface 30 to generate a pitch angle and an azimuth angle in the corresponding direction, so that a wave beam can be more quickly directed to the corresponding target, the capacity of identifying the position of the moving end 40 is improved, and the time delay is reduced. The whole scheme is a concrete implementation and innovative application of perception and communication integration.
Preferably, the millimeter wave/terahertz communication device based on visual perception is characterized in that the intelligent reflecting surface 30 identifies the position of the base station end 20 in the image according to the image information; the camera 10 is further configured to convert the position of the base station end 20 in the image into three-dimensional coordinates of the base station end 20 based on the depth information of the base station end;
the intelligent reflecting surface 30 is further configured to perform an incident angle compensation on the antenna of the intelligent reflecting surface 30 based on the pitch angle and the azimuth angle of the real-time position of the base station end 20 calculated by the three-dimensional coordinates of the base station end 20.
Note the angle of incidence of the intelligent reflective surface 30
Figure BDA0003129350770000122
Is fixed in general and can be input into the intelligent reflection after being measured manuallyThe surface 30 may be used to calculate the incident angle by recognizing the base station end 20 and determining the position thereof using the camera 10. Referring to fig. 4, when the camera 10 is located on the intelligent reflective surface 30, the intelligent reflective surface 30 identifies the real-time position of the base station 20 according to the image information, and performs the angle of incidence compensation on the antenna of the intelligent reflective surface 30.
The camera 10 needs to compensate the incident angle of the beam emitted from the base station 20 to the intelligent reflective surface 30, and then write the reflection angle reflected to the mobile terminal 40. The upper computer of the camera 10 is as follows
Figure BDA0003129350770000131
Written to the intelligent reflective surface 30.
Compared with the traditional method, the method has wider application scenes and lower time delay in the beam forming and tracking directions in the communication field, and is the specific realization and application of the perception communication integration.
Preferably, the millimeter wave/terahertz communication device based on visual perception is configured such that the antenna of the intelligent reflection surface 30 is a passive phased array antenna, and/or the camera 10 has a function of calculating a left-right distance and a depth of an object, and/or the camera 10 is a binocular camera 10. The camera 10 may be another camera having a function of calculating the left-right distance and the depth of the object, and preferably, the camera 10 takes a picture without being limited by a frequency band, so that the device is suitable for any high-frequency band such as millimeter waves and terahertz waves.
Preferably, the millimeter wave/terahertz communication device based on visual perception is characterized in that the origin of coordinates of the camera 10 is at the geometric center of the surface of the camera 10, and the origin of coordinates of the intelligent reflecting surface 30 is approximately the same as the origin of coordinates of the camera 10. Approximately the same as the origin of coordinates of the camera 10 means substantially close to the same state, since it is difficult to achieve exact identity, and therefore substantially the same degree is sufficient.
Preferably, the millimeter wave/terahertz communication device based on visual perception is provided, wherein the antenna of the mobile terminal 40 is an omnidirectional antenna or a horn antenna.
The invention also provides a millimeter wave/terahertz communication system based on visual perception, which comprises the millimeter wave/terahertz communication device based on visual perception, wherein the millimeter wave/terahertz communication device based on visual perception comprises: the system comprises a camera 10, a base station end 20, an intelligent reflecting surface 30 and a mobile end 40;
the camera 10 is used for acquiring image information and depth information of a mobile terminal in real time;
at this time, the camera 10 is disposed on the intelligent reflection surface 30;
the intelligent reflecting surface 30 identifies the position of the mobile terminal 40 in the image according to the image information; the camera 10 is further configured to convert the position of the mobile terminal 40 in the image into a three-dimensional coordinate of the mobile terminal 40 based on the depth information of the mobile terminal;
the intelligent reflecting surface 30 is further configured to control the antenna of the intelligent reflecting surface 30 to direct the signal beam to the direction of the moving end 40 based on the pitch angle and the azimuth angle of the real-time position of the moving end 40 calculated by the three-dimensional coordinates of the moving end 40.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims (10)

1. A millimeter wave/terahertz communication method based on visual perception is characterized by comprising the following steps:
acquiring image information and depth information of a mobile terminal in real time by using a camera;
identifying the position of the mobile terminal in the image according to the image information by using the intelligent reflecting surface;
converting the position of the mobile terminal in the image into a three-dimensional coordinate of the mobile terminal based on the depth information of the mobile terminal;
and controlling an antenna of the intelligent reflecting surface to point the signal beam to the direction of the mobile terminal based on the pitch angle and the azimuth angle of the real-time position of the mobile terminal calculated by the three-dimensional coordinate of the mobile terminal.
2. The millimeter wave/terahertz communication method based on visual perception according to claim 1, wherein the camera origin of coordinates is at a geometric center of a camera surface, and the origin of coordinates of the intelligent reflecting surface is approximately the same as the camera origin of coordinates;
the coordinates of the mobile terminal in the camera coordinate system are as follows:
(XOQ,YOQ,ZOQ)
the coordinate of the mobile terminal in the intelligent reflecting surface coordinate system is as follows:
Figure FDA0003129350760000011
the conversion formula of the coordinates of the mobile terminal in the camera coordinate system and the intelligent reflecting surface coordinate system is as follows:
Figure FDA0003129350760000012
the pitch angle θ of the moving end is obtained by the following formula:
Figure FDA0003129350760000013
azimuth angle of the moving end
Figure FDA0003129350760000014
The following equation is obtained:
Figure FDA0003129350760000021
the method further comprises the following steps: calculating the phase angle of the reflected wave beam corresponding to each antenna of the intelligent reflecting surface according to the following formula
Figure FDA0003129350760000022
Figure FDA0003129350760000023
In the above formula, j is an imaginary unit, e is the base of the natural logarithm, M is the number of rows of the matrix, N is the number of columns of the matrix, λ is the wavelength of the electromagnetic wave, and d is the antenna pitch.
3. The millimeter wave/terahertz communication method based on visual perception according to claim 2, wherein the method further comprises:
acquiring depth information of a base station end in real time by using a camera;
identifying the position of the base station end in the image according to the image information by using the intelligent reflecting surface;
converting the position of the base station end in the image into a three-dimensional coordinate of the base station end based on the depth information of the base station end;
and performing incident angle compensation on the antenna of the intelligent reflecting surface based on the pitch angle and the azimuth angle of the real-time position of the base station end calculated by the three-dimensional coordinates of the base station end.
4. The millimeter wave/terahertz communication method based on visual perception according to claim 3, wherein the pitch angle and the azimuth angle of the real-time position of the base station end calculated based on the three-dimensional coordinates of the base station end comprise:
calculating the incident angle corresponding to each antenna of the intelligent reflecting surface according to the following formula:
Figure FDA0003129350760000024
in the above formula, the first and second carbon atoms are,
Figure FDA0003129350760000025
at base station end pitch angle, θ0Is the azimuth of the base station side.
5. The millimeter wave/terahertz communication method based on visual perception according to claim 4, wherein before the antenna for controlling the intelligent reflecting surface directs the signal beam to the direction of the moving end, the phase-shifted phase of the antenna for the intelligent reflecting surface is obtained by the following formula
Figure FDA0003129350760000031
Figure FDA0003129350760000032
Where, is, the dot multiplication, i.e. the multiplication of elements at corresponding positions of two matrices with the same dimension.
6. The millimeter wave/terahertz communication method based on visual perception according to claim 5, wherein the method further comprises:
recognizing the positions of the moving end and the intelligent reflecting surface by using the image acquired by the camera; when the direct radiation path between the base station end and the mobile end can be reached, the base station end judges the position of the mobile end according to the image collected by the camera, and then controls the array antenna to enable the position of the mobile end to be reachedThe wave beam points to the position of the mobile terminal, the phase angle matrix is input
Figure FDA0003129350760000033
When the direct radiation path is blocked, the base station end judges the position of the intelligent reflecting surface according to the image collected by the camera, and then controls the array antenna to point the wave beam to the position of the intelligent reflecting surface, and the input phase angle matrix is
Figure 1
7. The millimeter wave/terahertz communication method based on visual perception according to claim 5 or 6, wherein the method further comprises:
the position of the base station end and the position of the intelligent reflecting surface are identified by using the image acquired by the camera, when the direct beam path between the base station end and the mobile end can be reached, the mobile end judges the position of the base station end according to the image acquired by the camera, then the array antenna is controlled to point the wave beam to the position of the base station end, and the input phase angle matrix is
Figure FDA0003129350760000035
When the direct radiation path is blocked, the mobile terminal judges the position of the intelligent reflecting surface according to the image collected by the camera of the mobile terminal, and then controls the array antenna to point the wave beam to the position of the intelligent reflecting surface, and the input phase angle matrix is
Figure FDA0003129350760000036
8. A millimeter wave/terahertz communication device based on visual perception is characterized by comprising: the system comprises a camera, a base station end, an intelligent reflecting surface and a mobile end;
the camera is used for acquiring image information and depth information of the mobile terminal in real time;
the intelligent reflecting surface identifies the position of the mobile terminal in the image according to the image information; the camera is also used for converting the position of the mobile terminal in the image into a three-dimensional coordinate of the mobile terminal based on the depth information of the mobile terminal;
the intelligent reflecting surface is also used for controlling an antenna of the intelligent reflecting surface to point the signal beam to the direction of the moving end based on the pitch angle and the azimuth angle of the real-time position of the moving end calculated by the three-dimensional coordinate of the moving end.
9. The millimeter wave/terahertz communication device based on visual perception according to claim 8, wherein the intelligent reflective surface identifies a position of a base station end in an image according to the image information; the camera is also used for converting the position of the base station end in the image into a three-dimensional coordinate of the base station end based on the depth information of the base station end;
and the intelligent reflecting surface is also used for compensating the incident angle of the antenna of the intelligent reflecting surface based on the pitch angle and the azimuth angle of the real-time position of the base station end calculated by the three-dimensional coordinates of the base station end.
10. A millimeter wave/terahertz communication system based on visual perception, characterized by comprising the millimeter wave/terahertz communication device based on visual perception according to any one of claims 8 to 9.
CN202110698104.4A 2021-06-23 2021-06-23 Millimeter wave/terahertz communication method, device and system based on visual perception Active CN113595608B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110698104.4A CN113595608B (en) 2021-06-23 2021-06-23 Millimeter wave/terahertz communication method, device and system based on visual perception

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110698104.4A CN113595608B (en) 2021-06-23 2021-06-23 Millimeter wave/terahertz communication method, device and system based on visual perception

Publications (2)

Publication Number Publication Date
CN113595608A true CN113595608A (en) 2021-11-02
CN113595608B CN113595608B (en) 2022-04-12

Family

ID=78244366

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110698104.4A Active CN113595608B (en) 2021-06-23 2021-06-23 Millimeter wave/terahertz communication method, device and system based on visual perception

Country Status (1)

Country Link
CN (1) CN113595608B (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114520980A (en) * 2022-01-12 2022-05-20 山西大学 Three-dimensional RIS (remote RIS) assisted coal mine sensing integrated network optimization method and device
CN114828128A (en) * 2022-02-25 2022-07-29 北京邮电大学 Multi-cell communication coverage method of high-speed moving train and related equipment
CN114845332A (en) * 2022-05-05 2022-08-02 东南大学 Millimeter wave communication link blocking prediction method based on visual information fusion
WO2023133766A1 (en) * 2022-01-13 2023-07-20 华为技术有限公司 Beam control method and apparatus
CN116488740A (en) * 2023-04-17 2023-07-25 军事科学院系统工程研究院网络信息研究所 Space terahertz communication load system

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102374860A (en) * 2011-09-23 2012-03-14 奇瑞汽车股份有限公司 Three-dimensional visual positioning method and system
US20150117602A1 (en) * 2013-10-29 2015-04-30 Nuctech Company Limited Stereoscopic imaging systems and methods
CN107579759A (en) * 2017-09-19 2018-01-12 清华大学 The antihunt means and device of antenna beam in a kind of unmanned plane satellite communication system
CN110411375A (en) * 2019-08-05 2019-11-05 上海亨临光电科技有限公司 It is a kind of based on passive millimeter wave/THz imaging technology three-D imaging method
CN111245494A (en) * 2020-01-13 2020-06-05 东南大学 Positioning information auxiliary beam control method based on intelligent reflecting surface
CN112994765A (en) * 2021-03-04 2021-06-18 电子科技大学 Wave beam alignment method of intelligent reflecting surface auxiliary millimeter wave communication system

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102374860A (en) * 2011-09-23 2012-03-14 奇瑞汽车股份有限公司 Three-dimensional visual positioning method and system
US20150117602A1 (en) * 2013-10-29 2015-04-30 Nuctech Company Limited Stereoscopic imaging systems and methods
CN107579759A (en) * 2017-09-19 2018-01-12 清华大学 The antihunt means and device of antenna beam in a kind of unmanned plane satellite communication system
CN110411375A (en) * 2019-08-05 2019-11-05 上海亨临光电科技有限公司 It is a kind of based on passive millimeter wave/THz imaging technology three-D imaging method
CN111245494A (en) * 2020-01-13 2020-06-05 东南大学 Positioning information auxiliary beam control method based on intelligent reflecting surface
CN112994765A (en) * 2021-03-04 2021-06-18 电子科技大学 Wave beam alignment method of intelligent reflecting surface auxiliary millimeter wave communication system

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114520980A (en) * 2022-01-12 2022-05-20 山西大学 Three-dimensional RIS (remote RIS) assisted coal mine sensing integrated network optimization method and device
WO2023133766A1 (en) * 2022-01-13 2023-07-20 华为技术有限公司 Beam control method and apparatus
CN114828128A (en) * 2022-02-25 2022-07-29 北京邮电大学 Multi-cell communication coverage method of high-speed moving train and related equipment
CN114828128B (en) * 2022-02-25 2023-08-22 北京邮电大学 Multi-cell communication coverage method and related equipment of high-speed mobile train
CN114845332A (en) * 2022-05-05 2022-08-02 东南大学 Millimeter wave communication link blocking prediction method based on visual information fusion
CN116488740A (en) * 2023-04-17 2023-07-25 军事科学院系统工程研究院网络信息研究所 Space terahertz communication load system
CN116488740B (en) * 2023-04-17 2023-11-07 军事科学院系统工程研究院网络信息研究所 Space terahertz communication load system

Also Published As

Publication number Publication date
CN113595608B (en) 2022-04-12

Similar Documents

Publication Publication Date Title
CN113595608B (en) Millimeter wave/terahertz communication method, device and system based on visual perception
Xu et al. Computer vision aided mmWave beam alignment in V2X communications
CN113917461B (en) MIMO radar imaging method and system
CN109946748A (en) Thinned array antenna design method applied to cylinder scanning safety check imaging system
CN113093094B (en) Intelligent incident wave direction detection method based on phase regulation and control super surface
CN109597076B (en) Data processing method and device for ground-based synthetic aperture radar
CN112995888B (en) Positioning method and system based on array antenna, electronic equipment and storage medium
EP3327463B1 (en) Electromagnetic wave imaging system and antenna array signal correction method
CN107729627B (en) Quick selection method for unit of opportunistic array antenna
CN101403791B (en) Fast real-time space spectrum estimation ultra-resolution direction-finding device and method thereof
CN215835399U (en) Millimeter wave/terahertz communication device and system based on visual perception
Guo et al. Real-time indoor 3D human imaging based on MIMO radar sensing
US11743741B1 (en) Computational sensing for telecommunication target localization
EP3482562B1 (en) An apparatus and a method for generating data representative of a pixel beam
CN112099102B (en) Antenna array mechanical scanning type holographic image reconstruction method
CN110873859A (en) Method, device, medium and equipment for determining transmitting and receiving end positions based on directional antenna
CN113176566B (en) Three-dimensional rapid imaging method for MIMO-SAR
Qi et al. Application of optimized sparse antenna array in near range 3D microwave imaging
CN217428122U (en) Beam reflection communication system based on mechanical rotation
US20240137776A1 (en) Computational sensing for telecommunication target localization
US20240236696A9 (en) Computational sensing for telecommunication target localization
Freking Computer Vision Control for Phased Array Beam Steering
Zou et al. Computer vision assisted mmwave beamforming for uav-to-vehicle links
WO2022252981A1 (en) Antenna module including antenna array, and communication device
CN115469299A (en) Radar control method, device, monitoring system and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant