CN116106894A

CN116106894A - Target tracking method, device and storage medium based on 5D millimeter wave radar

Info

Publication number: CN116106894A
Application number: CN202310133062.9A
Authority: CN
Inventors: 王向荣; 刘恒峰
Original assignee: Beihang University
Current assignee: Beihang University
Priority date: 2023-02-07
Filing date: 2023-02-07
Publication date: 2023-05-12

Abstract

The application provides a target tracking method, device and storage medium based on a 5D millimeter wave radar. The millimeter wave radar comprises a millimeter wave radar which utilizes a TDM MIMO technology to perform three-transmission four-reception, the method performs signal processing on transmitting and receiving signals of the radar, so as to obtain 5D pedestrian information comprising target distance, target azimuth angle, target pitch angle information, target radial speed and target tangential speed, performs target point detection according to the 5D pedestrian information, and performs target tracking and data updating according to the 5D pedestrian information by utilizing a Kalman filtering method. The method is more perfect in acquiring the speed information, and the Kalman filtering is performed on the basis of the radial speed and the tangential speed of the target to track and update the target, so that the target track is estimated and tracked better.

Description

Target tracking method, device and storage medium based on 5D millimeter wave radar

Technical Field

The application relates to radar signal processing technology, in particular to a target tracking method, device and storage medium based on a 5D millimeter wave radar.

Background

With the rapid development of economy and the continuous improvement of the living standard of people, the conservation rate of automobiles is gradually increased, but the traffic accident event is also increased. Therefore, the demand of an Advanced Driving Assistance System (ADAS) for the safer intelligent automobile field is remarkably improved, and the ADAS can make active judgment and preventive measures before a driver finds an emergency, so that the harm caused by improper operation of the driver is reduced, and the driving safety is improved.

In an ADAS system, pedestrians who are typical disadvantaged road users are of great importance in environmental perception. The ADAS detects pedestrians by collecting data reflected by surrounding environments through sensors installed in the vehicle, so that dangerous conditions of roads are predicted, and corresponding protection measures are made. The sensors commonly used for ADAS mainly include cameras, ultrasonic radars, laser radars (Lidar), and 4D millimeter wave radars. However, the former sensors suffer from various drawbacks, in that the camera is particularly light-sensitive; the ultrasonic radar has long detection period, short maximum detection distance and large influence by temperature and environment; the laser radar has large volume, high cost, fast attenuation and extremely narrow wave beam, and can only search and capture targets in a small range. The millimeter wave radar has the characteristics of all-weather operation, no influence of light, long detection distance, high detection precision, low cost, small volume and the like, and gradually occupies an increasingly important position in the ADAS sensor.

However, the conventional 4D millimeter wave vehicle radar is weak in perception in terms of movement direction and speed information, so that it is impossible to accurately detect and track a target.

Disclosure of Invention

The application provides a target tracking method, device and storage medium based on a 5D millimeter wave radar, which are used for solving the problem that the target tracking cannot be accurately performed in the prior art.

In a first aspect, the present application provides a target tracking method based on a 5D millimeter wave radar, where the 5D millimeter wave radar includes three transmitting antennas sequentially arranged in a horizontal direction and four receiving antennas sequentially arranged in the horizontal direction, horizontal distances between two adjacent transmitting antennas are λ, vertical distances between a second transmitting antenna in a middle position and a first transmitting antenna and a second transmitting antenna are λ/2, and vertical distances between the first transmitting antenna and the second transmitting antenna are 0; the horizontal distance between two adjacent receiving antennas is lambda/2; the method comprises the following steps:

the carrier frequency f is transmitted by three transmitting antennas respectively in a time division multiplexing mode ₀ FMCW signal of 77GHz (lambda=c/f ₀ C is the speed of light), and obtains the received signals of four receiving antennas for each transmitting antenna, and carries out mixing processing according to the transmitted signals corresponding to the received signals to obtain beat signals S corresponding to 12 virtual array elements _Bij (t); wherein i represents an ith transmitting antenna, j represents a jth receiving antenna, and i and j are positive integers;

for beat signal S _B1j (t) and beat signal S _B3j (t) processing by Fourier beamforming techniques, respectively, to obtain enhanced beat signals S _B1 And S is _B3 For the enhanced beat signal S _B1 And S is _B3 2D-FFT processing is carried out to obtain a target distance R and a target radial velocity v _r ；

According to the enhanced beat signal S _B1 And S is _B3 Performing emission interference processing, obtaining coherent signals, and performing time-frequency analysis processing to obtain target tangential velocity v _a ；

For beat signal S _B1j (t) and beat signal S _B3j (t) obtaining a target azimuth angle theta according to a sum-difference beam amplitude measurement method;

according to the array arrangement of three transmitting antennas in the vertical direction, a specific phase angle measurement method is adopted to obtain a target pitch angle

According to the target distance R, the azimuth angle theta and the target pitch angle information

Performing three-dimensionalCoordinate conversion processing is carried out to obtain a target detection point;

according to the target distance R, the azimuth angle theta and the target radial velocity v _r And a target tangential velocity v _a And updating the target state by adopting Kalman filtering to finish target tracking.

In one possible design, the signal S is based on an enhanced beat signal _B1 And S is _B3 Performing emission interference processing, obtaining coherent signals, and performing time-frequency analysis processing to obtain target tangential velocity v _a Comprising:

for the enhanced beat signal S _B1 And S is _B3 Doppler compensation processing is carried out to obtain a compensated beat signal S _B1 And S is _B3 ；

For Doppler compensated beat signal S _B1 And S is _B3 Performing emission interference treatment to obtain related signals;

performing FFT processing on the coherent signal to acquire tangential Doppler frequency;

obtaining a target tangential velocity v according to the tangential Doppler frequency _a 。

In one possible design, the target radial velocity v is determined according to the target distance R, the azimuth angle θ, and the target radial velocity v _r And a target tangential velocity v _a The method for updating the target state by adopting Kalman filtering to complete target tracking comprises the following steps:

the state of the setting target at time k-1 is:

then, according to the kalman filter, the predicted value of the next time state is:

wherein F is _k Is a state transition matrix in which is the time interval between different frame signals, v is the process noise, in which] ^T Is transposed toA matrix.

In one possible design, the signal S is a beat signal _B1j (t) and beat signal S _B3j (t) processing by Fourier beamforming techniques, respectively, to obtain enhanced beat signals S _B1 And S is _B3 For the enhanced beat signal S _B1 And S is _B3 2D-FFT processing is carried out to obtain a target distance R and a target radial velocity v _r Comprising:

To beat signal S _B1j (t) and beat signal S _B3j (t) processing by Fourier beamforming techniques, respectively, to obtain enhanced beat signals S _B1 And enhancing the beat signal S _B3 ；

For the enhanced beat signal S _B1 And enhancing the beat signal S _B3 Performing fast time FFT processing, obtaining a fast time Fourier signal, obtaining an intermediate frequency related to a target distance from the fast time Fourier signal, and obtaining the target distance according to the intermediate frequency;

performing slow time FFT processing on the fast time Fourier signal to obtain a slow time Fourier signal, obtaining radial Doppler frequency shift according to the slow time Fourier signal, and obtaining a target radial velocity v according to the Doppler frequency shift _r 。

In one possible embodiment, the signal S is a beat signal _B1j (t) and beat signal S _B3j (t) obtaining a target azimuth angle θ according to a sum-difference beam amplitude mapping method, including:

constructing auxiliary beams for 8 virtual array elements corresponding to the first transmitting antenna and the third transmitting antenna, setting DBF coefficients, constructing a group of auxiliary beams to form 7 groups of auxiliary beams, and acquiring an angle discrimination curve according to the auxiliary beams;

to beat signal S _B1j (t) and beat signal S _B3j (t) performing Doppler compensation processing after performing 2D-CFAR to obtain a compensated beat signal S _B1j (t) and beat signal S _B3j (t)；

Will compensate the beat signal S _B1j (t) and beat signal S _B3j (t) multiplying the DBF coefficient and extracting the maximum value and its subscript Idx, obtaining a measuring angle range of which the target direction is in an Idx group auxiliary beam;

and obtaining a target azimuth angle theta by using a table look-up method according to the measurement angle range and the angle discrimination curve. In one possible design, the DBF coefficient dbfCoff is calculated by the following formula:

dbfCoff＝exp{-j2π(n-1)dSinθ _dbf /λ}

where n=1, …,8, where the beam orientations θ of the 7 sets of auxiliary beams are set _dbf Two are two by two, respectively: (-45.5 °, -32.5 °), (-32.5 °, -19.5 °), (-19.5 °, -6.5 °), (-6.5 °,6.5 °), (6.5 °,19.5 °), (19.5 °,32.5 °) and (32.5 °,45.5 °).

In one possible design, the target pitch angle is extracted by adopting a phase contrast angle measurement method according to the array arrangement of the three transmitting antennas in the vertical direction

Comprising the following steps:

performing Fourier wave beam forming technology processing on the beat signals of the 4 virtual array elements corresponding to the second transmitting antenna to obtain an enhanced beat signal S _BB2 ；

Performing Fourier beam forming on beat signals corresponding to the first transmitting antenna, the third receiving antenna, the first transmitting antenna, the fourth receiving antenna, the third transmitting antenna, the first receiving antenna and the third transmitting antenna, and the second receiving antenna to obtain an enhanced beat signal S' _BB13 ；

According to the enhanced beat signal S _BB2 And enhancing the beat signal S' _BB13 Is of the phase difference phi _z Extracting target pitch angle

In a second aspect, the present application provides a target tracking device based on a 5D millimeter wave radar, where the 5D millimeter wave radar includes three transmitting antennas sequentially arranged in a horizontal direction and four receiving antennas sequentially arranged in the horizontal direction, horizontal distances between two adjacent transmitting antennas are λ, vertical distances between a second transmitting antenna in a middle position and a first transmitting antenna and a second transmitting antenna are λ/2, and vertical distances between the first transmitting antenna and the second transmitting antenna are 0; the horizontal distance between two adjacent receiving antennas is lambda/2; the apparatus comprises:

a target distance and target radial velocity acquisition module for the beat signal S _B1j (t) and beat signal S _B3j (t) processing by Fourier beamforming techniques, respectively, to obtain enhanced beat signals S _B1 And S is _B3 For the enhanced beat signal S _B1 And S is _B3 2D-FFT processing is carried out to obtain a target distance R and a target radial velocity v _r ；

A target tangential velocity acquisition module for acquiring a target tangential velocity according to the enhanced beat signal S _B1 And S is _B3 Performing emission interference processing, obtaining coherent signals, and performing time-frequency analysis processing to obtain target tangential velocity v _a ；

A target azimuth acquisition module for the beat signal S _B1j (t) and beat signal S _B3j (t) obtaining a target azimuth angle theta according to a sum-difference beam amplitude measurement method;

the target pitch angle acquisition module is used for acquiring a target pitch angle by adopting a phase contrast angle measurement method according to array arrangement of three transmitting antennas in the vertical direction

The target detection module is used for detecting the target distance R, the azimuth angle theta and the target pitch angle information according to the target distance R, the azimuth angle theta and the target pitch angle information

Performing three-dimensional coordinate conversion processing to obtain a target detection point;

a target tracking module for tracking the target radial velocity v according to the target distance R, the azimuth angle theta and the target radial velocity v _r And a target tangential velocity v _a And updating the target state by adopting Kalman filtering to finish target tracking.

In a third aspect, the present application provides an electronic device, comprising: a processor, and a memory communicatively coupled to the processor;

the memory stores computer-executable instructions;

and the processor executes the computer-executed instructions stored in the memory to realize a target tracking method based on the 5D millimeter wave radar.

In a fourth aspect, the present application provides a computer-readable storage medium having stored therein computer-executable instructions that, when executed by a processor, are configured to implement a 5D millimeter wave radar-based target tracking method.

The method comprises the steps of performing signal processing on transmitting and receiving signals of the radar, so as to obtain 5D pedestrian information comprising target distance, azimuth angle, target pitch angle information, target radial speed and target tangential speed, performing target point detection according to the 5D pedestrian information, and performing target tracking and data updating according to the 5D pedestrian information by using a Kalman filtering method. The following technical effects are realized:

according to the invention, two paths of enhanced beat signals can be obtained for 4 paths of beat signals respectively corresponding to the first transmitting antenna and the third transmitting antenna, so that 6dB receiving gain is realized; thereby performing emission interference and acquiring tangential velocity information; according to the invention, two paths of enhanced beat signals are obtained through a Fourier wave beam forming technology for 4 paths of beat signals corresponding to the second transmitting antenna and 4 paths of beat signals in the middle of 8 paths of beat signals corresponding to the 1 st transmitting antenna and the 3 rd transmitting antenna, so that 6dB receiving gain is realized; extracting pitch angle information from the vertical phase difference by utilizing the array arrangement relation of the 4 paths of beat signals corresponding to the second transmitting antenna and the middle 4 paths of beat signals in the vertical direction, wherein the obtained pitch angle information is more accurate; in addition, the invention adopts the method to perform emission interference and acquire tangential velocity information so that the target is more perfect in velocity information, and according to tangential velocity and other pedestrian information; and tracking and updating the target by utilizing Kalman filtering, so that the track of the target is estimated and tracked better.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a schematic flow chart of a target tracking method based on a 5D millimeter wave radar according to an embodiment of the present application;

FIG. 2A is an array layout diagram of transmit antennas in a millimeter wave radar as used in embodiments of the present application;

FIG. 2B is an array layout diagram of a receive antenna in a millimeter wave radar as used in embodiments of the present application;

fig. 3A is a schematic diagram of angular distance information in 5D pedestrian information obtained in an embodiment of the present application;

fig. 3B is a schematic diagram of velocity information in 5D pedestrian information obtained in an embodiment of the present application;

fig. 4 is an auxiliary beam acquired in an embodiment of the present application;

FIG. 5 is an angular curve obtained in an embodiment of the present application;

FIG. 6 is an array layout diagram of 12 virtual array elements according to an embodiment of the present application;

FIG. 7A is a graph of the 3-dimensional object output results when the object moves in a trajectory from back to front in an embodiment of the present application;

FIG. 7B is a graph of the 2-dimensional object output results when the object moves in a trajectory from back to front in an embodiment of the present application;

FIG. 8A is a graph of the output results of a 3-dimensional object when the object moves in a left-to-right trajectory in an embodiment of the present application;

FIG. 8B is a graph of the output results of a 2-dimensional object when the object moves in a left-to-right trajectory in an embodiment of the present application;

FIG. 9A is a graph of the 3-dimensional object output results when the object moves in a clockwise wrap trajectory in an embodiment of the present application;

FIG. 9B is a graph of the output results of a 2-dimensional object as it moves in a clockwise wrap trajectory in an embodiment of the present application;

FIG. 10A is a graph of the output results of a 3-dimensional object when the object moves along the trajectory of the W-route in an embodiment of the present application;

FIG. 10B is a graph of 2-dimensional object output results when the object moves along the trajectory of the W-route in an embodiment of the present application;

fig. 11 is a schematic structural diagram of a target tracking device based on a 5D millimeter wave radar according to an embodiment of the present application;

fig. 12 is a schematic hardware structure of an electronic device according to an embodiment of the present application.

Detailed Description

Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary examples are not representative of all implementations consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present application, as detailed in the accompanying claims, rather than all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

Related concepts or nouns referred to in this application are explained first:

TDM technology (Time Division, time Division multiplexing): techniques for transmitting signals in a stream at different times by multiple transmit antennas.

MIMO technology (Multiple Input Multiple Output ): radar includes multiple-input multiple-output radar transmitting technology by multiple transmitting antennas and multiple receiving antennas.

FMCW (Frequency Modulated Continuous Wave ): the FMCW technology is a technology used in high-precision radar ranging, the basic principle is that the transmitted wave is a high-frequency continuous wave, the received echo frequency is the same as the change rule of the transmitted frequency, and is a triangle wave or a sawtooth wave rule.

The method can be carried in an Advanced Driving Assistance System (ADAS) system, and a 77GHz millimeter wave radar is adopted as an ADAS sensor to detect and track a far-field target. The following describes in detail a vehicle following control method provided in an embodiment of the present application with reference to the accompanying drawings.

Fig. 1 is a schematic flow chart of a target tracking method based on a 5D millimeter wave radar according to an embodiment of the present application.

Wherein fig. 2 is an array layout diagram of antennas in a millimeter wave radar adopted in the embodiment of the present application, the millimeter wave radar adopted in the embodiment of the present application includes three transmitting antennas and four transmitting antennas, where fig. 2A is an array layout diagram of transmitting antennas in the millimeter wave radar, fig. 2B is an array layout diagram of receiving antennas in the millimeter wave radar, as in fig. 2, T1, T2 and T3 are a first transmitting antenna, a second transmitting antenna and a third transmitting antenna sequentially arranged in a horizontal direction, respectively, and R1, R2, R3 and R4 are four receiving antennas sequentially arranged at equal intervals in the horizontal direction. The millimeter wave radar comprises three transmitting antennas which are sequentially arranged in the horizontal direction and four receiving antennas which are sequentially arranged in the horizontal direction, wherein the horizontal distance between two adjacent transmitting antennas is lambda, the vertical distance between a second transmitting antenna at the middle position and the first transmitting antenna and the second transmitting antenna is lambda/2, and the vertical distance between the first transmitting antenna and the second transmitting antenna is 0; the horizontal distance between two adjacent receiving antennas is lambda/2;

the method of this embodiment uses MIMO technology, and adopts array arrangement as shown in fig. 2 to transmit carrier frequency f ₀ Transmitting FMCW signals for 77GHz millimeter wave radar in a Time Division Multiplexing (TDM) mode, and performing mixing processing according to the transmitting signals corresponding to the receiving signals to obtain beat signals corresponding to 12 virtual array elements;

The beat signals corresponding to the 12 virtual array elements are processed by the method of the embodiment to obtain the signal shown in figure 3The 5D pedestrian information is shown, wherein FIG. 3 is a millimeter wave radar 5D information pedestrian according to the method of the invention, and FIG. 3A is a schematic diagram of angular distance information in the 5D pedestrian information obtained according to the embodiment of the application, including target distance R, azimuth angle θ and target pitch angle information

FIG. 3B is a schematic diagram of velocity information in 5D pedestrian information obtained in an embodiment of the present application, including a target radial velocity v _r And a target tangential velocity v _a The method comprises the steps of carrying out a first treatment on the surface of the And processing by adopting a Kalman filtering method according to the obtained 5D pedestrian information, thereby completing detection and tracking of the target.

The method is more perfect in acquiring the speed information, and the Kalman filtering is performed on the basis of the radial speed and the tangential speed of the target to track and update the target, so that the target track is estimated and tracked better.

As shown in fig. 1, the method includes:

s110, respectively transmitting carrier frequency f through three transmitting antennas by adopting a time division multiplexing mode ₀ FMCW signal of 77GHz (lambda=c/f ₀ C is the speed of light), and obtains the received signals of four receiving antennas for each transmitting antenna, and carries out mixing processing according to the transmitted signals corresponding to the received signals to obtain beat signals S corresponding to 12 virtual array elements _Bij (t); wherein i represents an ith transmitting antenna, j represents a jth receiving antenna, and i and j are positive integers;

λ＝c/f ₀ c is the speed of light

Specifically, the processing method of step 110 includes:

let the transmission signal of the ith transmission antenna be T _i The calculation is performed by the following formula:

wherein f ₀ For the center frequency, K is the chirp rate, t=nt+t _s ，(0≤t _s T is pulse repetition period，t _s Time within a single Chirp; the transmission signal of the nth Chirp is expressed as:

the four receiving antennas receive the echo signals reflected by the target, and the distance between the far-field target and the ith transmitting antenna and the jth receiving antenna is assumed to be R _ij Radial velocity v _rij Delay τ of echo signal _ij The method comprises the following steps:

wherein c is the speed of light, R ₀ For the initial distance of the target, the received signals of the four receiving antennas are recorded as R _ij (t) obtained by the following formula

The beat signal S obtained by mixing the ith transmitting antenna and the jth receiving antenna _Bij (t) is:

wherein the method comprises the steps of

Is R _ij Conjugate signal of (t), f _LPF {. The pass through low pass filter.

S120, for beat signal S _B1j (t) and beat signal S _B3j (t) processing by Fourier beamforming techniques, respectively, to obtain enhanced beat signals S _B1 And S is _B3 For the enhanced beat signal S _B1 And S is _B3 2D-FFT processing is carried out to obtain a target distance R and a target radial velocity v _r ；

Specifically, step 120 includes the steps of:

s121, opposite beat signal S _B1j (t) and beat signal S _B3j (t) processing by Fourier beamforming techniques, respectively, to obtain enhanced beat signals S _B1 And enhancing the beat signal S _B3 ；

Specifically, the beat signal S can be processed using Fourier beamforming techniques _B1j (t) and beat signal S _B3j (t) performing phase compensation, so that the phase of the four paths of received signals reaching a radar receiving end achieves the purpose of enhancing the received signals, and thereby 6dB receiving gain is realized;

in step S101, the first transmitting antenna transmits four receiving antenna echo signals:

wherein τ _1i ＝[τ _1i τ _1i +τ ₀ τ _1i +2τ ₀ τ _1i +3τ ₀ ]，

For the delay time caused by the arrangement of two adjacent receiving antennas, d is the base line length between the two receiving antennas, θ is the azimuth angle, +.>

Is pitch angle, R _1i (t) mixing with the first transmit antenna transmit signal to obtain a beat signal as:

the phase compensation is carried out on the 4 paths of beat signals corresponding to the first transmitting antenna by utilizing the beam forming technology, so that the signals which are strengthened after summation processing in space can be obtained:

S _B1 (t)＝exp{j2π[f ₀ τ ₁₁ +Kτ ₁₁ t _s ]} (8)

similarly, 4 paths of received signals corresponding to the third transmitting antenna are mixed with the transmitting signals to obtain beat signals, and fourier wave beam formation is performed to obtain enhancement signals, wherein the enhancement signals are as follows:

S _B3 (t)＝exp{j2π[f ₀ τ ₁₁ +Kτ ₁₁ t _s +(f ₀ +Kt _s )τ _D ]} (9)

Wherein the method comprises the steps of

D is the baseline length of the first transmit antenna and the third transmit antenna.

S122, pair-enhancing beat signal S _B1 And enhancing the beat signal S _B3 Performing fast time FFT processing, acquiring a fast time Fourier signal, acquiring an intermediate frequency related to the target distance from the fast time Fourier signal, and acquiring the target distance according to the intermediate frequency;

specifically, a fast time FFT process is used to solve for the target distance R, the time variable of the time FFT is t _s The beat signal is subjected to fast time FFT to obtain a frequency domain signal from which an intermediate frequency can be extracted, and the obtained intermediate frequency is f _IF ＝2KR _ij And/c, obtaining the target distance R=f through formula conversion _IF c/(2K)。

S123, performing slow time FFT processing on the fast time Fourier signals to obtain slow time Fourier signals, obtaining radial Doppler frequency shift according to the slow time Fourier signals, and obtaining the target radial velocity v according to the Doppler frequency shift _r 。

Specifically, a slow time FFT process is used to solve for the target radial velocity v _r For the frequency domain signal obtained in step S122, a time variable nT is set, and a slow time FFT is performed for nT to obtain a doppler shift, where the doppler shift is f _r ＝2v _r (nT)/lambda, the target radial velocity v can be obtained by formula conversion _r (nT)＝f _r Lambda/2, wherein the velocity information is assumed to be constant for each frame of radar echo signal.

S130, according to the enhanced beat signalNumber S _B1 And S is _B3 Performing emission interference processing, obtaining coherent signals, and performing time-frequency analysis processing to obtain target tangential velocity v _a 。

Specifically, step S130 includes the steps of:

s131, pair-enhanced beat signal S _B1 And S is _B3 Doppler compensation processing is carried out to obtain a compensated beat signal S _B1 And S is _B3 ；

Specifically, the Doppler compensation process is performed by adopting a Fourier beamforming technology, wherein the compensation factor is delta, and delta= (2 phi) _dop )/3＝2πf _dop (3T)*2/3，fd _op Is the doppler frequency.

S132, the Doppler compensated beat signal S _B1 And S is _B3 Performing emission interference processing to obtain a coherent signal;

specifically, the coherent signal is:

wherein the method comprises the steps of

Is S _B3 (t) conjugated signals.

S133, performing FFT processing on the coherent signal to obtain tangential Doppler frequency, and obtaining a target tangential velocity v according to the tangential Doppler frequency _a 。

Wherein the coherent frequency shift caused by the angular velocity can be determined by the phase term Φ in (10) _inter Obtained by the time derivative of (a), the inverse time of which is:

wherein the method comprises the steps of

Is angular velocity, and sin theta is approximately equal to 0 and cos theta is approximately equal to 0 at small angles1 the target tangential velocity +.>

S140, for beat signal S _Blj (t) and beat signal S _B3j (t) obtaining a target azimuth angle theta according to a sum-difference beam amplitude measurement method;

Specifically, S140 includes the steps of:

s141, constructing auxiliary beams for 8 virtual array elements corresponding to the first transmitting antenna and the third transmitting antenna, setting DBF coefficients, constructing one group of auxiliary beams to form 7 groups of auxiliary beams, and acquiring an angle discrimination curve according to the auxiliary beams;

wherein the DBF coefficient of the auxiliary beam is,

dbfCoff＝exp{-j2π(n-1)dsinθ _dbf /λ} (12)

wherein n=1,..8; θ _dbf = -45.5 °:13 °:45.5 °; the beam directors of the first two auxiliary beams are set to (-45.5 °, -32.5 °), and then the beam directors of each group are shifted to the right by 13 °, so the beam directors of the 7 auxiliary beams are respectively: (-45.5 °, -32.5 °), (-32.5 °, -19.5 °), (-19.5 °, -6.5 °), (-6.5 ° ), (6.5 °,19.5 °), (19.5 °,32.5 °) and (32.5 °,45.5 °) are simultaneously formed, and an angle discrimination curve is formed, the angle measurement range of which is-52 ° to 52 °, and the resulting final formed 7 sets of auxiliary beams are shown in fig. 4, and the angle discrimination curve is shown in fig. 5.

S142 is a beat signal S _B1j (t) and beat signal S _B3j (t) performing Doppler compensation processing after performing 2D-CFAR to obtain a compensated beat signal S _B1j (t) and beat signal S _B3j (t)；

Specifically, after executing 2D-CFAR, traversing in distance and doppler dimensions, extracting the maximum value of 8 virtual array element signals at each distance speed, judging whether the maximum amplitude exceeds a threshold, if so, storing the current target distance speed information, and performing doppler compensation on the virtual array element beat signals, wherein the compensation factor is delta= (2 phi) _dop )/3＝2πf _dop (3T) x 2/3 is f _dop For Doppler frequency。

S143, the compensated beat signal S _B1j (t) and beat signal S _B3j (t) multiplying the target direction by the DBF coefficient, and extracting the maximum value and the subscript Idx thereof to obtain a measurement angle range of which the target direction is in the Idx-th auxiliary beam;

s144, obtaining the target azimuth angle theta by using a table look-up method according to the measured angle range and the angle discrimination curve.

S150, obtaining a target pitch angle by adopting a phase contrast angle measurement method according to array arrangement of three transmitting antennas in the vertical direction

/>

Let the virtual array element synthesized by the ith transmitting antenna and the jth receiving antenna be VR _ij According to the arrangement of the millimeter wave radar (as shown in fig. 2), the array arrangement of 12 virtual array elements is shown in fig. 6, wherein 4 virtual array elements corresponding to the first transmitting antenna and 4 virtual array elements [ VR ] among the 8 virtual array elements corresponding to the third transmitting antenna _l1 VR ₁₂ VR _l3 VR _l4 VR ₃₁ VR ₃₂ VR ₃₃ VR ₃₄ ]The phase difference between two adjacent virtual array elements in the horizontal direction is phi _x Wherein

4-path virtual array element [ VR ] corresponding to second transmitting antenna ₂₁ VR ₂₂ VR _2a VR ₂₄ ]4 virtual array elements [ VR ] among 8 paths of beat signals corresponding to 1 st and 3 rd transmitting antennas ₁₃ VR ₁₄ VR ₃₁ VR ₃₂ ]The phase difference phi in the vertical direction exists between every two _z Wherein->

Here we use the phase difference in the vertical direction to solve for the pitch angle +. >

Specifically, step S150 may be implemented by the following steps:

s151, performing Fourier wave beam forming technology processing on the beat signals of the 4 virtual array elements corresponding to the second transmitting antenna to obtain an enhanced beat signal S _BB2 ；

S152, performing Fourier wave beam formation on the beat signals corresponding to the first transmitting antenna-third receiving antenna, the first transmitting antenna-fourth receiving antenna, the third transmitting antenna-first receiving antenna and the third transmitting antenna-second receiving antenna to obtain enhanced beat signals S' _BB13 ；

Specifically, assume that the target is at VR ₁₁ The signal generated at the point is A ₀ exp{jΦ ₀ }，A ₀ Is the amplitude, phi ₀ Is the phase. The 8 virtual array elements formed by the 1 st transmitting antenna and the 3 rd transmitting antenna can be expressed as follows by vectors:

VEC _x ＝A ₀ exp{jΦ ₀ }[1 exp{jΦ _x }exp{j2Φ _x }...exp{j7Φ _x }] (13)

the 4 virtual array elements corresponding to the second transmitting antenna can be expressed as:

VEC _z ＝A ₂ exp{j(Φ ₀ +2Φ _x -Φ _z )}[1 exp{jΦ _x }exp{j2Φ _x }exp{j3Φ _x }] (14)

wherein phi is _z Is a vertical virtual array phase difference, in which VEC _x The 3 rd to 6 th elements are VEC' _x ＝A ₀ exp{jΦ ₀ }[exp{j2Φ _x }exp{j3Φ _x }exp{j4Φ _x }exp{i5Φ _x }]And VEC (vehicle) _z The phase difference of the four elements is phi _z 。

For VEC' _x And VEC (video cassette) _z Fourier beamforming to obtain an enhanced signal S _BB2 And S' _BB13 A 6dB receive gain is achieved.

S153 based on the enhanced beat signal S _BB2 And enhancing the beat signal S' _BB13 Is of the phase difference phi _z Extracting target pitch angle

Specifically, by enhancing the signal S _BB2 And S' _BB13 Can obtain the phase difference phi _z While the target pitch angle

The solution can be achieved by,

s160, according to the target distance R, the azimuth angle theta and the target pitch angle information

specifically, according to the steps, 5D millimeter wave Lei Dadian cloud data of each frame of radar signals, namely target distance, target radial speed, target azimuth angle, target pitch angle and target tangential speed, can be obtained, and the 5D millimeter wave radar point cloud data of each frame can be obtained, and the target distance, the target azimuth angle and the target pitch angle are converted into (x, y, z) in three-dimensional coordinates according to the following formula and output;

/>

then, a centroid algorithm is executed on each dimension information in the 5D millimeter wave Lei Dadian cloud data of each frame of radar signal, and each dimension information after the centroid algorithm is executed is used as 5D point cloud data of a target at a track point of a current frame and is converted into three-dimensional coordinates (x, y, z) to be output;

s170, according to the target distance R, the azimuth angle theta and the target radial velocity v _r And a target tangential velocity v _a And updating the target state by adopting Kalman filtering to finish target tracking.

Specifically, after the 5D millimeter wave Lei Dadian cloud data of each frame of track point is obtained, the 5D Lei Dadian cloud data of the track point is updated by using kalman filtering and converted into two-dimensional coordinates (x, y) to be output as shown in a right image of fig. 8, so that the target track is predicted and tracked. The detailed operation is as follows, and since only the target two-dimensional trajectory is tracked, the state at time k-1 can be expressed as,

The predicted value of the state at the next moment is,

wherein F is _k Is a state transition matrix, where t is the time interval between different frame signals

For kalman filtering, the first part is the prediction process and the second part is the update process.

The prediction process comprises the following steps:

P _k ′＝F _k P _k-1 F _k ^T +Q (21)

wherein the updating process:

P _k ＝P _k ′-K′H _k P _k ′ (23)

K′＝P _k ′H _k ^T (H _k P _k ′H _k ^T +R _k ) ^-1 (24)

wherein P is _k-1 For the estimation error of time k-1, P' _k For the estimation error predicted at the next moment, K' represents the optimal Kalman gain, v is the process noise taking Q as covariance, and Q obeys the following distribution in a continuous white noise acceleration motion model:

the process noise Q is determined by the motion characteristics of the target, and the matrix H is observed _k Is that

After the 5D millimeter wave Lei Dadian cloud data of each frame of track point is updated, the 5D millimeter wave Lei Dadian cloud data is converted into a two-dimensional track by utilizing the distance and the azimuth angle, and the two-dimensional track is output, wherein the conversion formula is as follows:

x＝Rsinθ

y＝Rcosθ (27)

in the embodiment, when the targets respectively move along 4 tracks, the target detection and tracking are carried out by adopting the method of the invention, and as shown in fig. 7-10, the target output result diagrams are respectively obtained when the targets move along tracks of a circle and a W route from back to front, from left to right and clockwise; wherein the A graph is a 3-dimensional target output result graph during target detection; and B, a 2-dimensional target output result graph during target tracking. In the graph A, X represents three-dimensional coordinate output of 5D millimeter wave Lei Dadian cloud of each frame signal, O represents three-dimensional coordinate output of 5D millimeter wave Lei Dadian cloud of target track point in each frame signal; the B panel represents the two-dimensional coordinate output of the 5D millimeter wave Lei Dadian cloud of the kalman-filtered target trajectory point in each frame signal.

According to the method provided by the embodiment, the TDM MIMO technology is utilized to perform three-transmission four-reception millimeter wave radar, signal processing is performed on the transmitting and receiving signals of the radar, so that 5D pedestrian information comprising target distance, azimuth angle, target pitch angle information, target radial speed and target tangential speed is obtained, target point detection is performed according to the 5D pedestrian information, and tracking and data updating of targets are performed according to the 5D pedestrian information by using a Kalman filtering method. The following technical effects are realized:

The embodiment of the invention can divide the functional modules of the electronic device or the main control device according to the method example, for example, each functional module can be divided corresponding to each function, and two or more functions can be integrated in one processing unit. The integrated units may be implemented in hardware or in software functional modules. It should be noted that, in the embodiment of the present invention, the division of the modules is schematic, which is merely a logic function division, and other division manners may be implemented in actual implementation.

Fig. 11 is a schematic structural diagram of a target tracking device based on a 5D millimeter wave radar according to an embodiment of the present application. The 5D millimeter wave radar comprises three transmitting antennas which are sequentially arranged in the horizontal direction and four receiving antennas which are sequentially arranged in the horizontal direction, wherein the horizontal distance between two adjacent transmitting antennas is lambda, the vertical distance between a second transmitting antenna at the middle position and the first transmitting antenna and the second transmitting antenna is lambda/2, and the vertical distance between the first transmitting antenna and the second transmitting antenna is 0; the horizontal distance between two adjacent receiving antennas is lambda/2; as shown in fig. 11, the apparatus 1100 includes:

A mixing module 1101 for transmitting carrier frequency f through three transmitting antennas in a time division multiplexing manner ₀ FMCW signal of 77GHz (lambda=c/f ₀ C is the speed of light), and obtains the received signals of four receiving antennas for each transmitting antenna, and carries out mixing processing according to the transmitted signals corresponding to the received signals to obtain beat signals S corresponding to 12 virtual array elements _Bij (t); wherein i represents an ith path of transmitting antenna, j represents a jth path of receiving antenna, and i and j are positive integers;

a target distance and target radial velocity acquisition module 1102 for the beat signal S _B1j (t) and beat signal S _B3j (t) processing by Fourier beamforming techniques, respectively, to obtain enhanced beat signals S _B1 And S is _B3 For the enhanced beat signal S _B1 And S is _B3 2D-FFT processing is carried out to obtain a target distance R and a target radial velocity v _r ；

A target tangential velocity acquisition module 1103 for generating an enhanced beat signal S _B1 And S is _B3 Performing emission interference processing, obtaining coherent signals, and performing time-frequency analysis processing to obtain target tangential velocity v _a ；

A target azimuth acquisition module 1104 for the beat signal S _B1j (t) and beat signal S _B3j (t) obtaining a target azimuth angle theta according to a sum-difference beam amplitude measurement method;

a target pitch angle acquisition module 1105, configured to acquire a target pitch angle by using a phase contrast angle measurement method according to array arrangement of three transmitting antennas in a vertical direction

A target detection module 1106 for detecting a target pitch angle according to the target distance R, the azimuth angle θ, and the target pitch angle information

a target tracking module 1107 for tracking the target radial velocity v according to the target distance R, the azimuth angle θ _r And a target tangential velocity v _a And updating the target state by adopting Kalman filtering to finish target tracking.

Further, the target tangential velocity acquisition module 1103 is specifically configured to:

performing FFT processing on the coherent signal to obtain tangential Doppler frequency;

obtaining the tangential velocity v of the target according to the tangential Doppler frequency _a 。

Further, the target tracking module 1107 is specifically configured to:

the state of the setting target at time k-1 is:

wherein F is _k Is a state transition matrix, where t is the time interval between different frame signals, v is the process noise, [] ^T Is the transposed matrix.

Further, the target distance and target radial velocity acquisition module 1102 is specifically configured to:

For the enhanced beat signal S _B1 And enhancing the beat signal S _B3 Performing fast time FFT processing, acquiring a fast time Fourier signal, acquiring an intermediate frequency related to the target distance from the fast time Fourier signal, and acquiring the target distance according to the intermediate frequency;

performing slow time FFT processing on the fast time Fourier signal to obtain a slow time Fourier signal, acquiring radial Doppler shift according to the slow time Fourier signal, and obtaining the target radial velocity v according to the Doppler shift _r 。

Further, the target azimuth acquisition module 1104 is specifically configured to:

constructing auxiliary beams for 8 virtual array elements corresponding to the first transmitting antenna and the third transmitting antenna, setting DBF coefficients, constructing a group of auxiliary beams two by two to form 7 groups of auxiliary beams, and acquiring an angle discrimination curve according to the auxiliary beams;

Will compensate the beat signal S _B1j (t) and beat signal S _B3j (t) multiplying the target direction by the DBF coefficient, and extracting the maximum value and the subscript Idx thereof to obtain a measurement angle range of which the target direction is in the Idx-th auxiliary beam;

And obtaining the target azimuth angle theta by using a table look-up method according to the measured angle range and the angle discrimination curve.

Further, the target azimuth acquisition module 1104 is further configured to:

the DBF coefficient dbfCoff is calculated by the following formula:

dbfCoff＝exp{-j2π(n-1)dsinθ _dbf /λ} (12)

where n=1.Beam pointing θ of 7 sets of auxiliary beams is set therein _dbf Two are two by two, respectively: (-45.5 °, -32.5 °), (-32.5 °, -19.5 °), (-19.5 °, -6.5 °), (-6.5 °,6.5 °), (6.5 °,19.5 °), (19.5 °,32.5 °) and (32.5 °,45.5 °).

Further, the target pitch angle acquisition module 1105 is specifically configured to:

The target tracking method based on the 5D millimeter wave radar provided in the present embodiment may perform the target tracking method based on the 5D millimeter wave radar in the above embodiment, and its implementation principle and technical effects are similar, and this embodiment will not be described here again.

In the foregoing specific implementation of a target tracking device based on a 5D millimeter wave radar, each module may be implemented as a processor, and the processor may execute computer-executable instructions stored in the memory, so that the processor executes a target tracking method based on a 5D millimeter wave radar.

Fig. 12 is a schematic structural diagram of an electronic device according to an embodiment of the present application. As shown in fig. 12, the electronic apparatus 1200 includes: at least one processor 1201 and memory 1202. The electronic device 1200 further comprises a communication part 1203. The processor 1201, the memory 1202, and the communication section 1203 are connected via a bus 1204.

In a specific implementation, the at least one processor 1201 executes computer-executable instructions stored in the memory 1202, such that the at least one processor 1201 performs a target tracking method based on a 5D millimeter wave radar as performed on the electronic device side above.

The specific implementation process of the processor 1201 can be referred to the above method embodiment, and its implementation principle and technical effects are similar, and this embodiment will not be described herein again.

In the above embodiment, it should be understood that the processor may be a central processing unit (english: central Processing Unit, abbreviated as CPU), or may be other general purpose processors, digital signal processors (english: digital Signal Processor, abbreviated as DSP), application specific integrated circuits (english: application Specific Integrated Circuit, abbreviated as ASIC), or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of a method disclosed in connection with the present invention may be embodied directly in a hardware processor for execution, or in a combination of hardware and software modules in a processor for execution.

The memory may comprise high speed RAM memory or may further comprise non-volatile storage NVM, such as at least one disk memory.

The bus may be an industry standard architecture (Industry Standard Architecture, ISA) bus, an external device interconnect (Peripheral Component, PCI) bus, or an extended industry standard architecture (Extended Industry Standard Architecture, EISA) bus, among others. The buses may be divided into address buses, data buses, control buses, etc. For ease of illustration, the buses in the drawings of the present application are not limited to only one bus or one type of bus.

The scheme provided by the embodiment of the invention is introduced aiming at the functions realized by the electronic equipment and the main control equipment. It will be appreciated that the electronic device or the master device, in order to implement the above-described functions, includes corresponding hardware structures and/or software modules that perform the respective functions. The present embodiments can be implemented in hardware or a combination of hardware and computer software in combination with the various exemplary elements and algorithm steps described in connection with the embodiments disclosed in the embodiments of the present invention. Whether a function is implemented as hardware or computer software driven hardware depends upon the particular application and design constraints imposed on the solution. Those skilled in the art may implement the described functionality using different approaches for each particular application, but such implementation is not to be considered as beyond the scope of the embodiments of the present invention.

The application also provides a computer readable storage medium, wherein computer execution instructions are stored in the computer readable storage medium, and when a processor executes the computer execution instructions, the target tracking method based on the 5D millimeter wave radar is realized.

The computer readable storage medium described above may be implemented by any type of volatile or non-volatile memory device or combination thereof, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk. A readable storage medium can be any available medium that can be accessed by a general purpose or special purpose computer.

An exemplary readable storage medium is coupled to the processor such the processor can read information from, and write information to, the readable storage medium. In the alternative, the readable storage medium may be integral to the processor. The processor and the readable storage medium may reside in an application specific integrated circuit (Application Specific Integrated Circuits, ASIC for short). The processor and the readable storage medium may reside as discrete components in an electronic device or a master device.

The present application also provides a computer program product comprising: a computer program stored in a readable storage medium, from which at least one processor of an electronic device can read, the at least one processor executing the computer program causing the electronic device to perform the solution provided by any one of the embodiments described above.

Those of ordinary skill in the art will appreciate that: all or part of the steps for implementing the method embodiments described above may be performed by hardware associated with program instructions. The foregoing program may be stored in a computer readable storage medium. The program, when executed, performs steps including the method embodiments described above; and the aforementioned storage medium includes: various media that can store program code, such as ROM, RAM, magnetic or optical disks.

Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the invention.

Claims

1. The target tracking method based on the 5D millimeter wave radar is characterized in that the millimeter wave radar comprises three transmitting antennas which are sequentially arranged in the horizontal direction and four receiving antennas which are sequentially arranged in the horizontal direction, the horizontal distance between two adjacent transmitting antennas is lambda, the vertical distance between a second transmitting antenna at the middle position and a first transmitting antenna and a second transmitting antenna is lambda/2, and the vertical distance between the first transmitting antenna and the second transmitting antenna is 0; the horizontal distance between two adjacent receiving antennas is lambda/2; the method comprises the following steps:

the carrier frequency f is transmitted by three transmitting antennas respectively in a time division multiplexing mode ₀ An FMCW signal of 77GHz, where λ=c/f ₀ C is the light speed, and obtains the receiving signals of four receiving antennas aiming at each transmitting antenna, and carries out mixing processing according to the transmitting signals corresponding to the receiving signals to obtain beat signals S corresponding to 12 virtual array elements _Bij (t); wherein i represents an ith transmitting antenna, j represents a jth receiving antenna, and i and j are positive integers;

for beat signal S _B1j (t) and beat signal S _B3j (t) respectively lead toProcessing by Fourier wave beam forming technology to obtain enhanced beat signal S _B1 And S is _B3 For the enhanced beat signal S _B1 And S is _B3 2D-FFT processing is carried out to obtain a target distance R and a target radial velocity v _r ；

2. The method according to claim 1, wherein the step-up beat signal S is based on _B1 And S is _B3 Performing emission interference processing, obtaining coherent signals, and performing time-frequency analysis processing to obtain target tangential velocity v _a Comprising:

For Doppler compensated beat signal S _B1 And S is _B3 Performing emission interference treatment to obtainTo the associated signal;

3. The method according to claim 1, wherein the target radial velocity v is determined according to the target distance R, target azimuth angle θ, target radial velocity v _r And a target tangential velocity v _a The method for updating the target state by adopting Kalman filtering to complete target tracking comprises the following steps:

the state of the setting target at time k-1 is:

wherein F is _k Is a state transition matrix, where t is the time interval between different frame signals, v is the process noise, where [ [] ^T Is the transposed matrix.

4. The method according to claim 1, characterized in that the signal S is a beat signal _B1j (t) and beat signal S _B3j (t) processing by Fourier beamforming techniques, respectively, to obtain enhanced beat signals S _B1 And S is _B3 For the enhanced beat signal S _B1 And S is _B3 2D-FFT processing is carried out to obtain a target distance R and a target radial velocity v _r Comprising:

5. The method according to claim 1, characterized in that the signal S is a beat signal _B1j (t) and beat signal S _B3j (t) obtaining a target azimuth angle θ according to a sum-difference beam amplitude mapping method, including:

and obtaining a target azimuth angle theta by using a table look-up method according to the measurement angle range and the angle discrimination curve.

6. The method of claim 5, wherein the DBF coefficient dbfCoff is calculated by the formula:

dbfCoff＝exp{-j2π(n-1)dsinθ _dbf /λ}

where n=1,..8, beam fingers in which 7 sets of auxiliary beams are setTo theta _dbf Two are two by two, respectively: (-45.5 °, -32.5 °), (-32.5 °, -19.5 °), (-19.5 °, -6.5 °), (-6.5 °,6.5 °), (6.5 °,19.5 °), (19.5 °,32.5 °) and (32.5 °,45.5 °).

7. The method of claim 6, wherein the target pitch angle is extracted by a phase contrast angle method according to the array arrangement of three transmitting antennas in the vertical direction

Comprising the following steps:

8. The target tracking device based on the 5D millimeter wave radar is characterized in that the 5D millimeter wave radar comprises three transmitting antennas which are sequentially arranged in the horizontal direction and four receiving antennas which are sequentially arranged in the horizontal direction, the horizontal distance between two adjacent transmitting antennas is lambda, the vertical distance between a second transmitting antenna in the middle position and a first transmitting antenna and a second transmitting antenna is lambda/2, and the vertical distance between the first transmitting antenna and the second transmitting antenna is 0; the horizontal distance between two adjacent receiving antennas is lambda/2; the apparatus comprises:

a frequency mixing module for passing through three transmitting antennas in a time division multiplexing modeRespectively transmitting carrier frequency f ₀ FMCW signal of 77GHz (lambda=c/f ₀ C is the speed of light), and obtains the received signals of four receiving antennas for each transmitting antenna, and carries out mixing processing according to the transmitted signals corresponding to the received signals to obtain beat signals S corresponding to 12 virtual array elements _Bij (t); wherein i represents an ith path of transmitting antenna, j represents a jth path of receiving antenna, and i and j are positive integers;

a target tracking module for tracking the target radial velocity v according to the target distance R, the azimuth angle theta and the target radial velocity v _r And a target tangential velocity v _a By usingAnd updating the target state by Kalman filtering to finish target tracking.

9. An electronic device, comprising: a processor, and a memory communicatively coupled to the processor;

The memory stores computer-executable instructions;

the processor executes computer-executable instructions stored in the memory to implement the method of any one of claims 1 to 7.

10. A computer readable storage medium having stored therein computer executable instructions which when executed by a processor are adapted to carry out the method of any one of claims 1 to 7.