WO2022217537A1 - Smart door and non-contact control method therefor - Google Patents

Abstract

A smart door for realizing non-contact control using a millimeter-wave radar, a non-contact control system, and a non-contact control method. The control system (10) comprises: a millimeter-wave radar sensor (101) used for transmitting a radar beam and receiving a reflected radar beam to detect user gesture commands; a state feedback component (102) used for feeding back state information to a user to implement state interaction; and a processor (103) connected to the millimeter-wave radar sensor (101), the state feedback component (102), and a motion drive component (104). The processor (103) detects the user gesture commands in real time, feeds back the execution state in real time, splits the complex gestures into a plurality of simple gesture sequences, confirms same one by one, and executes corresponding control operations after receiving the complete user gesture commands. The recognition of the processor (103) on each of the user gesture commands comprises an interaction process where a plurality of users and the control system mutually confirm, starts when the user gets close to the millimeter-wave radar sensor (101), and ends when the user is away from the millimeter-wave radar sensor (101). The processor (103) executes a corresponding action in each confirmation interaction, so as to split the recognition of the complex gesture commands into a plurality of progressive steps of recognizing and confirming the basic gestures one by one. According to the control system (10), algorithm optimization is replaced with the system design, and the design is simplified while both the control reliability and convenience are taken into consideration, so that the control system is applied to a resource-limited embedded terminal, and the technical effects of conditional enabling, and convenient and reliable control of non-contact are achieved.

Description

Smart door and non-contact control method of smart door technical field

The invention relates to a non-contact control smart door, a non-contact control module of the smart door, and a non-contact control method applied to the smart door.

Background technique

In daily life, the switch control of the door can usually be realized in various ways. The most common ordinary door provides mechanical switch control through the handle, and the door is opened by turning the handle, pulling the handle, etc.; for the control of elevators and other occasions , the switch can also be controlled by means of electronic buttons, etc. These contact control methods can easily become a shortcut for the spread of bacteria and viruses in public places, posing public health and safety hazards. For example, during the outbreak of infectious diseases, elevator panels need to be strictly disinfected to avoid contact infection.

In order to avoid the drawbacks of contact control, automatic doors that sense the approach of objects are also widely used in public places such as shopping malls and hotels, which will automatically open when an object is detected approaching; In the occasions where automatic opening is prohibited under certain circumstances, such as elevators, when the elevator car is not on the current floor, the automatic opening of the elevator doors will be very dangerous and should be strictly avoided. Similar automatic doors are also used in smart refrigerators and smart toilet lids. The disadvantage is that only the distance information is used to detect the triggering condition of the approaching object, which is too simple and easy to trigger by mistake.

In order to avoid false triggering, it is necessary to set more complex conditions for automatic door opening. For example, more complex gestures are used for control. However, for complex gesture recognition, the current system recognition method requires a lot of computational cost and is accurate. The rate is low, and the reliability of the door opening cannot be guaranteed.

Therefore, there is a need to provide a non-contact conditional opening, convenient and reliable smart door.

SUMMARY OF THE INVENTION

The invention provides a non-contact control smart door, a non-contact control module for the smart door, and a non-contact control method applied to the smart door; the millimeter wave sensor is applied to the smart door, and by detecting user gestures, the realization of Door opening/closing control, or other control functions, such as elevator upstairs, downstairs requests; can be applied to public places and occasions where public transportation requires frequent contact, such as shopping malls and office building elevators, airports, railway stations and trains and The toilet door on the plane, etc., to cut off the transmission of bacteria and viruses, ensure public health and safety, and at the same time ensure the reliability of opening and avoid misoperation. It can also be applied to the non-contact control of kitchen and bathroom appliances such as smart refrigerator doors to improve control reliability.

The present invention provides an intelligent door using millimeter-wave radar to realize non-contact control, comprising: a door frame (1), a door panel (2), a moving part (3) that drives the door panel (2) to open and close, and a control system (10); The control system (10) includes: a millimeter-wave radar sensor (101), which transmits a radar beam and receives a reflected radar beam to detect a user gesture command; a state feedback component (102), which is used for feeding back state information to the user to realize state interaction; processing a device (103), which is connected to the millimeter wave radar sensor (101), the state feedback part (102) and the motion driving part (104); according to the user gesture command received by the millimeter wave radar sensor (101), the state feedback part (102) is controlled ) outputs status information and executes corresponding control operations. The control operation is to control the motion of the motion driving component (104) to drive the motion component (3) to drive the door panel (2) to move, so as to realize the opening and closing of the smart door.

A millimeter-wave radar sensor (101) includes a transmitting circuit (1011) that transmits the radar beam, a receiving circuit (1012) that receives the reflected radar beam, and a digital signal processing circuit (1013). The transmitting circuit (1011) includes: M transmitting antennas for emitting the radar beam, configured in a phased array mode or a MIMO mode; a power amplifier, linear power amplifying, to drive the transmitting antenna; a numerically controlled oscillator circuit and a digital phase locking loop to achieve frequency modulation. In order to improve the modulation bandwidth and linearity, the digitally controlled oscillator circuit and the digital phase-locked loop adopt a 2-point injection structure. The receiving circuit (1012) includes: N receiving antennas arranged in a ULA or URA manner; a low-noise amplifying LNA for amplifying the amplitude of the received signal; frequency mixing, mixing the received signal amplified by the low-noise amplifying LNA and local frequency modulation signal to obtain intermediate frequency signal; low-pass filtering and variable gain amplification; analog-to-digital conversion circuit to convert analog signal into digital intermediate frequency signal. The digital signal processing circuit (1013) includes: a ramp generating circuit, which generates a sawtooth or triangular signal to adjust the oscillation frequency of the numerically controlled oscillation circuit; a fast Fourier transform (FFT), which obtains the target distance by searching for the peak value of the signal amplitude after the FFT transformation , radial velocity and incident angle; a window function for reducing spectrum spreading caused by FFT truncation; a first-in-first-out buffer FIFO for buffering FFT-transformed data for further processing by the processor (103).

The control system (10) further comprises a communication interface (105), which is connected with the processor (103) to realize the interconnection and information exchange between the control system (10) and the external environment.

The control system (10) may have the following physical form combinations according to application requirements: PCB level integration based on surface mount technology, package level integration based on SIP technology, or silicon wafer level integration based on SOC technology.

Two sets of millimeter-wave radar sensors (101) and state feedback components (102) in the control system (10) are respectively provided, which are respectively used for processing user gesture commands and state interaction on the inner and outer sides of the door, and the two sets of millimeter-wave radar sensors (101), and the priority of the two sets of the millimeter-wave radar sensors (101) needs to be set.

A non-contact control method, comprising the steps of:

a. The millimeter wave radar sensor (101) transmits a radar beam;

b. The millimeter-wave radar sensor (101) receives the radar signal reflected by the user's gesture movement and performs preliminary processing and buffering for the processor (103) to read and process; the millimeter-wave radar sensor (101) receiving circuit (1012) The radio frequency front-end receives The received reflected signal is subjected to low-noise amplification, frequency mixing, and analog-to-digital conversion to obtain a digital intermediate frequency signal. The digital signal processing circuit (1013) of the millimeter wave radar sensor (101) performs windowing and fast Fourier transform (FFT) on the digital intermediate frequency signal, and writes the result into a first-in, first-out buffer (FIFO) for the processor (103) read and process.

c. The processor (103) reads the Fourier-transformed data from the first-in, first-out buffer (FIFO) of the digital processing circuit (1013), searches for its peak value, and obtains an estimate of the target distance and radial velocity, and uses Capon or The MUSIC method estimates the angle of incidence, and these measurements are fed into a Kalman filter or particle filter as observations of the target state space, resulting in a smooth estimate of the user's gesture motion trajectory.

d. Based on the tracking of the motion trajectory of the user's gesture, the processor (103) controls the state feedback component (102) to output the current state;

e. Continue to run steps a-d. After receiving the complete control command, the processor (103) executes the corresponding control operation, or cancels the execution of the command when the premature end command is detected.

The user gesture movement is a complex control gesture, and the processor (103) decomposes the complex control gesture into multiple basic gesture sequences and confirms them one by one. The segment-by-segment confirmation based on the multi-segment basic gesture sequence includes multiple interaction processes of mutual confirmation between the user and the control system (10). For interaction, the processor (103) performs corresponding actions, and multiple basic gestures are progressively confirmed to realize complete control commands.

Wherein, step d includes:

d1. The processor (103) tracks the motion trajectory of the user's gesture, and detects that the user has approached within the set threshold range, then confirms that the system has been activated in the form of light or sound through the state feedback component (102), and is ready to receive the next step Order;

d2. When the user judges that it is close enough according to the feedback from the state feedback component (102), continue the next gesture action; or end the current command by moving away from the millimeter wave radar sensor (101).

Using complex control gestures as control commands, collecting a large number of motion trajectories as training data, and using pattern recognition or machine learning techniques to obtain models representing control gesture features, to improve the recognition rate of the complex control gesture commands.

A non-contact gesture control system, comprising: a millimeter-wave radar sensor (101), which transmits a radar beam and receives a reflected radar beam to detect a user gesture command; it is characterized in that, it further comprises: a state feedback component (102), which is used for sending the radar beam to the user. The user feeds back state information to realize state interaction; a processor (103) is connected to the millimeter wave radar sensor (101) and the state feedback component (102); the processor (103) detects the user gesture command in real time, and Real-time feedback of the execution status, decompose complex gestures into several simple gesture sequences, and confirm segment by segment, and execute corresponding control operations after receiving the complete user gesture command. The recognition of each of the user gesture commands by the processor (103) includes several interaction processes in which the user and the control system mutually confirm each other, starting with the user approaching the millimeter-wave radar sensor (101) and moving away from the millimeter-wave radar sensor ( 101) End, each time the interaction is confirmed, the processor (103) executes corresponding actions, thereby decomposing the recognition of complex gesture commands into a plurality of progressive steps of basic gesture recognition and step-by-step confirmation.

By integrating the millimeter wave radar sensor and the state feedback component in the system, the present invention realizes two-way interaction, detects gesture commands in real time, and feeds back the execution state, decomposes complex gestures into several simple gesture sequences, and confirms the control protocol segment by segment, thereby Avoid the problem that the simple recognition algorithm has insufficient recognition rate for complex gestures and takes up a lot of computing and storage resources, that is, replace the algorithm optimization with system design, and simplify the design while taking into account the control reliability and convenience, so that it can be applied to limited resources. embedded terminal. Each gesture control command can contain several mutually confirmed interaction processes, starting with approaching the radar sensor and ending with a distance from the radar sensor. Each time the interaction is confirmed, there will be corresponding execution actions, thereby decomposing complex control gestures into multiple progressive steps. The steps can also be executed by the early end cancel command.

In addition, the present invention can also achieve the following beneficial effects:

1. The millimeter wave sensor is applied to the smart door. By detecting user gestures and controlling the opening and closing of the door, it can cut off the transmission route of bacteria and viruses and ensure public health and safety;

2. By setting the status feedback component in the control system, the recognition problem of the user's complex gestures can be effectively solved, and the complex gestures of the user can be decomposed into multiple basic gesture sequences. Segment confirmation to ensure the reliability of gesture recognition, so as to ensure the reliability of smart door opening and avoid misoperation;

3. By setting the millimeter wave radar sensor as the input and the state feedback component as the output in the control system, the user's input state is fed back in real time to form a closed-loop control system to achieve simple and convenient human-computer interaction.

Description of drawings

Fig. 1 is the overall structure schematic diagram of the present invention;

Fig. 2 is the control system block diagram of the present invention;

Figure 3 is the frequency-time function of the sawtooth FMCW radar transmit and reflected signals;

FIG. 4 is a schematic diagram of an angle of incidence (DOA) estimation based on multiple receive antennas;

Figure 5 is a block diagram of the RF front-end and digital signal processing structure of the millimeter-wave radar sensor;

Figure 6 is a schematic diagram of ADPLL frequency modulation;

Figure 7 shows the relationship between the arrangement direction of the receiving antennas of the millimeter-wave radar sensor and the sliding direction of the gesture;

Fig. 8 is the gesture of simulating the key;

9 is a schematic diagram of gesture command sequence structure and interactive control state transition;

Fig. 10 is a schematic diagram of the arrangement direction of the non-contact elevator control panel and the corresponding millimeter-wave radar receiving antenna designed by applying the present invention.

Detailed ways

The implementation and use of the presently preferred embodiments are discussed in detail below. It should be appreciated, however, that the present invention provides many applicable inventive concepts that can be embodied in a wide variety of specific contexts. The specific embodiments discussed are merely illustrative of specific ways to make and use the invention, and do not limit the scope of the invention.

A millimeter-wave radar-based non-contact control method on smart doors and smart doors. The overall structure of the smart door is shown in Figure 1, including:

door frame 1;

The door panel 2 plays the role of space partition; the door panel 2 can be made of various materials, such as metal, glass, composite wood, etc., or can be in various forms, such as single-leaf, double-leaf, etc., various opening and closing movement methods, such as moving , rotation, etc.;

The moving part 3 is a mechanical part that drives the door panel 2 to open and close, for example, by means of a guide rail, a rotating shaft, etc., and is driven by a driving part such as a motor to realize the translation or rotation of the door panel 2;

The control system 10, the control system 10 transmits the radar beam through the millimeter wave radar sensor 101, and receives the reflected radar beam to detect the user gesture command, and simultaneously outputs the detection status in the form of light or sound through the state feedback component 102. The command controls the motion driving part 104 to drive the motion part 3 to realize the opening and closing of the door, and the control system 10 can also send the command to the upper system or the cloud through the communication interface 105 to realize more complex control functions.

FIG. 2 is a block diagram of the control system 10 having the smart door control device of the present invention, and the control system 10 will be described in detail below with reference to FIG. 2 .

The millimeter-wave radar sensor 101 includes a transmitting circuit 1011 that transmits radar beams, a receiving circuit 1012 that receives radar reflected signals, and a digital signal processing circuit 1013; Digital signal processing provides a fast and concise control input interface; however, because the input realized by the millimeter-wave radar sensor 101 is non-contact input, it cannot provide state output in the form of force feedback, and the user cannot know whether the input has been successful or not. Next input is required.

The status feedback component 102 usually includes a control panel that constitutes a human-computer interaction interface, and an output interface that can feedback status information with light or sound, such as a light-emitting diode (LED), a liquid crystal screen (LCD), or a speaker (Speaker), etc., Provide visual or auditory status feedback, so that users can perceive the execution status of their gesture commands, decide to continue or cancel the current command, and achieve more effective human-computer interaction.

The processor 103 is connected to the motion driving part 104 , the millimeter wave radar sensor 101 , the state feedback part 102 , the communication interface 105 and the storage medium 106 .

Storage medium 106 includes static or dynamic random access storage medium 1061 for caching code and runtime dynamic data; and non-transitory storage medium 1062 for storing a program executed by the processor; the program includes instructions for: According to the radar reflection signal received by the millimeter-wave radar sensor 101, the door opening and closing gesture command is recognized, the state feedback part 102 is controlled to output the state information, and the motion driving part 104 is driven to act to realize the opening or closing of the door; the upper layer communication based on the communication interface 105 Protocol processing, such as TCP/IP, http, etc.; and self-checking and calibration of the control system 10.

The motion drive part 104, controlled by the processor 103, converts electrical energy into mechanical energy to drive the motion part 3 to drive the door panel 2 to open and close; the motion drive part 104 can be in the form of a motor or the like.

The communication interface 105 realizes the interconnection and information exchange between the control system 10 and the external environment, including wired communication interfaces, such as control buses such as I2C and SPI, or communication links such as Ethernet, USB, and PCIExpress, and wireless communication interfaces, such as WiFi, Bluetooth, etc. Therefore, the control system 10 can be integrated into a more complex system as a component. For example, it can be used in the elevator panel. It can be interconnected with the elevator dispatching system through a control bus such as I2C and SPI, and send upstairs and downstairs requests. The wired and wireless communication link is connected to the Internet of Things (IoT), and the remote control is realized through the cloud.

The components of the above control system 10 can have the following physical form combinations according to application requirements:

a. PCB-level integration based on surface mount technology (SMT);

b. Package-level integration based on SIP technology (System in Package);

c. Silicon-level integration based on SOC technology (System on Chip).

In some specific application scenarios, in order to support the simultaneous control of the inside and outside of the door, two sets of millimeter-wave radar sensors 101 and state feedback components 102 are required to realize simultaneous user control gesture input and state output inside and outside the door; the middle of the two sets of millimeter-wave radar sensors 101 Shielding processing to isolate the radar signals on both sides; in addition, when receiving control commands from multiple millimeter-wave radar sensors 101 and the same component needs to be operated, the processor 103 needs to set the priority according to the received gesture commands on the inner and outer sides. Make judgments to resolve conflicts; for example, when a high-priority side receives a door-closing command, it will block the door-opening command on the other side.

The millimeter-wave radar sensor 101 can measure the distance, radial velocity and incident angle of the target at the same time. The working principle of the millimeter-wave radar sensor 101 is introduced by taking the sawtooth frequency modulated continuous wave (FMCW) as an example below; the millimeter-wave radar sensor 101 also Millimeter-wave radar technology with other modulation methods can be used, such as three-stage frequency-modulated continuous wave radar or pulsed radar.

The FMCW millimeter-wave radar sensor extracts information about the distance, radial velocity and angle of the target by sending the FMCW radar beam and receiving the signal reflected by the target on the propagation path of the radar beam, and obtains the discrete measurement of the gesture trajectory. Bayesian filtering of these measurements yields smooth trajectories. The basic principle is as follows:

Figure 3 is a linear FMCW transmitting radar signal, and a single target reflecting and receiving radar signal, and the frequency changes waveforms with time. With such frequency characteristics, assuming the initial phase is 0, the transmitted FM signal with normalized amplitude is:

s(t)=cos(2πf _c t+πα(t-mT) ² )

Among them, α is the slope of the linear frequency modulation, f _c is the carrier frequency, as shown in Figure 3, T is the frequency modulation signal cycle, m is the number of cycles.

Assuming that the distance between the target and the sensor is R, the radial velocity is v, and the speed of light is c, after the delay

The received signal is:

r(t)=cos(2πf _c (t-τ)+πα(t-τ-mT) ² )

The IF signal after mixing and low-pass filtering is:

Among them, H(t) is the impulse response function of the low-pass filter.

Let t _s = t - mT, _0≤ts ≤T, and assuming the target is moving slowly (usually it is reasonable to apply this assumption for gestures), substituting t = _ts + mT into g(t) yields

in,

So its Fourier transform is:

Take the frequency as the positive part:

so

Therefore, the target distance can be detected by

to estimate the peak value.

Assume

is the phasor of the distance FFT peak sequence, and the sequence is then Fourier transformed (Doppler FFT)

P(f)=δ(f-Tf _d )

so

Therefore, the radial velocity can be estimated by detecting the peak value of |P(f)|.

Since the distance and radial velocity only reflect the information of one dimension of the target, it can only detect whether the target is close; in order to obtain accurate target position information, an array composed of multiple receiving antennas is required to measure the 2D position of the target. N independent receiving antennas are arranged in the form of an equidistant linear array ULA (Uniform Linear Array), or MIMO radar technology can be used, using M transmitting antennas and N receiving antennas to be arranged in a specific form to obtain an M×N virtual array , to improve angular resolution.

Figure 4 shows a form of receive antenna array. For this receiving antenna array, set the distance between adjacent antennas to be d, then the phase difference between adjacent receiving channels is

where θ is the angle of incidence and λ is the wavelength.

Similar to estimating the velocity according to the Doppler FFT peak value, Fourier transform (angle FFT) is performed on the sequence composed of the phasors at the peak of the FFT amplitude of the N channels, and the peak value is used as the estimation of ω, so that according to

Estimate the angle of incidence. The angular resolution obtained by this method is limited by the number of arrays, such as for

The ULA array with an angular resolution of

For an array of N=8, the angular resolution is approximately 15°.

To obtain higher angular resolution, Capon or MUSIC (Multiple Signal Classifier) methods can be used. MUSIC is a vector space-based method, which needs to know the number of targets to be detected in advance. The advantage of Capon is that the number of targets does not need to be known, and the estimation of the incident angle and power of the reflected signals of multiple targets can be directly obtained, so it can be used to initialize MUSIC. .

Assuming that K targets reflect signals at different incident angles θ _k and are simultaneously received by N receiving antennas, the vectors of K reflected signals S(t) and N received signals X(t) are expressed as

S(t)=[s ₁ (t),...s _K (t)] ^T ; X(t)=[x ₁ (t)...x _N (t)] ^T

Corresponding to each reflected signal, the vector (steering vector) formed by the phase difference reaching N receiving antennas can be expressed as

Then the matrix of K steering vectors is expressed as

A(θ)=[a(θ ₁ ),...a(θ _K )]

so

X(t)=A(θ)S(t)+Q(t), Q(t)=[q ₁ (t)...q _N (t)] ^T

where Q(t) is the vector representation of the noise in the received signal. Assume

y(t)=W ^H X(t), W=[w ₁ ...w _N ] ^T

but

P(W)=E(|y(t)| ² )=W ^H E(X(t)X(t) ^H )W=W ^H RW

The Capon method obtains an optimization problem targeting the maximum signal-to-interference ratio by suppressing all undesired angle signal energy while keeping the desired angle signal gain at 1:

min(P(W)) subject to W ^H a(θ)=1

Its Lagrangian objective function is

so

Then according to W ^H a(θ)=1, we get

thereby

Then, by searching for the peak value of P _capon (θ) exceeding the threshold value, the target number K and the corresponding incident angle θ _k estimation can be obtained at the same time.

Various disturbances and noises in practical applications can cause measurement errors in distance, radial velocity, and angle, so these direct measurements need to be filtered to obtain a smooth estimate of the target trajectory. This filtering process can be expressed as: given all past measurements, estimate the posterior probability distribution of the current target state, namely p(x _k |y _1:k ):

where y _1:k represents all measurements before time k, p(x _k |x _k-1 , y _1:k-1 )=p(x _k |x _k-1 ) is based on the Markov assumption, that is, given x _{k -1} , x _k is independent of the earlier state x _1:k-2 and all previous measurements y _1:k-1 .

Assumption

is the position and velocity state of the target in the Cartesian coordinate system,

is the measurement of angle, distance and radial velocity in the polar coordinate system, assuming that the dynamic model of the target in the state space is a constant velocity model, namely

Among them, T is the sampling rate, that is, the time interval between two adjacent measurements, assuming that the distribution of velocity changes is zero-mean Gaussian

but

Then the observation model from the state space to the measurable feature space is the mapping from Cartesian coordinates to polar coordinates

in

For the measurement error, because h(x _k ) is nonlinear, p(y _k |x _k ) is non-Gaussian, so the exact analytical solution of the integral operation in the Bayesian recursive process cannot usually be obtained, but only approximate solutions can be obtained. The following steps can also be applied to other state or measurement space coordinate system selection, such as 3D measurement space

Or other dynamic models like Singer acceleration model etc.

Extended Kalman Filter (EKF) is a Bayesian recursive solution obtained by substituting a first-order linear approximation of the observation model h(x _k ). According to the first-order Taylor expansion of h(x _k ), y _k is

can be approximated as

in,

So p(y _k |x _k ) is in

The approximate Gaussian distribution around it is

According to the Gaussian approximation, the exact analytical solution of the integral operation in the Bayesian recursive process can be obtained, which reduces to the following EKF recursive process:

The initial conditions are

in,

For initial distance, radial velocity and incident angle measurements, initial state

is the mapping of y ₀ in the Cartesian coordinate system, the initial covariance matrix P _0|0 is a diagonal matrix, the parameter

Used to simulate random changes in the initial state.

According to the dynamic model, from the state

Update to

The prediction process is

P _k+1|k =FP _k|k F ^T +GQ _k G ^T

Kalman gain is

estimated from the state according to the measurement y _k+1

Update to

The process is

Recursively iterate the above steps to get the trajectory of the target in the state space

and an estimate of its covariance (P _0|0 , . . . , P _k|k ). Due to the first-order linear approximation, the Kalman filter is not optimal for the estimation of nonlinear moving objects.

In order to obtain the optimal filtering performance, particle filtering can be used. Particle filtering is to substitute the Monte-Carlo approximation of p(x _k |y _1:k ) into the Bayesian recursive process, so as to obtain the numerical solution of the integral operation, because the linearity is removed. It is assumed that it is suitable for the optimal estimation of nonlinearly changing states, and the approximation error decreases as the number of particles increases. Particle filtering takes N particles x _k ⁱ , i ∈ [1, N] and their weights

Approximate p(x _k |y _1:k ) as follows:

Then p(x _k+1 |y _1:k ) can be approximated as

Because p(x _k+1 |y _1:k+1 ) is difficult to sample directly, the particle approximation of p(x _k+1 |y _1:k+1 ) needs to be sampled by importance ( Importance Sampling), and the selection of the optimal proposed distribution needs to include both the current particle approximation and the current measurement information, so usually p(x _k+1 |x _k ⁱ , y _k+1 ) is selected as the proposed distribution, and its particle approximately

Sampling by Importance

in

Therefore, importance sampling reduces the sampling problem of p(x _k+1 |y _{1 : k+1} ) to the sampling problem of the proposed distribution p(x _k+1 |x _k ⁱ , y _k+1 ), and then because

Among them, the non-Gaussian linearity of the observation model p(y _k+1 |x _k+1 ) makes p(x _k+1 |x _k ⁱ , y _k+1 ) difficult to sample directly. However, as described below, referring to the method of linear approximation in EKF, the complex sampling problem can be reduced to Gaussian sampling using the Gaussian approximation of p(x _k+1 |x _k ⁱ , y _k+1 ):

Linear Gaussian approximation of the observation model p(y _k+1 |x _k+1 ) around x _k ⁱ

Substituting into the above EKF recursive process, the approximate analytical solution of p(x _k+1 |x _k ⁱ , y _k+1 ) can be obtained as

where the Kalman gain is

K _k+1 ⁱ −(GQ _k+1 G ^T )H(x _k ⁱ ) ^T (H(x _k ⁱ )(GQ _k+1 G ^T )H(x _k ⁱ ) ^T +R _k+1 ) ^{− 1}

Therefore, the sample points of p(x _k+1 |x _k ⁱ , y _k+1 ) can be obtained by Gaussian sampling.

In summary, the particle filtering process is as follows:

from a Gaussian distribution

Get N sample points, the weight of each sample

as the initial particle approximation.

According to the Gaussian approximation of p(x _k+1 |x _k ⁱ , y _k+1 ), N sample points of the current state and their weights are obtained through Gaussian sampling

Then the particle of p(x _k+1 |y _1:k ) is approximated as

According to the information of the current measurement y _k+1 , update the weight w _k+1|k ^j to w _k+1|k+1 ^j

Therefore

Thus, the current state can be estimated as

Iterate the above process recursively to get the trajectory of the target in the state space

's estimate.

Based on the above working principle and signal processing steps, a block diagram of the millimeter wave radar sensor 101 is shown in FIG. 5 . Wherein, the transmitting circuit 1011 includes but is not limited to:

a. M transmitting antennas (eg M=2), used for radar signal output. All transmit antenna outputs in Figure 5 come from the same signal source, so the output gain can be improved; the signal path of the transmit end in Figure 5 can also be slightly modified to transmit different signals through each or part of the antennas, and the transmit and receive antenna arrays can be adjusted accordingly The spacing of the MIMO radar is used to obtain an M×N virtual SIMO array, thereby improving the spatial resolution. In order to avoid mutual interference of different transmitted signals, code division multiplexing technology, such as Binary Phase Modulation or Hadamard Code, can be used, so that the transmitted signals of each antenna are orthogonal to each other;

b. Power amplifier (PA), linear power amplification, to drive the antenna to send FM signal;

c. Linear frequency modulation continuous waveform generation circuit, including digitally controlled oscillator circuit (DCO) and digital phase-locked loop (ADPLL), the structural block diagram is shown in Figure 6. Among them, the time-to-digital conversion circuit (TDC) is used to measure the DCO output phase, and the accumulator ∑ is used to calculate the reference phase. By comparing the DCO output phase and the reference phase, the phase difference e is obtained, and after loop filtering, it is used as the DCO input to control its Oscillating frequency to drive e=0, thus forming a negative feedback control loop.

As shown in Figure 6, the modulated signal is connected to the control loop through 2 points, in which the h path directly adjusts the DCO output frequency, and the l path is used to offset the change of the phase difference e caused by the direct modulation. From the analysis of the PLL loop transfer characteristics, l and The h paths have low-pass and high-pass characteristics respectively, so by properly combining the two modulated signals, an all-pass characteristic can be obtained. It can be seen that the 2-point injection architecture increases the bandwidth of frequency modulation and reduces linear distortion, thereby improving the linearity of the output signal. Another advantage is that the frequency modulation bandwidth is no longer limited by the loop bandwidth, thereby reducing the loop bandwidth to suppress TDC quantization noise, optimizing system performance and improving measurement accuracy.

As shown in FIG. 5 , the receiving circuit 1012 of the millimeter-wave radar sensor 101 includes N (eg N=4) receiving antennas arranged in a ULA (Uniform Linear Array) or URA (Uniform Rectangular Array) manner, and N receiving channels, The RF front-end for each receive channel includes but is not limited to:

a. Low Noise Amplification (LNA), used to amplify the amplitude of the received reflected radar signal;

b. Mixer, mixing the received signal amplified by the LNA and the transmitted frequency modulation signal to obtain an intermediate frequency signal;

c. Low-pass filtering, setting the filtering bandwidth according to the maximum effective distance or ADC sampling rate, suppressing noise and interference, and avoiding aliasing caused by discrete sampling in the subsequent ADC process;

d. Variable Gain Amplification (VGA) for adjusting the signal amplitude to fully utilize the ADC dynamic range;

e. An analog-to-digital conversion circuit (ADC) converts the input time-amplitude continuous analog signal into a time-amplitude discrete digital signal for further processing by digital signal processing technology (DSP), thereby extracting gesture control commands.

As shown in FIG. 5 , the digital signal processing circuit 1013 of the millimeter-wave radar sensor 101 includes but is not limited to:

a. Ramp generation circuit (Ramp Gen), used to generate sawtooth or triangular modulation waveform to control the DCO oscillation frequency;

b. Window function (Window), which is used to reduce the spectrum expansion caused by FFT truncation, thereby improving the frequency resolution. The discrete sampling values of window functions such as Hamming, Hanning or Blackman can be pre-stored in static random storage (SRAM), and the runtime Read out, multiply with the input digital signal;

c. Fast Fourier Transform (FFT), as mentioned above, the target distance, radial velocity and incidence angle can be estimated by searching for the amplitude peak value after the FFT transform;

d. A first-in, first-out buffer (FIFO) for buffering the Fourier-transformed data for further processing by the processor.

Based on the above working principle and system architecture of the millimeter-wave radar sensor 101, the specific steps of the non-contact control are as follows:

a. The ramp generating circuit of the digital signal processing circuit 1013 in the millimeter-wave radar sensor 101 generates a sawtooth or triangular signal, controls the DCO oscillation frequency in the transmitting circuit 1011, generates a frequency modulation signal, and transmits the radar beam through the M transmitting antennas;

b. The receiving circuit 1012 of the millimeter-wave radar sensor 101 detects the user's gesture movement reflection radar signal through N receiving antennas, and performs low-noise amplification, frequency mixing, and analog-to-digital conversion on the received signal by the radio frequency front-end to obtain a digital intermediate frequency signal;

c. The digital signal processing circuit 1013 of the millimeter-wave radar sensor 101 adds windowing and fast Fourier transform (FFT) to the digital intermediate frequency signal, and writes the result into a first-in, first-out buffer (FIFO) for the processor 103 to read and process;

d. The processor 103 reads the Fourier-transformed data from the first-in, first-out buffer (FIFO) of the digital processing circuit 1013, searches for its peak value, obtains estimates of the distance and radial velocity, and uses the Capon or MUSIC method to estimate the angle of incidence , these measurements will be input into the Kalman filter or particle filter as the observation of the target state space to filter out the interference and noise in the measurement process, and obtain a smooth estimation of the user's gesture trajectory;

e. Based on the tracking of the motion trajectory of the user's gesture, the state feedback component 102 is controlled to output the current state to the user and prompt the next action;

f. After the processor 103 receives the complete control command, it executes the corresponding command, such as controlling the motion driving part 104 to realize the opening and closing of the door; or when the processor 103 receives the cancel command, it ends the current command.

In order to meet the requirements of improving the anti-interference ability of the system and avoiding false triggers caused by interference, it is necessary to define relatively complex gestures as control commands, and processing complex gestures will increase system cost and power consumption. For example, complex gesture recognition may require pattern recognition. Or machine learning technology, which requires larger random storage to store the trained gesture model, and stronger and faster computing resources. In addition, complex gestures are not intuitive, and are not easy to remember and use.

In order to improve the convenience of use without reducing the anti-interference ability, complex gestures can be decomposed into multiple basic gesture sequences and confirmed one by one during use. Therefore, it is necessary to combine the millimeter wave radar sensor 101 to receive user gesture commands to realize the input function. The function of outputting the state with the state feedback component 102 to achieve simple and convenient human-computer interaction.

Therefore, based on the above non-contact control steps, the control gesture definitions and interaction methods are as follows:

a. Proximity to the sensor

Due to the limitation of the transmit power of the millimeter-wave radar sensor 101 and the change of the reflection characteristics of the surrounding environment, the gesture action is required to be within the effective range to ensure a sufficient signal-to-noise ratio and avoid false triggering caused by interference. Therefore, when the system tracks the target trajectory and detects that the target has approached within the set threshold range, the state feedback component 102 confirms that the system has been activated in the form of light or sound, ready to receive the next command, and can prompt the next step Valid commands, the movement direction of the following gesture trajectory. This prompt is similar to the auto-complete function of some text editors, which can improve the efficiency of interaction.

b. Control commands

When the user judges that it is close enough according to the light or sound fed back by the status feedback component 102, and according to the next valid command prompt, the user can continue the next gesture action, including: swipe from left to right, swipe from right to left, and from top to bottom Swipe, swipe from bottom to top, press, lift, circle, tick, cross, etc., or end the current command by moving away from the sensor.

The system tracks the target trajectory, and updates the current state in real time through the state feedback component 102 until a complete control command is detected, such as the up-down or left-right sliding angle exceeds the set threshold, then confirms and executes the current command, such as driving the motion drive component 104 to switch The opening and closing state of the door.

In some application scenarios, when only the horizontal or vertical sliding gesture command needs to be detected, in order to improve the sensitivity and accuracy of the measurement, the ULA arrangement direction of the receiving antenna should be consistent with the sliding direction of the gesture command, as shown in Figure 7. If it is necessary to support both left and right And sliding up and down, multiple millimeter-wave sensors can be used to detect horizontal and vertical sliding respectively, or the receiving antennas can be arranged in a URA manner to detect horizontal and vertical sliding at the same time.

c. away from the sensor

When it is detected that the target trajectory exceeds the set threshold range, the current command is ended.

Pressing and lifting, as shown in Figure 8, can simulate key actions, and with sliding gestures, specific electronic control panels can be operated non-contact, such as the up and down keys on the elevator panel.

d. According to specific application scenarios, combine steps a, b and c to obtain multi-segment control gesture definitions

Approach sensor + press lift + move away from sensor, move toward sensor + swipe left to right + move away from sensor, move toward sensor + swipe right to left + move away from sensor, move toward sensor + swipe from top to bottom + move away from sensor, move toward sensor+ Swipe from bottom to up + away from the sensor, close to the sensor + swipe from right to left + press to lift + away from the sensor, etc.

FIG. 9 is a multi-segment control command sequence and a schematic diagram of the state transition of the control system in the interactive process. According to the control command sequence and interaction process defined in FIG. 9 , the interaction process of multi-segment control commands is described by taking the application in the design of the elevator panel non-contact control system as an example.

As shown in FIG. 10, the status feedback component 102 includes 3 LEDs: upstairs, downstairs, and active, which are respectively represented as arrows on the solid line, down arrows, and borders to output the current status; according to the arrangement direction of the status feedback components 102, The control command sequence is defined as: approach the sensor + control command + move away from the sensor, where the control command includes: slide from top to bottom, slide from bottom to top, press and lift. Because only the vertical sliding gesture command needs to be detected here, the receiving antennas of the millimeter-wave radar are arranged in the vertical direction, as shown in the shaded square in Figure 10. Note that this does not indicate the actual size of the millimeter-wave receiving antennas, but only the arrangement direction.

The upstairs control interaction process is as follows:

a. The idle state indication is that none of the 3 LEDs are on;

b. When the user reaches out and approaches the elevator panel within the range of the set threshold, it enters the activation state, and the activation LED lights up; slides the hand upward to exceed the set threshold, and enters the execution state, and the upstairs LED reports that the upstairs has been received with brightness 1 or color 1 Request, return to the active state; press and lift the hand, enter the execution state again, and send the upstairs request to the elevator dispatching system, the upstairs LED indicates that the elevator dispatching system has been notified of the upstairs request and returns to the active state with brightness 2 or color 2;

c. The hand leaves the elevator panel and returns to the idle state, the activation LED does not light up, but the upstairs LED keeps brightness 2 or color 2;

d. If you are away from the elevator panel before the control system confirms the press to lift, the elevator dispatching system will not be notified of the up/down request, and all LEDs will not light up when returning to the idle state.

The control interaction process of going downstairs is similar, except that the sliding direction is from top to bottom.

According to the design method in Figure 9, the above interaction process can also be simplified as follows: only use upstairs and downstairs LEDs, and omit the activation LEDs:

a. In idle state, the upstairs and downstairs LEDs do not light up;

b. When the user reaches out and approaches the elevator panel within the set threshold range, it enters the active state. According to the area pointed to by the detected gesture trajectory extension, assuming that the gesture trajectory points to the upstairs LED, the upstairs LED is lit with brightness 1 or color 1. Confirm that the request to go upstairs has been received;

c. If you really want to go upstairs, press along the current track to enter the execution state, send an upstairs request to the elevator dispatching system, and light up the upstairs LED with brightness 2 or color 2, indicating that the elevator dispatching system has been notified of the upstairs request, and return active state;

d. If you actually want to go downstairs, slide from top to bottom to enter the execution state, turn off the upstairs LED, light the downstairs LED with brightness 1 or color 1, confirm that the downstairs request has been received, return to the active state, and press , enter the execution state again, and light up the upstairs LED with brightness 2 or color 2, indicating that the elevator dispatch system has been notified of the downstairs request, and returns to the active state

e. If you move away from the elevator panel before the control system confirms pressing the gesture, the elevator dispatch system will not be notified of the request to go up/down the stairs, return to the idle state, and all LEDs will not light up.

Claims (21)

1. An intelligent door using millimeter-wave radar to realize non-contact control, comprising: a door frame (1), a door panel (2), a moving part (3) that drives the door panel (2) to open and close, and a control system (10); it is characterized in that, The control system (10) includes:

   A millimeter wave radar sensor (101), which transmits a radar beam and receives a reflected radar beam to detect user gesture commands;

   a state feedback component (102), used for feeding back state information to the user to realize state interaction;

   The processor (103) is connected to the millimeter wave radar sensor (101), the state feedback part (102) and the motion driving part (104); according to the user gesture command received by the millimeter wave radar sensor (101), it controls the state feedback part ( 102) Output status information and perform corresponding control operations.
2. The smart door according to claim 1, characterized in that: the control operation is to control the motion of the motion driving component (104) to drive the motion component (3) to drive the door panel (2) to move, so as to realize the opening and closing of the smart door.
3. The smart door according to claim 1, characterized in that: the millimeter-wave radar sensor (101) comprises a transmitting circuit (1011) that transmits the radar beam, a receiving circuit (1012) that receives the reflected radar beam, and a digital signal Processing circuit (1013).
4. The smart door according to claim 3, characterized in that: the transmitting circuit (1011) comprises:

   M transmit antennas for emitting the radar beam, configured in a phased array mode or a MIMO mode;

   Power amplifier, linear power amplification to drive the transmitting antenna;

   Digitally controlled oscillator circuit and digital phase-locked loop to achieve frequency modulation.
5. The smart gate according to claim 4, characterized in that: in order to improve the modulation bandwidth and linearity, the digitally controlled oscillator circuit and the digital phase-locked loop adopt a 2-point injection structure.
6. The smart door according to claim 3, wherein the receiving circuit (1012) comprises:

   N receiving antennas, arranged in ULA or URA manner;

   Low-noise amplifying LNA for amplifying the amplitude of the received signal;

   Mixing, mixing the received signal amplified by the LNA and the local FM signal to obtain an intermediate frequency signal;

   Low-pass filtering and variable gain amplification;

   The analog-to-digital conversion circuit converts the analog signal into a digital intermediate frequency signal.
7. The smart door according to claim 3, characterized in that: the digital signal processing circuit (1013) comprises:

   The ramp generating circuit generates sawtooth or triangular signals to adjust the oscillation frequency of the numerically controlled oscillator circuit;

   Fast Fourier transform FFT, the target distance, radial velocity and incident angle are obtained by searching for the peak value of the signal amplitude after FFT transformation;

   Window function to reduce spectral spreading due to FFT truncation;

   A first-in-first-out buffer FIFO is used to buffer the FFT transformed data for further processing by the processor (103).
8. The smart door according to any one of claims 1-7, characterized in that: the control system (10) further comprises a communication interface (105), which is connected with the processor (103) to realize the control system (10) and the external environment interconnection and information interaction.
9. The smart door according to any one of claims 1-7, characterized in that: the control system (10) can have the following physical form combinations according to application requirements: PCB level integration based on surface mount technology, package level based on SIP technology Integrated, or silicon-level integration based on SOC technology.
10. The smart door according to any one of claims 1-7, characterized in that: two sets of the millimeter-wave radar sensor (101) and the state feedback component (102) in the control system (10) are respectively provided, which are respectively used for The user gesture commands and state interaction on the inside and outside of the door are processed, the processing is shielded between the two sets of the millimeter wave radar sensors (101), and the priority of the two sets of the millimeter wave radar sensors (101) needs to be set.
11. A non-contact control method for controlling the smart door according to any one of claims 1-10, characterized in that, comprising the steps of:

    a. The millimeter wave radar sensor (101) transmits a radar beam;

    b. The millimeter-wave radar sensor (101) receives the radar signal reflected by the user's gesture movement and performs preliminary processing and buffering for the processor (103) to read and process;

    c. The processor (103) reads out the preliminarily processed data, obtains the estimation of the target distance, radial velocity, and incident angle, and obtains the smooth estimation of the user's gesture motion trajectory through filtering processing;

    d. Based on the tracking of the motion trajectory of the user's gesture, the processor (103) controls the state feedback component (102) to output the current state;

    e. Continue to run steps a-d. After receiving the complete control command, the processor (103) executes the corresponding control operation, or cancels the execution of the command when the premature end command is detected.
12. The control method according to claim 11, wherein: the user gesture movement is a complex control gesture, and the processor (103) decomposes the complex control gesture into multiple basic gesture sequences and confirms them segment by segment.
13. The control method according to claim 12, characterized in that the step-by-step confirmation based on the multi-segment basic gesture sequence includes a plurality of interaction processes of mutual confirmation between the user and the control system (10), starting when the user approaches the millimeter-wave radar sensor (101). , and end away from the millimeter wave radar sensor (101). Each time the interaction is confirmed, the processor (103) executes corresponding actions, and multiple basic gestures are progressively confirmed to realize a complete control command.
14. The control method according to claim 11, wherein step d comprises:

    d1. The processor (103) tracks the motion trajectory of the user's gesture, and detects that the user has approached within the set threshold range, then confirms that the system has been activated in the form of light or sound through the state feedback component (102), and is ready to receive the next step Order;

    d2. When the user judges that it is close enough according to the feedback from the state feedback component (102), continue the next gesture action; or end the current command by moving away from the millimeter wave radar sensor (101).
15. The control method according to claim 11, characterized in that: in the above step b, the radio frequency front end of the receiving circuit (1012) of the millimeter-wave radar sensor (101) performs low-noise amplification, frequency mixing, and analog-to-digital conversion on the received reflected signal , to get a digital IF signal.
16. The control method according to claim 15, characterized in that: the digital signal processing circuit (1013) of the millimeter-wave radar sensor (101) adds windowing and fast Fourier transform (FFT) to the digital intermediate frequency signal, and writes the result into A first-in, first-out buffer (FIFO) for the processor (103) to read and process.
17. The control method according to claim 16, characterized in that: the processor (103) reads out the Fourier-transformed data from the first-in, first-out buffer (FIFO) of the digital processing circuit (1013), searches for its peak value, and obtains Estimation of target distance, radial velocity, and angle of incidence with Capon or MUSIC methods, these measurements will be input as observations of the target state space into a Kalman filter or particle filter, resulting in a smooth estimate of the user's gesture trajectory.
18. The control method according to claim 11, characterized in that: using complex control gestures as control commands, collecting a large number of motion trajectories as training data, and using pattern recognition or machine learning technology to obtain models representing control gesture features, so as to improve the Recognition rate of complex control gesture commands.
19. A non-contact gesture control system, comprising: a millimeter-wave radar sensor (101), transmitting a radar beam, and receiving a reflected radar beam to detect a user gesture command; it is characterized in that, further comprising:

    a state feedback component (102), used for feeding back state information to the user to realize state interaction;

    a processor (103), which is connected to the millimeter wave radar sensor (101) and the state feedback component (102); the processor (103) detects the user gesture command in real time, and feeds back the execution state in real time, and decomposes complex gestures It is a sequence of several simple gestures, and is confirmed one by one. After receiving the complete user gesture command, the corresponding control operation is performed.
20. The system according to claim 19, wherein the recognition of each of the user gesture commands by the processor (103) includes a number of interactive processes for mutual confirmation between the user and the control system, so that the user approaches the millimeter-wave radar sensor (101) starts, and ends away from the millimeter-wave radar sensor (101), each time the interaction is confirmed, the processor (103) performs corresponding actions, so that the recognition of complex gesture commands is decomposed into multiple basic gesture recognition and step by step Confirmed progressive steps.
21. A method for realizing non-contact control by utilizing the control system described in any one of claims 19-20, characterized in that it comprises the steps of:

    a. The millimeter wave radar sensor (101) transmits a radar beam;

    b. The millimeter-wave radar sensor (101) receives the radar signal reflected by the user's gesture movement;

    c. The processor (103) controls the state feedback component (102) to output the current state based on the tracking of the motion trajectory of the user's gesture;

    d. The processor (103) realizes the recognition of complex gestures by confirming several simple gesture sequences segment by segment;

    e. Continue to run steps a-d. After receiving the complete control command, the processor (103) executes the corresponding control operation, or cancels the execution of the command when the premature end command is detected.

Images