CN113177581A - Liquid crystal screen content remote sensing method based on millimeter wave sensing - Google Patents

Liquid crystal screen content remote sensing method based on millimeter wave sensing Download PDF

Info

Publication number
CN113177581A
CN113177581A CN202110409718.6A CN202110409718A CN113177581A CN 113177581 A CN113177581 A CN 113177581A CN 202110409718 A CN202110409718 A CN 202110409718A CN 113177581 A CN113177581 A CN 113177581A
Authority
CN
China
Prior art keywords
liquid crystal
screen
frequency modulation
crystal screen
millimeter wave
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110409718.6A
Other languages
Chinese (zh)
Other versions
CN113177581B (en
Inventor
许文曜
林峰
李勤
李正雄
陈百成
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Huanmu Information Technology Co ltd
Original Assignee
Hangzhou Huanmu Information Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Huanmu Information Technology Co ltd filed Critical Hangzhou Huanmu Information Technology Co ltd
Priority to CN202110409718.6A priority Critical patent/CN113177581B/en
Publication of CN113177581A publication Critical patent/CN113177581A/en
Application granted granted Critical
Publication of CN113177581B publication Critical patent/CN113177581B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
    • G06F18/2132—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods based on discrimination criteria, e.g. discriminant analysis
    • G06F18/21322—Rendering the within-class scatter matrix non-singular
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/254—Fusion techniques of classification results, e.g. of results related to same input data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
    • G06F18/2132—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods based on discrimination criteria, e.g. discriminant analysis
    • G06F18/21322—Rendering the within-class scatter matrix non-singular
    • G06F18/21324—Rendering the within-class scatter matrix non-singular involving projections, e.g. Fisherface techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
    • G06F2218/08—Feature extraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
    • G06F2218/12—Classification; Matching

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Artificial Intelligence (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Probability & Statistics with Applications (AREA)
  • Radar Systems Or Details Thereof (AREA)

Abstract

The invention discloses a liquid crystal screen content remote sensing method based on millimeter wave sensing. The method comprises the following steps: millimeter wave sensing of screen liquid crystal orientation diagrams: transmitting frequency modulation continuous waves to the surface of the liquid crystal screen by using a millimeter wave probe, and then receiving the frequency modulation continuous waves with liquid crystal nematic information reflected by the surface of the liquid crystal screen; the millimeter wave probe converts a received periodic linear frequency modulation signal of the frequency modulated continuous wave into an intermediate frequency signal, and the intermediate frequency signal is subjected to fast Fourier transform and motion compensation to obtain displacement information of the liquid crystal screen; removing background clutter dynamic clutter of liquid crystal screen displacement information; extracting screen liquid crystal characteristics from the range-Doppler matrix; and inputting the extracted screen liquid crystal characteristics into a fine-grained authentication model to complete screen content perception. The invention realizes a remote wall-through liquid crystal screen image acquisition system, can remotely acquire complete screen content, and can be conveniently transferred to wireless sensing modules of mobile equipment such as smart phones.

Description

Liquid crystal screen content remote sensing method based on millimeter wave sensing
Technical Field
The invention relates to the technical field of wireless, in particular to a liquid crystal screen content remote sensing method based on millimeter wave sensing.
Background
Because the use of the liquid crystal screen is directed at human visual sense, the liquid crystal nematic adopted by the liquid crystal screen has different arrangement and combination states under different electromagnetic factory states, thereby achieving different visible light transmission and color effects. Due to the development of electronic device technology, liquid crystal screens are ubiquitous in electronic devices throughout the world. And the output of the screen is only in a visible light spectrum, so that more screen interaction resources are limited. The millimeter wave sensing in the wireless technical field can reach a long distance, can pass through a wall liquid crystal screen content acquisition system, and can share screen content in more aspects under the condition of not influencing the existing screen use mode.
Disclosure of Invention
In order to solve the problems in the background art, the invention provides a remote sensing method of liquid crystal screen content, which is based on millimeter wave sensing (mmWave), is remote, accurately oriented and can be worn on a wall.
The technical scheme adopted by the invention is as follows:
a liquid crystal screen content remote sensing method based on millimeter wave sensing comprises the following steps:
step one, sensing millimeter waves of a screen liquid crystal orientation chart:
firstly, transmitting Frequency Modulated Continuous Waves (FMCW) to the surface of a liquid crystal screen by using a high-precision millimeter wave probe, wherein the sensing is non-contact, and then receiving the frequency modulated continuous waves with liquid crystal nematic information reflected by the surface of the liquid crystal screen; the millimeter wave probe converts received periodic linear frequency modulation signals of frequency modulation continuous waves into Intermediate Frequency (IF) signals, and fast Fourier transform and motion compensation are carried out on the IF signals to obtain displacement information of the liquid crystal screen.
The specific process of obtaining the displacement information of the liquid crystal screen according to the intermediate frequency signal comprises the following steps:
1.1) setting a chirp period to TrObtaining the m-th linear tone according to the intermediate frequency signalLiquid crystal screen displacement d (mT) of frequency periodr) The calculation formula of (2) is as follows:
Figure BDA0003023676520000011
wherein M is the number of a chirp cycle, and M is 0,1,2, M-1; m is the number of chirp cycles; f. of0Is the carrier frequency; delta psimIs the result of performing Fast Fourier Transform (FFT) on the intermediate frequency signal of the mth chirp period; thereby obtaining the displacement of the liquid crystal screen with M linear frequency modulation periods.
ΔψmIncluding the distance information of the mth chirp period.
1.2) in order to avoid the interference of the random motion of the screen carrier on the result of the step 1.1), performing motion compensation on the displacement of the liquid crystal screen with M linear frequency modulation periods obtained in the step 1.1) through translational change, namely calibrating the distance information of the linear frequency modulation periods:
definition of Sm(l) For the distance information of the mth chirp period obtained in step 1.1),
Figure BDA0003023676520000028
distance information of the m-th linear frequency modulation period after calibration;
l is 0,1,2, and L-1, where L is the number of sampling points in the mth chirp period, and L is the number of sampling points;
definition of Qm(l) The relative distance information of the mth chirp cycle specifically includes:
Figure BDA0003023676520000021
wherein Q ism-1(l) Relative distance information of the (m-1) th chirp cycle;
the relative distance information Qm(l) For calibrated distance information
Figure BDA0003023676520000022
Distance information S obtained in step 1.1)m(l) The relative distance therebetween.
Definition of XmIs Sm(l) The distance of the translation of (a) is,
Figure BDA0003023676520000023
for optimum translation distance, based on relative distance information Qm(l) Calculating an optimal translation distance
Figure BDA0003023676520000024
Optimal translation distance
Figure BDA0003023676520000025
By maximizing the following equation:
Figure BDA0003023676520000026
wherein S ism-1(l-Xm) Representing the distance information after the m-1 linear frequency modulation period translation;
the distance information after the mth chirp period calibration is:
Figure BDA0003023676520000027
wherein j is a unit complex number and Δ is a vector [0, 1.., L-1 ];
and obtaining the calibrated distance information of the M linear frequency modulation periods according to the calibrated distance information of the M linear frequency modulation periods, and then obtaining the displacement information of the liquid crystal screen after motion compensation according to the calibrated distance information of the M linear frequency modulation periods.
Step two, clutter suppression processing:
performing second fast Fourier transform on the displacement information of the liquid crystal screen in the first step to obtain a range-Doppler matrix, and completing separation of background clutter and removal of dynamic clutter by updating the range-Doppler matrix;
the second step is specifically as follows:
2.1) separating background clutter:
we consider the background clutter as being generated by the influence of a large number of small objects, the amplitude and phase of these background clutter are random, but their spectral envelope a follows a weber distribution:
Figure BDA0003023676520000031
they can perform clutter separation as follows.
Performing second fast Fourier transform on the distance information after the M linear frequency modulation cycles are calibrated in the step 1 to obtain a distance Doppler matrix (RDM), wherein the distance Doppler matrix comprises speed information, and then performing logarithmic regularization on the distance Doppler matrix;
calculating the threshold mu by substituting the set clutter separation ratio into the clutter separation ratio defining formula0The clutter separation rate is artificially set, and the clutter separation rate is defined by the formula:
Figure BDA0003023676520000032
wherein gamma is Euler constant, pc is clutter separation rate;
defining an elastic matrix
Figure BDA0003023676520000033
Each element in the elastic matrix
Figure BDA0003023676520000034
The definition is as follows:
Figure BDA0003023676520000035
wherein the content of the first and second substances,
Figure BDA0003023676520000036
represents a matrix of range-doppler signals,
Figure BDA0003023676520000037
is the jth element, R, of the ith row in the elastic matrixijFor the jth element in the ith row of the range-doppler matrix,
Figure BDA0003023676520000038
in order to estimate the error in an unbiased way,
Figure BDA0003023676520000039
for distance Doppler matrix
Figure BDA00030236765200000310
Updating is carried out, so that background clutter separation is completed, and the updating specific process is as follows:
Figure BDA00030236765200000311
j is a 01 matrix, o is a Hadamard product, and sgn is a sign function.
2.2) removing dynamic clutter caused by dynamic obstacles detected by frequency-modulated continuous waves:
since the dynamic clutter does not follow the weber distribution, the above steps cannot remove the dynamic clutter. The present invention uses information across multiple RDMs to eliminate dynamic clutter.
Continuously transmitting Frequency Modulation Continuous Waves (FMCW) to obtain n continuous range-Doppler matrixes, wherein each range-Doppler matrix comprises M linear frequency modulation cycles, calculating the average moving speed of the obstacles in the range-Doppler matrixes, and removing the value corresponding to the obstacles with the average moving speed being greater than the range profile resolution delta RES from the range-Doppler matrixes to further remove the dynamic clutter.
The value corresponding to the obstacle is the moving speed of the obstacle.
Step three, extracting screen liquid crystal characteristics:
extracting the screen liquid crystal arrangement characteristics and the screen liquid crystal arrangement detail characteristics related to the screen content from the distance Doppler matrix subjected to impurity removal in the second step;
3.1) extracting the arrangement detail features of the screen liquid crystal: the frequency modulated signal represents the nematic composition of the screen liquid crystal, and the associated geometric parameters. The nematic combination of liquid crystals not only provides a channel for screen backlighting, but also has a unique nonlinear flow pattern. The phase information of the screen liquid crystal frequency modulation potential excitation waveform is represented by Residual Phase Cepstrum Coeffients (RPCC). And measuring the spectral amplitude characteristic of the liquid crystal nematic combination angle information corresponding to the screen brightness by using a Teager Phase Cepstrum Coefficients (TPCC).
3.2) extracting the alignment characteristics of the screen liquid crystal: liquid crystal frequency modulation characteristics used in the present invention include spectral characteristics (e.g., centroid, energy band, peak, flatness, and entropy), Mel-frequency cepstral coefficients (MFCCs), Linear Predictive Coefficients (LPCs), Linear Predictive Cepstral Coefficients (LPCCs), and line spectral frequencies (LSPs).
Step four, screen content perception:
and (4) constructing a fine-grained authentication model, and inputting the screen liquid crystal frequency modulation characteristics extracted in the third step into the fine-grained authentication model to complete screen content perception.
The fine-grained authentication model in the fourth step comprises a Fisher Score (Fisher Score) -based feature selection module and a screen content-aware ensemble classifier;
1) the specific functions of the characteristic selection module based on the Fisher's score are as follows: selecting a feature subset with the maximum Fisher score from the original features by using a cutting plane algorithm;
the original characteristics are screen liquid crystal arrangement characteristics and screen liquid crystal arrangement detail characteristics;
2) the integrated classifier is obtained by fusing algorithms of three models, namely a Gaussian mixture model, a general background model, a support vector machine and a hidden Markov model;
the specific functions of the integrated classifier are as follows: inputting the feature subset with the maximum Fisher score in the original features into the integrated classifier, adjusting the weights of the three models of the integrated classifier by using a regression model, performing weighted summation on output results of the three models according to the adjusted weights, and finally outputting a classification result to finish screen content perception.
Training a fine-grained authentication model: selecting a liquid crystal screen sample with known screen content, extracting screen liquid crystal characteristics of the liquid crystal screen sample according to the first step to the third step to serve as a sample data set, and dividing the sample data set into a training set and a testing set to train the fine-grained authentication model.
The training optimization of the cutting plane algorithm in the feature selection module uses a ridge regression algorithm and a gradient descent algorithm.
A liquid crystal screen content remote sensing system based on millimeter wave sensing comprises a hardware module and a software module, wherein the hardware module comprises a millimeter wave sensing module and a clutter suppression processing module for a screen liquid crystal direction chart, and the software module comprises a screen content feature extraction module and a screen content sensing module.
The invention has the beneficial effects that:
the invention uses millimeter waves to directionally acquire and analyze the liquid crystal nematic image of the screen, and realizes an end-to-end screen content perception system for acquiring millimeter wave information, processing signals, extracting characteristics to display image content on the liquid crystal screen.
The invention realizes a remote wall-penetrating liquid crystal screen image acquisition system based on a portable hardware system with low energy consumption, can remotely acquire complete screen content, and can be conveniently transferred to wireless sensing modules of mobile equipment such as smart phones and the like.
Drawings
FIG. 1 is a diagram of a system employing the method of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings.
As shown in fig. 1, the present invention includes a hardware module and a software module, wherein the hardware module includes a millimeter wave sensing module and a clutter suppression processing module for a screen liquid crystal direction diagram, and the software module includes a screen content feature extraction module and a screen content sensing module.
We collected 30 different screens for use. These screens collectively comprise 6 categories, computer screens, notebooks, smart phones, tablets, smart watches. A total of 21 LCD screens and 9 OLED screens, the size of the screens varying from 1.5 inches to 70 inches, with a service life covering 1 year to 15 years. Then, we aim the millimeter wave probe at the screen direction, and measure the distance from 80 cm to 5 m.
1. The millimeter wave probe converts a received periodic linear frequency modulation signal of the frequency modulation continuous wave into an Intermediate Frequency (IF) signal, and displacement information of the liquid crystal screen is calculated according to the IF signal;
1.1) setting a chirp period to TrObtaining the displacement d (mT) of the liquid crystal screen in the mth linear frequency modulation period according to the Intermediate Frequency (IF) signalr) Specifically, the calculation is;
Figure BDA0003023676520000051
wherein M is the number of a chirp cycle, and M is 0, 1. M is the number of chirp periods, f0Is the carrier frequency; delta psimIs the result of performing Fast Fourier Transform (FFT) on the intermediate frequency signal of the mth chirp period;
1.2) random movement of the screen carrier is the main obstacle to obtaining accurate screen liquid crystal nematic data, and when the screen movement amplitude is larger than half of the range profile resolution, it is difficult to accurately calculate delta psim. To avoid the effect of random motion of the screen carrier on the result of step 1.1) to obtain accurate fourier transform results of the IF signal, we use a fine-grained distance bin alignment method to solve this problem.
Definition of Sm(l) The distance information of the mth chirp cycle is obtained, where M is 0, 1., M-1, 1 is 0, 1., L-1, M is the number of signal sequences, and L is the number of samples of the distance.
Figure BDA0003023676520000052
The value of (d) represents the calibrated distance magnitude.
Relative distance information Q defining mth distance informationm(l) The past calibration information is made available as knowledge. Qm(l) The formalized definition is:
Figure BDA0003023676520000061
definition of XmIs Sm(l) The distance of the translation of (a) is,
Figure BDA0003023676520000062
for the optimal translation distance, the optimal translation distance is obtained by maximizing the following formula
Figure BDA0003023676520000063
Figure BDA0003023676520000064
Finally, calibrated distance information can be obtained:
Figure BDA0003023676520000065
where Δ is the vector [0, 1.., L-1 ].
2. After accurate Fourier transform data are obtained, background clutter conforming to Weber distribution is removed by using an elastic algorithm. A second fast Fourier transform is performed on the Fourier transform of the IF signal to obtain a range-Doppler matrix (RDM). The clutter is then separated using a log-normalized doppler matrix.
Calculating a threshold value mu by substituting an artificially set clutter separation ratio into a clutter separation ratio defining formula0The clutter separation ratio is defined by the formula:
Figure BDA0003023676520000066
wherein gamma is Euler constant, pc is clutter separation rate;
defining an elastic matrix
Figure BDA0003023676520000067
Figure BDA0003023676520000068
Representing a doppler matrix.
For distance Doppler matrix
Figure BDA00030236765200000610
Updating is carried out, and the specific process is as follows:
Figure BDA0003023676520000069
j is a 01 matrix, o is a Hadamard product, and sgn is a sign function.
Finally, a range-doppler matrix (RDM) after clutter separation can be obtained. However, since the dynamic clutter detected by the frequency modulated continuous wave does not follow the weber distribution, the above steps cannot remove the dynamic clutter. The information across multiple RDMs is then used to eliminate dynamic clutter. The range-doppler matrix obtained by the quadratic fourier transform contains velocity information, the average moving velocity of the target in the doppler matrices is calculated, and an object with the velocity greater than the resolution of the range profile is removed, so that the removal of the dynamic clutter is realized.
3. Extracting screen content features: extracting screen liquid crystal frequency modulation characteristics (RPCC, TPCC) and screen liquid crystal frequency modulation detail characteristics (frequency spectrum characteristics, MFCC, LPC, LPCC, LSP);
4. constructing a fine-grained authentication model
The fine-grained authentication model comprises a Fisher Score (Fisher Score) -based feature selection module and a screen content-aware ensemble classifier;
training a fine-grained authentication model: selecting a liquid crystal screen sample with known screen content, extracting screen liquid crystal characteristics of the liquid crystal screen sample to serve as a data set, dividing the data set into hundreds of thousands of samples by using a segmentation algorithm based on Zero crossing Rate and Root Mean Square (the Zero Cross Rate and Root Mean Square in), randomly dividing the samples into a training set and a testing set, and training a fine-grained authentication model.
Screen content perception: firstly, selecting a subset with the maximum Fisher score in the screen liquid crystal arrangement characteristics and the screen liquid crystal arrangement detail characteristics by using a secant plane algorithm, and using a ridge regression algorithm and a gradient descent algorithm in the optimization process of the secant plane algorithm. And then classifying by using algorithm fusion models of three models, namely a Gaussian mixture model-general background model (GMM-UBM), a support vector machine and a hidden Markov model. We adjust the weights of the three models by using a regression model and classify with weighted summation to achieve perception of common screen content.
The average perception accuracy of the screen content can be more than 93% by selecting 100 common screen contents of the known screen contents as a test.

Claims (7)

1. A liquid crystal screen content remote sensing method based on millimeter wave sensing is characterized by comprising the following steps:
step one, sensing millimeter waves of a screen liquid crystal orientation chart:
firstly, emitting frequency modulation continuous waves to the surface of a liquid crystal screen by using a millimeter wave probe, and then receiving the frequency modulation continuous waves with liquid crystal nematic information reflected by the surface of the liquid crystal screen; the millimeter wave probe converts a received periodic linear frequency modulation signal of the frequency modulated continuous wave into an intermediate frequency signal, and the intermediate frequency signal is subjected to fast Fourier transform and motion compensation to obtain displacement information of the liquid crystal screen;
step two, clutter suppression processing:
performing second fast Fourier transform on the displacement information of the liquid crystal screen in the first step to obtain a range-Doppler matrix, and completing separation of background clutter and removal of dynamic clutter by updating the range-Doppler matrix;
step three, extracting screen liquid crystal characteristics:
extracting the screen liquid crystal arrangement characteristics and the screen liquid crystal arrangement detail characteristics related to the screen content from the distance Doppler matrix subjected to impurity removal in the second step;
step four, screen content perception:
and (4) constructing a fine-grained authentication model, and inputting the screen liquid crystal characteristics extracted in the third step into the fine-grained authentication model to complete screen content perception.
2. The method for remotely sensing the content of the liquid crystal screen based on the millimeter wave sensing as claimed in claim 1, wherein the specific process of performing fast fourier transform and motion compensation according to the intermediate frequency signal to obtain the displacement information of the liquid crystal screen in the step one is as follows:
1.1) setting a chirp period in a periodic chirp signal to TrObtaining the displacement d (mT) of the liquid crystal screen of the mth linear frequency modulation period according to the intermediate frequency signalr) The specific calculation formula is as follows:
Figure FDA0003023676510000011
wherein M is the number of a chirp cycle, and M is 0,1,2 …, and M-1; m is the number of chirp cycles; f. of0Is the carrier frequency; delta psimThe method is a result of performing fast Fourier transform on the intermediate frequency signal of the mth linear frequency modulation period;
so that the liquid crystal screen displacement d (mT) according to the mth chirp periodr) Obtaining the displacement of the liquid crystal screen with M linear frequency modulation periods;
1.2) in order to avoid the interference of the random motion of the screen carrier on the result of the step 1.1), motion compensation is carried out on the displacement of the liquid crystal screen with M linear frequency modulation periods obtained in the step 1.1) through translation change, namely, the distance information of the M linear frequency modulation periods is calibrated:
definition of Sm(l) For the distance information of the mth chirp period obtained in step 1.1),
Figure FDA0003023676510000021
distance information of the m-th linear frequency modulation period after calibration;
l is 0,1,2, and L-1, where L is the number of sampling points in the mth chirp period, and L is the number of sampling points;
definition of Qm(l) The relative distance information of the mth chirp cycle specifically includes:
Figure FDA0003023676510000022
wherein Q ism-1(l) Relative distance information of the (m-1) th chirp cycle;
definition of XmIs Sm(l) The distance of the translation of (a) is,
Figure FDA0003023676510000023
for optimum translation distance, based on relative distance information Qm(l) Calculating an optimal translation distance
Figure FDA0003023676510000024
Optimal translation distance
Figure FDA0003023676510000025
By maximizing the following equation:
Figure FDA0003023676510000026
wherein S ism-1(l-Xm) Representing the distance information after the m-1 linear frequency modulation period translation;
the distance information after the mth chirp period calibration is:
Figure FDA0003023676510000027
where j is the unit complex number and Δ is the vector [0,1, …, L-1 ];
and obtaining the calibrated distance information of the M linear frequency modulation periods according to the calibrated distance information of the M linear frequency modulation periods, and then obtaining the displacement information of the liquid crystal screen after motion compensation according to the calibrated distance information of the M linear frequency modulation periods.
3. The method for remotely sensing the content of the liquid crystal screen based on the millimeter wave sensing as claimed in claim 1, wherein the second step is specifically as follows:
2.1) separating background clutter:
performing second fast Fourier transform on the distance information after the M linear frequency modulation cycles are calibrated in the step 1 to obtain a distance Doppler matrix, and then performing logarithmic regularization on the distance Doppler matrix;
calculating the threshold mu by substituting the set clutter separation ratio into the clutter separation ratio defining formula0The clutter separation ratio is defined by the formula:
Figure FDA0003023676510000028
wherein γ is the Euler constant, pcThe clutter separation rate;
defining an elastic matrix
Figure FDA0003023676510000029
Each element in the elastic matrix
Figure FDA00030236765100000210
The definition is as follows:
Figure FDA00030236765100000211
wherein the content of the first and second substances,
Figure FDA0003023676510000031
represents a matrix of range-doppler signals,
Figure FDA0003023676510000032
is the jth element, R, of the ith row in the elastic matrixijFor the jth element in the ith row of the range-doppler matrix,
Figure FDA0003023676510000033
in order to estimate the error in an unbiased way,
Figure FDA0003023676510000034
for distance Doppler matrix
Figure FDA0003023676510000035
Updating to complete background clutter separation;
2.2) removing dynamic clutter caused by dynamic obstacles detected by frequency-modulated continuous waves:
continuously transmitting frequency modulation continuous waves to obtain n continuous range-doppler matrixes, wherein each range-doppler matrix comprises M linear frequency modulation cycles, calculating the average moving speed of the obstacle in the range-doppler matrixes, and removing the value corresponding to the obstacle with the average moving speed being greater than the range profile resolution delta RES from the range-doppler matrixes to further remove the dynamic clutter.
4. The method as claimed in claim 3, wherein the distance Doppler matrix is a matrix of distance Doppler
Figure FDA0003023676510000036
The specific process of updating is as follows:
Figure FDA0003023676510000037
j is a matrix of 01 and is a,
Figure FDA0003023676510000038
for the Hadamard product, sgn is a sign function.
5. The method for remotely sensing the content of the liquid crystal screen based on the millimeter wave sensing as claimed in claim 1,
the arrangement characteristics of the screen liquid crystal in the third step are frequency spectrum characteristics, Mel frequency cepstrum coefficients, linear prediction cepstrum coefficients and line frequency spectrum frequencies;
and the screen liquid crystal arrangement detail characteristics in the third step are residual phase cepstrum coefficients and Teager phase cepstrum coefficients.
6. The method for remotely sensing the content of the liquid crystal screen based on the millimeter wave sensing as claimed in claim 1,
the fine-grained authentication model in the fourth step comprises a Fisher-score-based feature selection module and a screen content-aware integrated classifier;
1) the specific functions of the characteristic selection module based on the Fisher's score are as follows: selecting a feature subset with the maximum Fisher score from the original features by using a cutting plane algorithm;
the original characteristics are screen liquid crystal arrangement characteristics and screen liquid crystal arrangement detail characteristics;
2) the integrated classifier is obtained by fusing algorithms of three models, namely a Gaussian mixture model, a general background model, a support vector machine and a hidden Markov model;
the specific functions of the integrated classifier are as follows: inputting the feature subset with the maximum Fisher score in the original features into the integrated classifier, adjusting the weights of the three models of the integrated classifier by using a regression model, performing weighted summation on output results of the three models according to the adjusted weights, and finally outputting a classification result to finish screen content perception.
7. The method as claimed in claim 6, wherein the training of the cut plane algorithm in the feature selection module optimizes the use of a ridge regression algorithm and a gradient descent algorithm.
CN202110409718.6A 2021-04-16 2021-04-16 Liquid crystal screen content remote sensing method based on millimeter wave sensing Active CN113177581B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110409718.6A CN113177581B (en) 2021-04-16 2021-04-16 Liquid crystal screen content remote sensing method based on millimeter wave sensing

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110409718.6A CN113177581B (en) 2021-04-16 2021-04-16 Liquid crystal screen content remote sensing method based on millimeter wave sensing

Publications (2)

Publication Number Publication Date
CN113177581A true CN113177581A (en) 2021-07-27
CN113177581B CN113177581B (en) 2022-11-04

Family

ID=76923459

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110409718.6A Active CN113177581B (en) 2021-04-16 2021-04-16 Liquid crystal screen content remote sensing method based on millimeter wave sensing

Country Status (1)

Country Link
CN (1) CN113177581B (en)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100153544A1 (en) * 2008-12-16 2010-06-17 Brad Krassner Content rendering control system and method
CN102568335A (en) * 2012-02-15 2012-07-11 东信和平智能卡股份有限公司 Liquid crystal display with built-in antenna and mobile terminal interaction method
CN106844701A (en) * 2017-01-03 2017-06-13 宁波亿拍客网络科技有限公司 A kind of specific markers and application method that identification is perceived based on computer vision
US20170228138A1 (en) * 2016-02-08 2017-08-10 University Of Ontario Institute Of Technology System and method for spatial interaction for viewing and manipulating off-screen content
CN110443138A (en) * 2019-07-05 2019-11-12 东南大学 Combine SVM and CNN Multi-Target Classification Method based on vehicle-mounted millimeter wave radar
WO2020042121A1 (en) * 2018-08-30 2020-03-05 Oppo广东移动通信有限公司 Gesture recognition method, terminal, and storage medium

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100153544A1 (en) * 2008-12-16 2010-06-17 Brad Krassner Content rendering control system and method
CN102568335A (en) * 2012-02-15 2012-07-11 东信和平智能卡股份有限公司 Liquid crystal display with built-in antenna and mobile terminal interaction method
US20170228138A1 (en) * 2016-02-08 2017-08-10 University Of Ontario Institute Of Technology System and method for spatial interaction for viewing and manipulating off-screen content
CN106844701A (en) * 2017-01-03 2017-06-13 宁波亿拍客网络科技有限公司 A kind of specific markers and application method that identification is perceived based on computer vision
WO2020042121A1 (en) * 2018-08-30 2020-03-05 Oppo广东移动通信有限公司 Gesture recognition method, terminal, and storage medium
CN110443138A (en) * 2019-07-05 2019-11-12 东南大学 Combine SVM and CNN Multi-Target Classification Method based on vehicle-mounted millimeter wave radar

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
梁鹏飞等: "采用复调制选带傅里叶变换的毫米波特征提取", 《探测与控制学报》 *

Also Published As

Publication number Publication date
CN113177581B (en) 2022-11-04

Similar Documents

Publication Publication Date Title
Kuo et al. Improved measurements of the CMB power spectrum with ACBAR
Hernández et al. Deep-learning-based earthquake detection for fiber-optic distributed acoustic sensing
US20160238438A1 (en) Determining a health condition of a structure
CN111811617B (en) Liquid level prediction method based on short-time Fourier transform and convolutional neural network
Avanzato et al. A convolutional neural networks approach to audio classification for rainfall estimation
Van Komen et al. Seabed type and source parameters predictions using ship spectrograms in convolutional neural networks
CN110443830B (en) Precipitation cloud cluster detection method based on Gaussian mixture background model
CN110427878A (en) A kind of sudden and violent signal recognition method of Rapid Radio and system
CN113791410A (en) Road environment comprehensive cognition method based on multi-sensor information fusion
CN102254185B (en) Background clutter quantizing method based on contrast ratio function
Niemi et al. A simple and effective method for quantifying spatial anisotropy of time series of precipitation fields
Annau et al. Algorithmic hallucinations of near-surface winds: Statistical downscaling with generative adversarial networks to convection-permitting scales
CN113177581B (en) Liquid crystal screen content remote sensing method based on millimeter wave sensing
Van Komen et al. A CNN for range and seabed estimation on normalized and extracted time-series impulses
CN111275744B (en) Non-contact vibration frequency measurement method based on deep learning and image processing
CN112966710B (en) FY-3D infrared hyperspectral cloud detection method based on linear discriminant analysis
CN106022354A (en) SVM-based image MTF measurement method
CN110991507A (en) Road underground cavity identification method, device and system based on classifier
JP2021154935A (en) Vehicle simulation system, vehicle simulation method and computer program
CN113178198A (en) Non-contact voiceprint biometric authentication method based on millimeter wave perception
CN113176001A (en) Passive temperature monitoring method based on millimeter wave sensing
Gao et al. Automated violin bowing gesture recognition using FMCW-radar and machine learning
CN110806444B (en) Seabed sediment recognition and classification method based on shallow stratum profiler and SVM
Sievert et al. Analysis of reflections in GNSS radio occultation measurements using the phase matching amplitude
Nystuen et al. High temporal resolution of extreme rainfall rate variability and the acoustic classification of rainfall

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
CB03 Change of inventor or designer information
CB03 Change of inventor or designer information

Inventor after: Lin Feng

Inventor after: Li Qin

Inventor after: Li Zhengxiong

Inventor after: Chen Baicheng

Inventor before: Xu Wenyao

Inventor before: Lin Feng

Inventor before: Li Qin

Inventor before: Li Zhengxiong

Inventor before: Chen Baicheng

GR01 Patent grant
GR01 Patent grant