CN115390058B - Single-channel ultra-wideband radar human body posture estimation method based on micro Doppler features - Google Patents

Single-channel ultra-wideband radar human body posture estimation method based on micro Doppler features Download PDF

Info

Publication number
CN115390058B
CN115390058B CN202211016403.6A CN202211016403A CN115390058B CN 115390058 B CN115390058 B CN 115390058B CN 202211016403 A CN202211016403 A CN 202211016403A CN 115390058 B CN115390058 B CN 115390058B
Authority
CN
China
Prior art keywords
human body
body posture
micro doppler
wideband radar
ultra
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202211016403.6A
Other languages
Chinese (zh)
Other versions
CN115390058A (en
Inventor
金添
周小龙
戴永鹏
宋永坤
邱志峰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
National University of Defense Technology
Original Assignee
National University of Defense Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by National University of Defense Technology filed Critical National University of Defense Technology
Priority to CN202211016403.6A priority Critical patent/CN115390058B/en
Publication of CN115390058A publication Critical patent/CN115390058A/en
Application granted granted Critical
Publication of CN115390058B publication Critical patent/CN115390058B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/02—Systems using reflection of radio waves, e.g. primary radar systems; Analogous systems
    • G01S13/06—Systems determining position data of a target
    • G01S13/42—Simultaneous measurement of distance and other co-ordinates
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/02—Systems using reflection of radio waves, e.g. primary radar systems; Analogous systems
    • G01S13/50—Systems of measurement based on relative movement of target
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D30/00—Reducing energy consumption in communication networks
    • Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks

Abstract

The invention discloses a single-channel ultra-wideband radar human body posture estimation method based on micro Doppler characteristics, which comprises the following steps: s1, preprocessing for removing interference and noise on human body posture echo data acquired by a single-channel ultra-wideband radar; s2, performing down-conversion processing on the preprocessed human body posture echo data, and performing self-adaptive filtering processing; s3, generating a micro Doppler spectrogram corresponding to the human body gesture according to the signals subjected to the self-adaptive filtering processing; s4, inputting the generated micro Doppler spectrogram data into a back bone network of a pre-training model to extract the characteristics of each part of the human body contained in the micro Doppler; s5, mapping the extracted micro Doppler spectrogram characteristic matrix into probability distribution of each joint point of each skeleton of the human body by adopting a keypoint head network; s6, solving the coordinate positions of all bone joint points of the human body by adopting a softmax function. The invention improves the universality of the ultra-wideband radar human body posture estimation and can be suitable for various human body posture estimation scenes.

Description

Single-channel ultra-wideband radar human body posture estimation method based on micro Doppler features
Technical Field
The invention relates to the technical field of ultra-wideband radar signal processing, in particular to a single-channel ultra-wideband radar human body posture estimation method based on micro Doppler characteristics.
Background
In recent years, with the rapid development of the internet of things and smart cities, a great deal of research work is put into building an intelligent wireless sensing system, human activities are perceived and understood by using ubiquitous wireless sensing signals, and the gestures of a human body to be detected are tracked and recognized by analyzing human body reflection signals.
Camera-based human body pose estimation systems have been very successful today. However, camera-based human body pose estimation solutions are limited by issues such as diversity of clothing, background differences, light and darkness, occlusion of human body objects, and privacy. In the past, camera-based human body posture estimation studies estimate the occluded body part on the basis of the visible part of the human body target, however, such estimation causes a large estimation error since the human body target is viable.
The human perception technology based on the ultra-wideband radar shows the potential of new generation application by overcoming the technical challenges faced by the human perception solution of the traditional camera, can support more complex interaction between human and physical environment, promotes the appearance of the human posture estimation technology based on the radar, is suitable for accurate perception with low cost in various scenes, and protects the privacy of human beings. The intelligent wireless system senses the gesture and the body shape of a human body and the activities in the back of a wall and in a dark environment by utilizing ultra-wideband radar signals, has wide detection sensing range, does not need any equipment of a target, does not relate to privacy problems, and has become a research hot spot for human sensing at home and abroad.
Disclosure of Invention
The invention aims to provide a single-channel ultra-wideband radar human body posture estimation method based on micro Doppler characteristics, which overcomes the defects existing in the prior art.
In order to achieve the above purpose, the technical scheme adopted by the invention is as follows:
a single-channel ultra-wideband radar human body posture estimation method based on micro Doppler features comprises the following steps:
s1, preprocessing for removing interference and noise on human body posture echo data acquired by a single-channel ultra-wideband radar;
s2, performing down-conversion processing on the preprocessed human body posture echo data, and performing self-adaptive filtering processing;
s3, generating a micro Doppler spectrogram corresponding to the human body gesture according to the signals subjected to the self-adaptive filtering processing;
s4, inputting the generated micro Doppler spectrogram data into a back bone network of a pre-training model to extract the characteristics of each part of the human body contained in the micro Doppler;
s5, mapping the extracted micro Doppler spectrogram characteristic matrix into probability distribution P of each joint point of each skeleton of the human body by adopting a keypoint head network k ；
S6, solving the coordinate positions of all bone joint points of the human body by adopting a softmax function.
Further, the saidThe preprocessing in the step S1 specifically comprises the following steps: echo data matrix for human body posture
Figure BDA0003812679470000021
Interpolation processing is carried out to remove interference and noise; in (1) the->
Figure BDA0003812679470000022
And K is the number of frequency points of the single-channel ultra-wideband radar signal, and N is the number of acquired echo frames.
Further, the step S2 specifically includes: preprocessed radar echo data
Figure BDA0003812679470000023
Performing down-conversion treatment to obtain +.>
Figure BDA0003812679470000024
And performing adaptive filtering to obtain tensor signal +.>
Figure BDA0003812679470000025
And realizing direct wave inhibition in echo signals.
Further, the tensor signal A (x, t) after the adaptive filtering processing is subjected to short-time Fourier transform processing according to the following formula to obtain micro Doppler characteristics corresponding to the signal:
Figure BDA0003812679470000026
where g (t) is a window function of the short-time fourier transform.
Further, the backup network in the step S4 adopts a Resnet50 network.
Further, the probability distribution in step S5 satisfies a gaussian distribution assumption condition, where k is a kth bone node, and P represents probability distribution of each bone node of the human body posture, which refers to probability of each bone node of the human body passing through different positions of the micro doppler spectrogram in the physical space.
Further, the step S6 specifically includes: probability distribution P from individual skeletal joints in human body poses using softmax function k To find the maximum position index to obtain the coordinate position S of the joint point in the physical space skeleton (x, y); and then connecting all the joint points in the human body posture according to a certain sequence to generate a visual two-dimensional human body posture skeleton map.
Compared with the prior art, the invention has the advantages that: the invention provides a single-channel ultra-wideband radar human body posture estimation method based on micro Doppler characteristics,
aiming at the problems that the resolution of imaging is lower and the imaging quality is rapidly reduced along with the increase of imaging distance in the human body posture estimation of the ultra-wideband radar based on the imaging mode, and the universality is poor, the ultra-wideband radar human body posture estimation is carried out by adopting the micro Doppler characteristic. Generating a micro Doppler spectrogram for human body posture echo data in a scene, extracting features in the spectrogram by using a Resnet50 network according to information of each joint part of a human body in the spectrogram, converting the extracted spectrogram features into probability distribution of each bone joint point of a human body target by using a key point head network, and finally obtaining coordinate positions of each bone joint point of the human body by using a softmax function and connecting the coordinate positions according to a certain sequence to obtain a human body posture estimation result. The method estimates the human body posture from the human body micro Doppler spectrogram by means of the deep neural network tool, improves the universality of ultra-wideband radar human body posture estimation, and can be suitable for various human body posture estimation scenes.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flow chart of a single-channel ultra-wideband radar human body posture estimation method based on micro-Doppler characteristics.
Fig. 2 is a block diagram of a human body posture estimation depth neural network based on micro-doppler features, which is adopted in the invention.
Fig. 3 is an image of the echo after the background noise is filtered by the difference processing.
Fig. 4 is an image after down-conversion and adaptive filtering.
Fig. 5 is a micro doppler spectrum corresponding to a certain human body posture.
Fig. 6 is a view of a backhaul network visualization in a deep neural network structure diagram.
Fig. 7 is a visualization of a keypoint head network in a deep neural network architecture diagram.
Fig. 8 is a graph of human body posture estimation results based on micro-doppler features.
Detailed Description
The preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby making clear and defining the scope of the present invention.
The single-channel ultra-wideband radar in the embodiment is a single-shot ultra-wideband radar with single receiving, the transmitted signal is a pulse signal, the center frequency is 7.29GHz, the bandwidth is 2.3GHz, the pulse repetition frequency is 100MHz, and the frame rate of the ultra-wideband radar is F ps =30 Hz, i.e. a sampling time interval per frame of 0.033 seconds.
Referring to fig. 1 and 2, the present embodiment discloses a single-channel ultra-wideband radar human body posture estimation method based on micro doppler features, which includes the following steps:
and S1, preprocessing human body posture echo data acquired by the single-channel ultra-wideband radar, and removing interference and noise.
In this embodiment, the echo signal of the ultra wideband radar can be expressed as:
Figure BDA0003812679470000031
where N represents the nth scattering point of the human target. In the embodiment, 10 groups of different human targets are collected in total, each group of data comprises 4000 frames of echo data, and each group of echo data is subjected to difference processing to filter background noise, as shown in fig. 3.
And S2, performing down-conversion processing on the preprocessed human body posture echo data, and performing self-adaptive filtering processing.
In this embodiment, the difference-processed radar echo data is used for
Figure BDA0003812679470000041
Down-conversion processing is carried out to obtain
Figure BDA0003812679470000042
And performing adaptive filtering to obtain +.>
Figure BDA0003812679470000043
The direct wave suppression effect in the echo signal is achieved, wherein the coefficient α=0.05 of the adaptive filtering process, as shown in fig. 4.
And S3, generating a micro Doppler spectrogram corresponding to the human body posture according to the signals subjected to the self-adaptive filtering processing.
In this embodiment, the tensor signal a (x, t) is subjected to short-time fourier transform processing to obtain the micro-doppler characteristic corresponding to the signal, as shown in fig. 5. One cycle of walking a human target is about 2 seconds, and the frame rate of the ultra-wideband radar is 30MHz, so that the short-time fourier transform in this embodiment uses 60 frames of continuous data for acquisition, and uses a Gabor window function.
Figure BDA0003812679470000044
Where g (t) is a window function of the short-time fourier transform, the present embodiment preferably employs a Gabor function.
And S4, inputting the generated micro Doppler spectrogram data into a back bone network of a pre-training model to extract the characteristics of each part of the human body contained in the micro Doppler.
In this embodiment, the characteristics of each part of the human body are extracted from the back bone network using the micro doppler spectrogram data corresponding to the human body posture as the trained model. The backup network adopts a Resnet50 network, and is used for extracting features in the micro Doppler spectrogram and generating a feature matrix. In order to better characterize micro-Doppler characteristics in human body gestures, a micro-Doppler spectrogram is jointly generated by adopting radar echo data of 60 continuous frames, the size of the spectrogram is 256 multiplied by 56 and is used as input data of a back bone network, the size of the input data is (32,1,256,56), wherein 32 is a trained batch size.
And S5, mapping the extracted micro Doppler spectrogram characteristic matrix into probability distribution of each bone node of the human body by using a key point head network.
In this embodiment, the key head network is composed of three deconvolution layers and a full connection layer, wherein the deconvolution layers include deconvolution operation, batch normalization operation and Relu operation. The network can map the extracted Doppler spectrum characteristic matrix into probability distribution P of each bone node k . The probability satisfies the assumption condition of obeying gaussian distribution, wherein k is the kth bone joint, and in the embodiment, k=14 and p are adopted to represent probability distribution of each bone joint of the human body gesture, which means probability of each bone joint of the human body passing through different positions of the micro doppler spectrogram in the physical space. k=14 represents the total number of skeletal joints of 14 individuals, corresponding to 14 individual body parts, head, neck, left shoulder, left elbow, left hand, right shoulder, right elbow, right hand, left hip, right hip, left knee, right knee, left foot and right foot, respectively, as shown in fig. 7.
And S6, solving the coordinate positions of all bone joint points of the human body by adopting a softmax function.
In this embodiment, the softmax function is used from each bone in the human body poseProbability distribution P of iliac node k The maximum position index is searched to obtain the coordinate position S of the joint point in the physical space skeleton (x, y). And connecting the coordinates of the 14 joint points in the human body posture according to a certain sequence to generate a visual two-dimensional human body posture skeleton map.
In this embodiment, the size of data output by the key point head network is (32,14,64,64), and the physical space coordinate positions S of 14 skeletal joints in the human body posture are estimated through a softmax function skeleton (x, y) of size (32,16,2). The key points in the human body posture are connected in the order of (head-neck, neck-left shoulder-left elbow-left hand, neck-right key-right elbow-right hand, left shoulder-left hip, right shoulder-right hip, left hip-left knee-left foot, right hip-right knee-right foot), and the specific results are shown in fig. 8.
In this embodiment, in order to better measure and evaluate the effectiveness and universality of the single-channel ultra-wideband radar human body posture estimation method based on micro-doppler, the performance of the human body posture estimation network under different methods is compared, as shown in table 1.
TABLE 1
Figure BDA0003812679470000051
The method comprises the steps of generating a micro Doppler spectrogram for human body posture echo data in a scene, extracting features in the spectrogram by utilizing information of all joint parts of a human body in the spectrogram through a Resnet50 network, converting the extracted spectrogram features into probability distribution of all bone joints of a human body target through a key point head network, finally obtaining coordinate positions of all bone joints of the human body through a softmax function, and connecting the coordinate positions according to a certain sequence to obtain a human body posture estimation result. The method estimates the human body posture from the human body micro Doppler spectrogram by means of the deep neural network tool, improves the universality of ultra-wideband radar human body posture estimation, and can be suitable for various human body posture estimation scenes.
Although the embodiments of the present invention have been described with reference to the accompanying drawings, the patentees may make various modifications or alterations within the scope of the appended claims, and are intended to be within the scope of the invention as described in the claims.

Claims (1)

1. The single-channel ultra-wideband radar human body posture estimation method based on the micro Doppler characteristics is characterized by comprising the following steps of:
s1, preprocessing for removing interference and noise on human body posture echo data acquired by a single-channel ultra-wideband radar;
s2, performing down-conversion processing on the preprocessed human body posture echo data, and performing self-adaptive filtering processing;
s3, generating a micro Doppler spectrogram corresponding to the human body gesture according to the signals subjected to the self-adaptive filtering processing;
s4, inputting the generated micro Doppler spectrogram data into a back bone network of a pre-training model to extract the characteristics of each part of the human body contained in the micro Doppler;
s5, mapping the extracted micro Doppler spectrogram characteristic matrix into probability distribution P of each joint point of each skeleton of the human body by adopting a keypoint head network k ；
S6, solving the coordinate positions of all bone joint points of the human body by adopting a softmax function;
the preprocessing in the step S1 specifically includes: echo data matrix for human body posture
Figure FDA0004269319460000011
Interpolation processing is carried out to remove interference and noise; in (1) the->
Figure FDA0004269319460000012
The real number field is K, the number of frequency points of the single-channel ultra-wideband radar signal is K, and N is the number of acquired echo frames;
the step S2 specifically comprises the following steps: preprocessed radar echo data
Figure FDA0004269319460000013
Performing down-conversion treatment to obtain +.>
Figure FDA0004269319460000014
And performing adaptive filtering to obtain tensor signal +.>
Figure FDA0004269319460000015
Direct wave suppression in echo signals is realized;
and carrying out short-time Fourier transform processing on the tensor signal A (u) subjected to the adaptive filtering processing according to the following formula to obtain micro Doppler characteristics corresponding to the signal:
Figure FDA0004269319460000016
where g (t) is a window function of the short-time Fourier transform;
the backup network in the step S4 adopts a Resnet50 network;
the probability distribution in the step S5 meets the assumption condition of obeying Gaussian distribution, wherein k is a kth bone joint point, P represents the probability distribution of each bone joint point of the human body gesture, and the probability of each bone joint point of the human body passing through different positions of a micro Doppler spectrogram in a physical space is indicated;
the step S6 specifically includes: probability distribution P from individual skeletal joints in human body poses using softmax function k To find the maximum position index to obtain the coordinate position S of the joint point in the physical space skeleton (x, y); and then connecting all the joint points in the human body posture according to a certain sequence to generate a visual two-dimensional human body posture skeleton map.
CN202211016403.6A 2022-08-24 2022-08-24 Single-channel ultra-wideband radar human body posture estimation method based on micro Doppler features Active CN115390058B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211016403.6A CN115390058B (en) 2022-08-24 2022-08-24 Single-channel ultra-wideband radar human body posture estimation method based on micro Doppler features

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211016403.6A CN115390058B (en) 2022-08-24 2022-08-24 Single-channel ultra-wideband radar human body posture estimation method based on micro Doppler features

Publications (2)

Publication Number Publication Date
CN115390058A CN115390058A (en) 2022-11-25
CN115390058B true CN115390058B (en) 2023-07-04

Family

ID=84121692

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211016403.6A Active CN115390058B (en) 2022-08-24 2022-08-24 Single-channel ultra-wideband radar human body posture estimation method based on micro Doppler features

Country Status (1)

Country Link
CN (1) CN115390058B (en)

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9568594B2 (en) * 2015-06-29 2017-02-14 Echocare Technologies Ltd. Human posture feature extraction in personal emergency response systems and methods
CN108564005B (en) * 2018-03-26 2022-03-15 电子科技大学 Human body falling identification method based on convolutional neural network
CN110647788B (en) * 2018-12-28 2023-04-18 南京华曼吉特信息技术研究院有限公司 Human daily behavior classification method based on micro-Doppler characteristics
CN111208509B (en) * 2020-01-15 2020-12-29 中国人民解放军国防科技大学 Ultra-wideband radar human body target posture visualization enhancing method
CN111505632B (en) * 2020-06-08 2023-03-03 北京富奥星电子技术有限公司 Ultra-wideband radar action attitude identification method based on power spectrum and Doppler characteristics
CN111914707A (en) * 2020-07-22 2020-11-10 上海大学 System and method for detecting drunkenness behavior
CN114814775A (en) * 2022-05-24 2022-07-29 哈尔滨工业大学 Radar fall detection method and equipment based on ResNet network

Also Published As

Publication number Publication date
CN115390058A (en) 2022-11-25

Similar Documents

Publication Publication Date Title
CN108830819B (en) Image fusion method and device for depth image and infrared image
CN108226892B (en) Deep learning-based radar signal recovery method in complex noise environment
CN110837768B (en) Online detection and identification method for rare animal protection
Li et al. Towards domain-independent and real-time gesture recognition using mmwave signal
CN104766334B (en) Small IR targets detection tracking and its device
CN110400294B (en) Infrared target detection system and detection method
CN112711979A (en) Non-contact vital sign monitoring under slow random motion based on biological radar
CN109215025A (en) A kind of method for detecting infrared puniness target approaching minimization based on non-convex order
Li et al. A small target detection algorithm in infrared image by combining multi-response fusion and local contrast enhancement
CN110991559A (en) Non-contact cooperative sensing method for indoor personnel behaviors
CN105595971A (en) Pulse condition information collecting system based on video and collecting method thereof
CN103400360A (en) Multi-source image fusing method based on Wedgelet and NSCT (Non Subsampled Contourlet Transform)
CN115390058B (en) Single-channel ultra-wideband radar human body posture estimation method based on micro Doppler features
Lianfang et al. Multi focus image fusion using combined median and average filter based hybrid stationary wavelet transform and principal component analysis
CN112669332A (en) Method for judging sea and sky conditions and detecting infrared target based on bidirectional local maximum and peak local singularity
CN106611409B (en) A kind of Small object enhancing detection method based on secondary image fusion
CN113050083A (en) Ultra-wideband radar human body posture reconstruction method based on point cloud
Tang et al. Learning from natural noise to denoise micro-doppler spectrogram
CN116244673A (en) Behavior and identity recognition method based on wireless signals
CN114580464A (en) Human heart rate variability and respiratory rate measurement method based on variational modal decomposition and constraint independent component analysis
Lekhashri et al. Use of motion-print in sign language recognition
CN112926381B (en) Underwater target recognition system based on laser detection technology
CN113887374A (en) Brain-controlled drinking water system based on dynamic convergence differential neural network
CN113158942A (en) Segmentation algorithm and device for detecting motion human behavior
Zhou et al. A Micro-Doppler Based Human Pose Estimation Framework for Single-channel Ultra-Wideband Radar

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant