JP2022156964A - Radar heart rate detection method, and system thereof - Google Patents

Abstract

To detect a heart rate by detection based on non-contact type 'Vital Signs' of a radar.SOLUTION: In a radar heart rate detection method and its system, raw signals are collected by facing a radar sensor to at least one object, and according to a concept for visualizing an image, after converting the raw signals into two-dimensional image information, a network is allowed to learn automatically by deep learning, which in the two-dimensional image information is a notable heart rate frequency, or which is a heart rate frequency to be filtered, to thereby take out the heart rate frequency effectively.SELECTED DRAWING: Figure 1

Description

The present invention relates to a radar heartbeat detection method and system, and particularly has the advantage of automatically searching for features by deep learning, and the network itself determines the relationship between heartbeat frequencies and other non-heartbeat frequencies in two-dimensional image information. The present invention relates to a method and system for locating and ultimately detecting heartbeats.

The arrival of the artificial intelligence era has changed the structure of society and affected our daily life in direct or indirect ways. We hope to solve problems and improve the quality of life through artificial intelligence. The combination of artificial intelligence and medicine has always been a hot topic, and the application of disease judgment and prediction is not uncommon. Due to the complexity of medical signals and the required accuracy, there are many limitations to the use of conventional algorithmic processing schemes. By using big data to provide objective data and combining deep learning, computers can learn quickly and can make predictions, judgments, classifications and decisions. For this reason, in the past, we had to go to a hospital and see a doctor who had undergone long-term training to be diagnosed with a disease. If it can be incorporated into our daily life, it can improve our quality of life, achieve early prevention effects, and solve the plight of lack of medical human resources. The combination of advances in AI makes it possible to judge artificial intelligence beyond humanity.

Radar non-contact Vital Signs based detection, many radars have this application (eg UWB, CW Doppler Radar And FMCW). All methods can successfully detect only in controlled environments and under certain conditions, but without resolving the uncertainties posed by the real environment, radar has sufficient capability to detect life in various domains. Still unable to detect symptoms. These uncertainties include unknown body conditions, sensor shaking and affiliation, effective rejection of respiratory harmonics and reliable detection even in noisy environments, all of which are It has been common to add more signal processing analysis, more radar or coupling of radar and other sensors to solve the problem.

Therefore, in order to obtain reliable measurements and motion compensation, the proposed deep learning algorithm converts the signal into two-dimensional image information by short-time Fourier transform (STFT). (Spectrogram), and after automatically finding features by deep learning, the network finds the relevance between the heartbeat frequency and other non-heartbeat frequencies in the two-dimensional image information (Spectrogram), and finally the heartbeat can be detected, thereby resolving uncertainties such as those described above. Therefore, the present invention should be the best solution.

The radar heartbeat detection method of the present invention comprises the steps of: (1) aligning at least one radar sensor toward at least one object and collecting a raw signal; and (2) transforming the raw signal into two-dimensional image information. After transforming, learning the two-dimensional image information in a neural network model, automatically filtering out noise by the neural network model, and learning the relationship between heartbeat frequency and non-heartbeat frequency of the two-dimensional image information. and retrieving the heartbeat frequency.

More particularly, said radar sensor is a 1 millimeter radar.

More specifically, the raw signal is first filtered and then Fourier transformed to transform the signal into the two-dimensional image information.

More specifically, the filtering process retains the raw signal within the heartbeat frequency region of the human body by means of a high-pass filter and/or a band-pass filter.

More specifically, the neural network model refines the residual block (Res_block) using average pooling (AvgPooling).

More specifically, for signal sampling accuracy, first detect and determine whether the object has left.

The radar heartbeat detection system of the present invention includes at least one radar sensor collecting raw signals toward at least one target, and at least one server device connected to the radar sensor, the server device comprising: receiving the raw signal, converting the raw signal into two-dimensional image information, learning the two-dimensional image information in a neural network model, automatically filtering noise by the neural network model, and The relationship between the heartbeat frequency and the non-heartbeat frequency of the two-dimensional image information is learned, and the heartbeat frequency is automatically extracted.

More specifically, said server apparatus includes at least one processor and at least one computer-readable storage medium, wherein a plurality of said computer-readable storage media store at least one application and said computer-readable storage The medium further stores computer readable instructions, and when the plurality of processors executes the computer readable instructions, executes the application, transforms the raw signal into the two-dimensional image information, and then produces the two-dimensional image. learning information in the neural network model, automatically filtering the noise by the neural network model, learning the relationship between heartbeat frequency and non-heartbeat frequency of the two-dimensional image information, and automatically heartbeat frequency take out

More particularly, said application comprises an input module for inputting at least one data file comprising said raw signal, a filtering module connected to said input module for filtering said raw signal, and said filtering module. a two-dimensional image transformation module connected to perform a Fourier transform to transform a signal into the two-dimensional image information; and the neural connected to the two-dimensional image transformation module to input the two-dimensional image information to a neural network learning module. a network learning module, wherein the neural network learning module refines a residual block (Res_block) into the neural network model using average pooling (AvgPooling), and converts the two-dimensional image information into the neural network model. automatically filter the noise by the neural network model, learn the relationship between the heartbeat frequency and the non-heartbeat frequency of the two-dimensional image information, and extract the heartbeat frequency.

More particularly, the filtering module retains the raw signal within the heartbeat frequency region of the human body by means of a high-pass filter and/or a band-pass filter.

1 is a flowchart of the radar heartbeat detection method and system of the present invention; 1 is a diagram showing a system configuration of a radar heartbeat detection method and system thereof according to the present invention; FIG. It is a figure which shows the radar heartbeat detection method of this invention, and the server apparatus of the system. 1 is a diagram showing the configuration of an application program for the radar heartbeat detection method and system of the present invention; FIG. 1 is a diagram showing a conventional neural network model; FIG. FIG. 3 is a diagram showing an improved neural network model of the radar heartbeat detection method and system of the present invention; 1 is a raw signal waveform diagram in which the raw signal of the radar heartbeat detection method and system of the present invention is not filtered; 2 is a Fourier transform waveform diagram in which the raw signal of the radar heartbeat detection method and system of the present invention is not filtered; 3 is a spectrogram diagram of the radar heartbeat detection method and system of the present invention after the raw signal is unfiltered and transformed; 1 is a raw signal waveform diagram obtained by subjecting the raw signal of the radar heartbeat detection method and system of the present invention to high-pass filtering. 2 is a Fourier transform waveform diagram obtained by subjecting the raw signal of the radar heartbeat detection method and system of the present invention to high-pass filtering. 3 is a spectrogram diagram after the raw signal of the radar heartbeat detection method and system of the present invention is high-pass filtered and transformed; 1 is a raw signal waveform diagram obtained by subjecting the raw signal of the radar heartbeat detection method and system of the present invention to band-pass filtering. 2 is a Fourier transform waveform diagram obtained by band-pass filtering the raw signal of the radar heartbeat detection method and system of the present invention. 3 is a spectrogram diagram after conversion of the raw signal of the radar heartbeat detection method and system of the present invention which has been band-pass filtered. 1 is a detection flowchart of the radar heartbeat detection method and system of the present invention; 1 is a voting implementation diagram of the radar heartbeat detection method and system thereof of the present invention; FIG.

Other technical content, features and effects of the present invention will be clarified in the detailed description of preferred embodiments with reference to the drawings.

Please refer to FIG. FIG. 1 is a flow diagram of the radar heartbeat detection method and system of the present invention, and as can be seen from the figure, includes the following steps.
(1) Align at least one radar sensor toward at least one target and collect raw signals (101).
(2) converting the raw signal into two-dimensional image information, learning the two-dimensional image information in the neural network model, automatically filtering noise through the neural network model, and obtaining the heartbeat frequency of the two-dimensional image information; and non-heartbeat frequencies are learned to retrieve the heartbeat frequencies (102).

The proposed radar heartbeat detection method and system includes the following, as shown in FIGS.
(1) At least one radar sensor 1 used to collect raw signals towards at least one target, the radar sensor being a 1 millimeter radar.
(2) at least one server device 2 connected with the radar sensor 1, the server device 2 can receive the raw signal, convert the raw signal into two-dimensional image information, and then convert the two-dimensional image information into neural Learning in a network model, automatically filtering noise through a neural network model, learning the relationship between the heartbeat frequency and the non-heartbeat frequency of two-dimensional image information, and automatically extracting the heartbeat frequency.

The server device 2 at least includes at least one processor 21 and at least one computer-readable storage medium 22 . A plurality of computer-readable storage media 22 store at least one application 221 . Computer readable storage medium 22 further stores computer readable instructions. When the plurality of processors 21 execute the plurality of computer readable instructions, they execute the application 221 to convert the raw signals into two-dimensional image information, and then train the two-dimensional image information in a neural network model to generate a neural network model. Automatically filter noise through , learn the relationship between the heartbeat frequency and the non-heartbeat frequency of the two-dimensional image information, and automatically extract the heartbeat frequency.

Applications 221 include:
(1) an input module 2211 for inputting at least one data file having raw signals;
(2) a filtering module 2212 connected with the input module 2211 for filtering the raw signal, which retains the raw signal within the heartbeat frequency range of the human body by means of a high-pass filter and/or a band-pass filter;
(3) a two-dimensional image transform module 2213 connected to the filtering module 2212 and performing Fourier transform to transform the signal into two-dimensional image information;
(4) a neural network learning module 2214 connected with the two-dimensional image transformation module 2213 and inputting two-dimensional image information into a neural network model, wherein the neural network learning module uses average pooling (AvgPooling); The difference block (Res_block) is refined into a neural network model, the two-dimensional image information is learned in the neural network model, noise is automatically filtered by the neural network model, and the heartbeat frequency and non-heartbeat frequency of the two-dimensional image information are heartbeat frequency can be retrieved while learning the relationship between

Since the Raw Signal is the original and unprocessed signal, after performing a Fast Fourier Transform (FFT) on it in order to improve the quality of the input signal, this signal is very close to 0 Hz. It can be seen that there is a large noise signal. Therefore, we do the following two steps.
(1) A high-pass filter removes very large noise signals, the purpose of which is to retain the frequency region before 0.8 Hz of the human heartbeat.
(2) A band pass filter converts the frequency between 0.8 Hz and 2.0 Hz, since the heart rate is 50-120 beats per minute, and passes the entire raw signal (Raw Signal) is locked within the 0.8-2.0 Hz frequency range of the human heart rate, so this step can be used to limit the signal to this frequency and calculate the heart rate.

式中、ｗ（ｔ）はウィンドウ関数であり、ｘ（ｔ）は変換される信号である。Ｘ（ｔ， ｗ）はｗ（ｔ－τ）ｘ（τ）のフーリエ変換であり、ｔの変化に伴い、ウィンドウ関数が時間軸上で変位する。本案が短時間フーリエ変換を選択する原因は、生信号（Ｒａｗ Ｓｉｇｎａｌ）を変換した後に元の関数に逆変換するためであるが、ロスがほぼゼロであるため、時間－周波数分析（Ｔｉｍｅ－Ｆｒｅｑｕｅｎｃｙ ａｎａｌｙｓｉｓ）を行うのに適合する。 The proposed method utilizes the short-time Fourier transform (STFT) to transform the signal into the time-frequency dimension, transforming the 1D signal into a 2D image by means of the time-frequency diagram. The time Fourier transform is described below.
(1) Short-time Fourier transforms are serial Fourier transforms after adding a window signal to determine the sinusoidal frequency and phase of a portion of the signal that changes with time. The biggest difference between it and the Fourier transform is that the Fourier transform does not give any information that the signal frequency changes with time.
(2) Briefly, in the continuous-time example, this window function is shifted along the time axis, and the resulting series of Fourier transform results is formed into a two-dimensional representation, and mathematically, such an operation can be written as

where w(t) is the window function and x(t) is the signal to be transformed. X(t, w) is the Fourier transform of w(t-τ)x(τ), and the window function displaces on the time axis as t changes. The reason why we choose the short-time Fourier transform is that after transforming the raw signal, we transform it back to the original function, but since the loss is almost zero, the time-frequency analysis analysis).

The present method is improved based on the residual block (Res_block) as shown in FIG. , and finally add one layer of Global AvgPooling 2D and two layers of Dense to perform the final feature extraction. For example, as shown in FIG. 5B, the reason for using average pooling (AvgPooling) is to use the original architecture in the image. Since the pixels and pixels on the image are relatively irrelevant and the image itself is a very complex input, in this paper the residual block (Res_block) uses MaxPooling. Dimensionality reduction is a very reliable dimensionality reduction scheme. Alternatively, one could retain the pixel-to-pixel neighborhood relationship, or shift the dimensionality reduction task to a convolution layer, which was not bad.
The two methods can convolve the network faster, but the reason for choosing AvgPooling is that the image of our regression task is the Spectrogram, and each time axis of the Spectrogram and Secondly, as it relates to the time axis, intuitively convolution is better because many features are lost if MaxPooling is used. Therefore, immediately after the start, the folding layer (convolution) was selected and dimensionality was reduced, but this proposal replaced the task of dimensionality reduction with average pooling (AvgPooling). It worked so well that we changed all convolutional layer dimensionality reduction tasks to AvgPooling, which also gave good results in the database.

The *8 in FIG. 5B indicates that it is overlapped 8 times within the boxed range, and the lower + sign within the boxed range indicates that the position is added.

The frequency modulated continuous wave radar used in the embodiment of the present invention is used as the radar sensor 1, and minute signal characteristics are detected using millimeter waves and recorded in a database. In this embodiment, the installation height of the radar sensor 1 is 1.8 meters above the heart, the range signal received is 20 cm, the survey angle is plus or minus 20 degrees, and the pulse meter is used. The combined and measured pulse is used as the ground truth in this example.

In this embodiment, we first isolate the position near 0 frequency through a high-pass filter, retain the part before 0.8 Hz, and finally apply a band-pass filter. It is used to keep between 0.8 and 2.0 Hz and reserve some frequencies outside the mask range. The purpose is to make the network widely visible, otherwise some heartbeats between 0.8-1 Hz or 1.8-2.0 Hz will lose their original band due to a Filter, The final filtered Raw Signal is converted to a Spectrogram using the Short Time Fourier Transform (STFT).
For example, each of FIGS. 6A-1, 6A-2, 6A-3, 6B-1, 6B-2, 6B-3, 6C-1, 6C-2, and 6C-3 This is the result of passing through a band-pass filter after passing through a high-pass filter without passing through a filter.

After the input signal is preprocessed and transformed to obtain a spectrogram, it is trained in a neural network, and Adam optimizer is used in this embodiment, with an initial learning rate of 10~. set to 3, batch size set to 25, total training epochs set to 100, and based on the validation set loss function, Patient had 10 early stops. and finally a loss function uses the mean absolute error (MAE) to calculate the loss and call back.

In the present example, if we obtain a residual network (Res_Net) 18 and perform a 1D regression on the Raw Signal, the effect is unfavorable, very difficult to convolve, and a bandpass filter (Bandpass There was a fact that the use of filter) was also difficult to improve and convolution was difficult. The reason for this is that the raw signal (Raw Signal) is too complicated. Raw Signal of is sent to the residual network (Res_Net) for training, and the training result obviously changes, and the original overfitting can be convoluted.

The present invention seeks a loss function that can be relatively representative of the actual and predicted heart beats, and this loss function is the mean absolute error (MAE), given by the mean absolute error (MAE) We can calculate the overall ground truth error during all tests, the lower the loss, the closer it represents the overall result to the true heartbeat. If the mean absolute error (MAE) loss function is used for authentication in the present embodiment, and a suitable filter is added to the preprocessing, the convolution effect of the network will be even better, resulting in a high-pass filter (HPF ) and a bandpass filter (BPF) for preprocessing can effectively convolve a neural network.

Table 1 below proves in the database we recorded in this example (All_Data refers to all recorded data, Avg_Data refers to randomly selected and used as the heart rate average database), and if the original The residual network (Res-Net) of is improved to pass all the dimensionality reduction work to the average pooling (AvgPooling), which yields favorable results, and can also be used in situations with complex missions and low sample rates, such as radar. , more dependent information must be retained in order for the neural network to perform Time-Frequency analysis, and it also forces the neural network to pay particular attention to specific regions. had to be, otherwise many features would be lost and eventually the learning results would be the same. Also, as can be seen by comparing the actual and predicted values, the operation of the method presented in this application to the database presented in this example can be convoluted, further reducing the network to ground truth. can get closer.

様々なｌａｙｅｒを使用して次元削減により得た表現であり、表中のＣはＣｏｎｖ２Ｄに等しく、Ａは平均プーリング（ＡｖｇＰｏｏｌｉｎｇ）に等しい。

Representations obtained by dimensionality reduction using different layers, where C in the table equals Conv2D and A equals average pooling (AvgPooling).

As can be seen from the above examples, the use of a high pass filter and a bandpass filter can reduce the influence of the respiration signal on the heartbeat signal and enhance the effectiveness of neural network training. , which transforms the 1D signal into a 2D signal by the Short-time Fourier transform, ameliorating the problem that the 1D residual network (Res-Net) is difficult to convolve. Along with this, the residual network (Res-Net) after refinement is used for feature extraction, the average pooling (AvgPooling) plays the role of retaining features in the two-dimensional image information (Spectrogram), the residual network (Res-Net) No features in the 2D image information (Spectrogram) are lost when training Net. Neural networks can be accurately trained in two-dimensional image information (Spectrogram) and can be more easily convolved.

Also, for the accuracy of signal sampling, it must first detect and determine whether the object has left. The flow shown in FIG. 7 is as follows.
(1) Each set of Overall Signals is a signal for 6 seconds, and there are 120 sampling points after reading (701 ), and fast Fourier transform (FFT) is performed on the input 120 sampling points (702) to obtain the spectrum of the input 6-second signal.
(2) This part of the Fast Fourier Transform (FFT) cut retains the frequencies 0.2Hz-3.4Hz and filters the rest (703) because the 0Hz of the original signal Because we have found that there is very much noise in the vicinity, this step must filter out signals we do not want.
(3) then perform an inverse fast Fourier transform (FFT) on the filtered spectrum to obtain a filtered signal (704), and finally normalize and perform subsequent calculations (705); .
(4) Based on the normalized signal, after calculating the average value of the 6-second signal, determine whether it is within a prescribed threshold (706); if the average value is within the threshold, then the 6-second A second signal identifies manned.
(5) Threshold setting is calculated by us based on the database (707), and after averaging each data in the database, we find the maximum and minimum values and use them as thresholds for this stage.
(6) Use a threshold to determine whether manned causes a problem, and use a voting mechanism to resolve this problem (708), as the determination result may be unstable.

Also, as shown in FIG. 8, the upper number is the time point, the lower number is the number of votes, and the initial value of the number of votes is zero. Since the manned time point is 0, the number of votes is added by 1 when the time point is 1, and the person leaves when the time point is 10, so the number of votes in this time period continues to be added by 1, with a maximum value of 9 Stop adding 1 when . Whether or not there is a person is determined to be in a manned state when the threshold reaches 5 votes. Since people leave when the time point is 10, they reduce the number of votes at time point 11 by 1 and recognize no one when the number of votes drops to the threshold.

The radar heartbeat detection method and system provided by the present invention have the following advantages over other prior art.
(1) The present invention proposes a deep learning algorithm, which transforms the signal into two-dimensional image information by short-time Fourier transform, has the advantage of automatically finding features by deep learning, and generates a spectrogram by network. By self-discovering the relationship between the heart rate and other non-heart rate, and finally detecting the heart rate, the above uncertainties can be resolved.
(2) The present invention uses the improved residual network (Res-Net) for feature extraction, average pooling (AvgPooling) plays the role of retaining features in the Spectrogram, and the residual network (Res-Net) The neural network can learn accurately in the Spectrogram and can also convolve more easily without losing any features in the Spectrogram when the N-Net trains.

Although the present invention discloses embodiments as described above, they are not intended to limit the invention in any way, and anyone skilled in the art will appreciate that the above-described technical features and embodiments of the present invention can be understood as the present invention. Various changes and modifications may be made without departing from the spirit and scope of Therefore, the patent protection scope of the present invention shall be based on what is specified in the claims attached hereto.

1 radar sensor 2 server device 21 processor 22 computer readable storage medium 221 application 2211 input module 2212 filtration processing module 2213 two-dimensional image conversion module 2214 neural network learning module

Claims (10)

A radar heartbeat detection method comprising:
aligning at least one radar sensor toward at least one target and collecting raw signals;
After transforming the raw signal into two-dimensional image information, learning the two-dimensional image information in a neural network model, automatically filtering noise by the neural network model, and determining the heartbeat frequency of the two-dimensional image information. learning associations with non-heartbeat frequencies and retrieving heartbeat frequencies;
Radar heartbeat detection method. 2. The radar heartbeat detection method of claim 1, wherein the radar sensor is a one millimeter radar. 2. The method of claim 1, wherein said raw signal is first filtered and then converted into said two-dimensional image information by a Fourier transform. 4. The radar heartbeat detection method according to claim 3, wherein said filtering process retains said raw signal within a heartbeat frequency region of a human body by means of a high-pass filter and/or a band-pass filter. The method of claim 1, wherein the neural network model refines the residual block (Res_block) using average pooling (AvgPooling). 2. The radar heartbeat detection method according to claim 1, wherein for the accuracy of signal sampling, firstly detecting and judging whether the object has left. at least one radar sensor that collects raw signals towards at least one target;
at least one server device connected to the radar sensor;
The server device receives the raw signal, converts the raw signal into two-dimensional image information, learns the two-dimensional image information in a neural network model, and automatically removes noise by the neural network model. filtering, learning the relationship between the heartbeat frequency and the non-heartbeat frequency of the two-dimensional image information, and automatically extracting the heartbeat frequency,
Radar heart rate detection system. The server device at least includes at least one processor and at least one computer-readable storage medium, wherein a plurality of the computer-readable storage media store at least one application, and the computer-readable storage medium is a computer-readable further storing instructions, and executing the application when the plurality of processors execute the computer readable instructions; transforming the raw signal into the two-dimensional image information; and converting the two-dimensional image information into the neural network. learning in a model, automatically filtering the noise by the neural network model, learning the relationship between the heartbeat frequency and the non-heartbeat frequency of the two-dimensional image information, and automatically extracting the heartbeat frequency. 8. The radar heartbeat detection system of claim 7. The application includes an input module for inputting at least one data file comprising the raw signal, a filtering module connected to the input module for filtering the raw signal, and a filtering module connected to the filtering module to perform a Fourier transform. and a neural network learning module connected to the two-dimensional image conversion module and inputting the two-dimensional image information to the neural network learning module; wherein the neural network learning module uses average pooling (AvgPooling) to refine a residual block (Res_block) into the neural network model and learns the two-dimensional image information in the neural network model; 9. The method of claim 8, wherein the neural network model automatically filters out the noise, learns the relationship between the heartbeat frequency and the non-heartbeat frequency of the two-dimensional image information, and extracts the heartbeat frequency. Radar heart rate detection system. 10. The radar heartbeat detection system of claim 9, wherein the filtering module retains the raw signal within the heartbeat frequency region of the human body by a high-pass filter and/or a band-pass filter.

Images