CN116453227B - Gait recognition method based on double millimeter wave radar under ship environment - Google Patents
Gait recognition method based on double millimeter wave radar under ship environment Download PDFInfo
- Publication number
- CN116453227B CN116453227B CN202310721908.0A CN202310721908A CN116453227B CN 116453227 B CN116453227 B CN 116453227B CN 202310721908 A CN202310721908 A CN 202310721908A CN 116453227 B CN116453227 B CN 116453227B
- Authority
- CN
- China
- Prior art keywords
- millimeter wave
- wave radar
- gait
- detection data
- characteristic diagram
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000005021 gait Effects 0.000 title claims abstract description 108
- 238000000034 method Methods 0.000 title claims abstract description 36
- 238000001514 detection method Methods 0.000 claims abstract description 86
- 210000003141 lower extremity Anatomy 0.000 claims abstract description 59
- 238000010586 diagram Methods 0.000 claims abstract description 58
- 210000001364 upper extremity Anatomy 0.000 claims abstract description 41
- 230000033764 rhythmic process Effects 0.000 claims abstract description 18
- 238000007781 pre-processing Methods 0.000 claims description 11
- 238000013527 convolutional neural network Methods 0.000 claims description 9
- 208000030175 lameness Diseases 0.000 claims description 8
- 230000009977 dual effect Effects 0.000 claims description 6
- 206010022562 Intermittent claudication Diseases 0.000 description 7
- 208000024980 claudication Diseases 0.000 description 7
- 230000033001 locomotion Effects 0.000 description 6
- 239000002184 metal Substances 0.000 description 6
- 210000002414 leg Anatomy 0.000 description 4
- 239000000523 sample Substances 0.000 description 3
- 230000002159 abnormal effect Effects 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000009897 systematic effect Effects 0.000 description 1
- 210000000689 upper leg Anatomy 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/05—Detecting, measuring or recording for diagnosis by means of electric currents or magnetic fields; Measuring using microwaves or radio waves
- A61B5/0507—Detecting, measuring or recording for diagnosis by means of electric currents or magnetic fields; Measuring using microwaves or radio waves using microwaves or terahertz waves
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
- A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
- A61B5/112—Gait analysis
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
- A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
- A61B5/1126—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb using a particular sensing technique
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/68—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient
- A61B5/6887—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient mounted on external non-worn devices, e.g. non-medical devices
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/7253—Details of waveform analysis characterised by using transforms
- A61B5/7257—Details of waveform analysis characterised by using transforms using Fourier transforms
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
- G01S7/02—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S13/00
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
- G01S7/02—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S13/00
- G01S7/41—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S13/00 using analysis of echo signal for target characterisation; Target signature; Target cross-section
- G01S7/415—Identification of targets based on measurements of movement associated with the target
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
- G06N3/0442—Recurrent networks, e.g. Hopfield networks characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/30—Noise filtering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/762—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using clustering, e.g. of similar faces in social networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T90/00—Enabling technologies or technologies with a potential or indirect contribution to GHG emissions mitigation
Abstract
The invention discloses a gait recognition method under a ship environment based on double millimeter wave radars, which comprises the following steps that S1, a first millimeter wave radar and a second millimeter wave radar are arranged, so that the first millimeter wave radar detects the upper limb characteristics of a person, the second millimeter wave radar detects the lower limb characteristics of the person, and the detection ranges of the first millimeter wave radar and the second millimeter wave radar are not overlapped; s2, processing detection data of the first millimeter wave radar and the second millimeter wave radar to obtain a 3D point cloud characteristic diagram of the upper limb, a speed-time characteristic diagram of the lower limb and a rhythm frequency-speed characteristic diagram of the lower limb; s3, after noise reduction is carried out on each feature map obtained in the step S2, a mmGRnet network is input, and the gait type is output after the mmGRnet network operation. The method realizes accurate identification of the gait characteristics of personnel under the complex detection environment of the ship.
Description
Technical Field
The invention relates to the technical field of personnel feature recognition, in particular to a ship environment gait recognition method based on a double millimeter wave radar.
Background
As an important water transportation means, the self-safety guarantee and the information capture of the ship are paid more attention to, people actively utilize various means to improve the information refinement degree of the ship, and the aspects of the ship are perfected through a series of methods, but the shipborne environment with numerous cabins, complex structure and serious metal interference brings a series of dilemmas and constraints to gait information detection means.
The existing gait recognition and classification method generally acquires the gait of a person by a monitoring camera, acquires a gait video sequence through detection and tracking, and extracts the gait characteristics of the person through preprocessing analysis, but the recognition method needs to be carried out under the conditions of good light and no view field shielding, has limited recognition precision, is easy to imitate under special conditions, can not recognize or has low recognition efficiency under severe conditions, and can not realize rapid multi-user recognition. At the same time, identification with a camera may also involve personal privacy concerns. Therefore, in the case where the problem can be solved using the millimeter wave radar, the millimeter wave radar should be preferentially selected, and neither the influence of weather, light, or bad conditions is received, nor personal privacy is violated. However, most of the existing methods for performing gait recognition by using the millimeter wave radar are based on recognition under normal gait, and the target object cannot be correctly recognized when the target object is under other gait.
In summary, although some gait recognition classification techniques are mature, the more complex on-board environment has a great constraint on the application of these techniques. When the single millimeter wave radar is used for detecting gait, the single millimeter wave radar has limited visual field, and the ship has the scenes of vibration of the ship body, metal interference of the bulkhead and the like in navigation, and the factors can cause the problems of detection precision, incomplete characteristic acquisition and the like. Under the condition that the double millimeter wave radar is normally placed, signals acquired by the radar can be aliased, the upper radar can detect the characteristics of the lower limbs, and the lower radar can detect the characteristics of the upper limbs. Therefore, gait recognition classification based on the shipborne environment still does not have a comprehensive, systematic and pure characteristic method.
Disclosure of Invention
The invention aims to provide a gait recognition method under a ship environment based on a double millimeter wave radar, so as to realize accurate recognition of gait characteristics of personnel under a complex ship detection environment.
In order to solve the technical problems, the invention provides a technical scheme that: a gait recognition method under ship environment based on double millimeter wave radar comprises the following steps,
s1, setting a first millimeter wave radar and a second millimeter wave radar in a personnel detection area of a cabin, enabling the first millimeter wave radar to detect the upper limb characteristics of a detected target in a shipborne environment, enabling the second millimeter wave radar to detect the lower limb characteristics of the detected target in the shipborne environment, and enabling the detection ranges of the first millimeter wave radar and the second millimeter wave radar not to overlap;
s2, performing point cloud preprocessing on detection data of the first millimeter wave radar to obtain a 3D point cloud characteristic diagram of an upper limb, performing short-time Fourier transform on the detection data of the first millimeter wave radar and the detection data of the second millimeter wave radar to respectively obtain a speed-time characteristic diagram of the upper limb and a speed-time characteristic diagram of a lower limb, and performing fast Fourier transform on the speed-time characteristic diagram of the lower limb to obtain a rhythm frequency-speed characteristic diagram of the lower limb;
s3, building a mmGRnet network according to the set gait category; the method comprises the steps of inputting a mmGRnet network after denoising a 3D point cloud characteristic diagram of an upper limb, a speed-time characteristic diagram of the upper limb, a speed-time characteristic diagram of a lower limb and a rhythm frequency-speed characteristic diagram of the lower limb, and outputting the gait type of a measured target after the mmGRnet network is operated; wherein the mmGRnet network comprises an input layer, a convolution layer, an LSTM layer and a full connection layer;
the data form of the input layer is T multiplied by X multiplied by Y multiplied by 4, wherein T is the number of frames, and X, Y is determined by the data form of signals acquired by the first millimeter wave radar and the second millimeter wave radar; the convolution layer comprises 4 groups of parallel CNNs, and each group of CNNs is used for respectively processing a 3D point cloud characteristic map of the upper limb, a speed-time characteristic map of the lower limb and a rhythm frequency-speed characteristic map of the lower limb of each frame; the total number of LSTM layers is 4, and the LSTM layers respectively correspond to each group of CNNs, each LSTM layer comprises T unidirectional LSTM CELL, and the T unidirectional LSTM CELL of the same LSTM layer is used for respectively processing T frames of a certain type of feature map;
in mmGRnet networks, the loss function for a single feature map input is,
wherein L is s A loss function input for the single feature map; t is the number of the input feature image frames; k is sample data, and is obtained through experimental analysis; n is a gait type number; m is the total number of gait types; y is kn Is an actual gait type label; p is p kn A tag for predicting gait type;
the gait recognition loss function is a function of the gait recognition loss,
L＝loss f +ωL s
in the above, loss f Calculating losses for the network when the various feature graphs are fused; omega is the loss weight of the single feature map; l is gait recognition loss function.
According to the scheme, the point cloud preprocessing in S2 comprises point cloud clustering.
According to the scheme, the point cloud clustering specifically adopts a DBSCAN algorithm.
According to the scheme, the CA-CFAR is specifically adopted in the S3 to reduce noise.
According to the scheme, the gait types output by the mmGRnet network comprise normal walking, running, walking and non-walking crutch claudication.
The gait recognition device based on the mmGRnet network is used for realizing the processing of detection data of the first millimeter wave radar and the second millimeter wave radar and the recognition of the gait type of the detected target in the method, and comprises a detection data processing module and a gait type recognition module;
the detection data processing module performs point cloud preprocessing on detection data of the first millimeter wave radar to obtain a 3D point cloud characteristic diagram of an upper limb, performs short-time Fourier transform on the detection data of the first millimeter wave radar and the detection data of the second millimeter wave radar to respectively obtain a speed-time characteristic diagram of the upper limb and a speed-time characteristic diagram of a lower limb, and performs fast Fourier transform on the speed-time characteristic diagram of the lower limb to obtain a rhythm frequency-speed characteristic diagram of the lower limb; the detection data processing module is used for transmitting the processed detection data to the gait type recognition module after finishing processing the detection data of the first millimeter wave radar and the second millimeter wave radar;
the gait type recognition module is provided with a mmGRnet network established according to the set gait category, wherein the mmGRnet network comprises an input layer, a convolution layer, an LSTM layer and a full connection layer; after receiving the processed detection data from the detection data processing module, the gait type recognition module firstly carries out noise reduction on the processed detection data, then inputs the detection data into the mmGRnet network to carry out gait type recognition, and finally outputs the gait type of the detected target.
The system is used for realizing the ship environment gait recognition method based on the double millimeter wave radar, and comprises a gait detection device and a gait recognition device; wherein, the liquid crystal display device comprises a liquid crystal display device,
the gait detection device comprises a first millimeter wave radar and a second millimeter wave radar, wherein the first millimeter wave radar detects the upper limb characteristics of a detected target in a shipborne environment, the second millimeter wave radar detects the lower limb characteristics of the detected target in the shipborne environment, and the detection ranges of the first millimeter wave radar and the second millimeter wave radar are not overlapped;
the gait recognition device comprises a detection data processing module and a gait type recognition module, wherein the detection data processing module performs point cloud preprocessing on detection data of a first millimeter wave radar to obtain a 3D point cloud characteristic diagram of an upper limb, performs short-time Fourier transform on the detection data of the first millimeter wave radar and a second millimeter wave radar to respectively obtain a speed-time characteristic diagram of the upper limb and a speed-time characteristic diagram of a lower limb, and performs fast Fourier transform on the speed-time characteristic diagram of the lower limb to obtain a rhythm frequency-speed characteristic diagram of the lower limb; the detection data processing module is used for transmitting the processed detection data to the gait type recognition module after finishing processing the detection data of the first millimeter wave radar and the second millimeter wave radar; the gait type recognition module is provided with a mmGRnet network established according to the set gait category, wherein the mmGRnet network comprises an input layer, a convolution layer, an LSTM layer and a full connection layer; after receiving the processed detection data from the detection data processing module, the gait type recognition module firstly carries out noise reduction on the processed detection data, then inputs the detection data into the mmGRnet network to carry out gait type recognition, and finally outputs the gait type of the detected target.
The beneficial effects of the invention are as follows: 1. taking the characteristics of vibration of a ship body and metal reflection of the ship body in a cabin environment into consideration, respectively detecting the upper limb and the lower limb of a detected object in a special double millimeter wave radar setting mode; in the prior art, the double millimeter wave radar setting mode only places two millimeter wave radars at different heights in the vertical direction, and in the scheme, the detection angles of the millimeter wave radars are adjusted, so that the detection ranges of the first millimeter wave radar and the second millimeter wave radar are not repeated, the obtained upper limb features and lower limb features are purer, and the subsequent gait type recognition precision is improved. The invention also provides a method for recognizing the gait type by the mmGRnet network, which realizes the self-timing high-precision recognition of the gait in the ship environment, can ensure the recognition precision during short-time recognition and can reduce the calculated amount during long-time recognition.
2. Denoising the feature map by using a CA-CFAR algorithm; meanwhile, the DBSCAN algorithm is utilized to perform point cloud clustering, complete target characteristics are reserved, abnormal values are identified as noise, and interference caused by metal reflection in a ship environment is reduced.
Drawings
FIG. 1 is a schematic diagram of a first millimeter wave radar and a second millimeter wave radar according to an embodiment of the present invention;
FIG. 2 is a flow chart of a gait recognition method in a ship environment based on a dual millimeter wave radar according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of an mmGRnet network architecture according to an embodiment of the invention;
FIG. 4 is a graph of velocity versus time characteristics of a normally walking lower limb obtained in this example;
fig. 5 is a rhythm frequency-speed characteristic map of the normally walking lower limb obtained in the present embodiment;
FIG. 6 is a graph of velocity versus time characteristics of a normally walking upper limb obtained in this example;
FIG. 7 is a graph of velocity versus time characteristics of a running lower limb taken in this example;
fig. 8 is a graph of cadence frequency vs. speed characteristics of a running lower limb obtained in the present embodiment;
FIG. 9 is a graph of speed versus time characteristics of a running upper limb taken in this embodiment;
FIG. 10 is a graph of velocity versus time characteristics of the lower limb of the crutch lameness obtained in this example;
FIG. 11 is a graph of the cadence frequency versus speed profile of the lower limb of the crutch claudication taken in this example;
FIG. 12 is a graph of velocity versus time characteristics of the upper limb of the crutch lameness obtained in this example;
FIG. 13 is a graph of velocity versus time characteristics of the lower limb of the non-leaning crutch claudication obtained in this example;
FIG. 14 is a graph of the cadence frequency versus speed profile of the lower limb of the non-leaning crutch claudication taken in this example;
fig. 15 is a graph of the velocity versus time profile of an upper limb of an non-leaning crutch claudication taken in this example.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present disclosure more apparent, the technical solutions of the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present disclosure. It will be apparent that the described embodiments are some, but not all, of the embodiments of the present disclosure. All other embodiments, which can be made by one of ordinary skill in the art without the need for inventive faculty, are within the scope of the present disclosure, based on the described embodiments of the present disclosure.
Referring to fig. 2, a method for recognizing gait in a ship environment based on a dual millimeter wave radar, includes the steps of,
s1, setting a first millimeter wave radar and a second millimeter wave radar in a personnel detection area of a cabin, enabling the first millimeter wave radar to detect the upper limb characteristics of a detected target in a shipborne environment, enabling the second millimeter wave radar to detect the lower limb characteristics of the detected target in the shipborne environment, and enabling the detection ranges of the first millimeter wave radar and the second millimeter wave radar not to overlap;
s2, performing point cloud preprocessing on detection data of the first millimeter wave radar to obtain a 3D point cloud characteristic diagram of an upper limb, performing short-time Fourier transform on the detection data of the first millimeter wave radar and the detection data of the second millimeter wave radar to respectively obtain a speed-time characteristic diagram of the upper limb and a speed-time characteristic diagram of a lower limb, and performing fast Fourier transform on the speed-time characteristic diagram of the lower limb to obtain a rhythm frequency-speed characteristic diagram of the lower limb;
s3, building a mmGRnet network according to the set gait category; the method comprises the steps of inputting a mmGRnet network after denoising a 3D point cloud characteristic diagram of an upper limb, a speed-time characteristic diagram of the upper limb, a speed-time characteristic diagram of a lower limb and a rhythm frequency-speed characteristic diagram of the lower limb, and outputting the gait type of a measured target after the mmGRnet network is operated; wherein the mmGRnet network comprises an input layer, a convolution layer, an LSTM layer and a full connection layer.
The height of the first millimeter wave radar is positioned at or above the waist of the person, and the height of the second millimeter wave radar is positioned from the thigh to the waist of the person; the detection range of the first millimeter wave radar is inclined upwards and the lower boundary of the detection angle is horizontal, and the detection range of the second millimeter wave radar is inclined downwards and the upper boundary of the detection angle is horizontal; the setting distance between the first millimeter wave radar and the second millimeter wave radar and the detection area is calibrated according to the recognition condition of the experimental result.
The acquisition process of the detection data of the first millimeter wave radar and the second millimeter wave radar comprises the steps of transmitting signals through a plurality of transmitting antennas, receiving echo signals through a plurality of receiving antennas, mixing the transmitting signals with the echo signals to generate intermediate frequency signals, carrying out low-pass filtering on the intermediate frequency signals generated by mixing, inputting the low-pass filtered signals into an analog-digital converter to convert the low-pass filtered signals into discrete signals in a digital form, and finally processing the discrete signals to obtain the detection data. Wherein the probe data includes range information, micro-doppler information, and angle information of the target.
Further, the point cloud preprocessing in S2 includes point cloud clustering.
Further, the point cloud clustering specifically adopts a DBSCAN algorithm; DBSCAN has some great advantages over other clustering algorithms. First, it does not need to input the number of clusters to be partitioned. And secondly, even if the data points are very different, the data points are also included in the clusters, the characteristic of keeping the complete target characteristics in the ship environment, meanwhile, the DBSCAN can identify the abnormal value as noise, and the noise filtering parameters can be input when required, so that the interference of the ship metal environment can be effectively reduced by using the DBSCAN algorithm in the ship environment, the cloud characteristics of the upper limbs are purer, the detection precision is improved, and the due target characteristics can not be lost. After the desired 3d point cloud characteristics are obtained, a boundary line of the target point cloud can be found through a grid dividing method, the rough outline of the upper limb of the target object can be obtained as the identification characteristics after the target point cloud line is obtained, and based on the method, the embodiment also provides a method for detecting the height of the detected target, and the method is obtained by adding the length of the target point cloud boundary and the placement height of the first millimeter wave radar.
Furthermore, in S3, CA-CFAR is specifically adopted for noise reduction, and the noise reduction algorithm can effectively reduce the interference of uniform clutter and is suitable for the characteristic of stronger metal reflection (belonging to uniform clutter) in the ship environment.
Further, the data format of the input layer is t×x×y×4, where T is the number of frames, and X, Y is determined by the data format of signals collected by the first millimeter wave radar and the second millimeter wave radar (the millimeter wave radar adopted in the embodiment is iwr6843 millimeter wave radar, and the data format collected by the radar is a matrix of 128×256, so in the embodiment, x=128, y=256); the convolution layer comprises 4 groups of parallel CNNs, and each group of CNNs is used for respectively processing a 3D point cloud characteristic map of the upper limb, a speed-time characteristic map of the lower limb and a rhythm frequency-speed characteristic map of the lower limb of each frame; the LSTM layers are 4 in total and respectively correspond to each CNN group, each LSTM layer comprises T unidirectional LSTM CELL, and the T unidirectional LSTM CELL of the same LSTM layer is used for respectively processing T frames of a certain type of feature map.
Further defined gait categories include types of gait including normal walking, running, crutch lameness, non-crutch lameness. Further, in mmGRnet networks, the loss function for a single feature map input is,
wherein L is s A loss function input for the single feature map; t is the number of the input feature image frames; k is sample data, and is obtained through experimental analysis; n is a gait type number; m is the total number of gait types; y is kn Is an actual gait type label; p is p kn A tag for predicting gait type;
gait recognition loss function is l=loss f +ωL s
In the above, loss f Calculating losses for the network when the various feature graphs are fused; omega is the loss weight of the single feature map; l is gait recognition loss function.
The feature diagrams of the asynchronous state type obtained by the method provided by the embodiment are shown in fig. 4 to 15 in detail. It can be seen that the frequency distribution range of the rhythm frequency-speed profile of the lower limb during normal walking is smaller than that of the rhythm frequency-speed profile of the lower limb during running, because the frequency of the leg movement during running is higher, so that there are more frequency bins distributed in the rhythm frequency-speed profile. As can be seen from the comparison of the speed-time characteristic diagrams of the upper and lower limbs in fig. 4 and 6 with the speed-time characteristic diagrams of the upper and lower limbs in fig. 7 and 9, the waveform period during normal walking is longer than that during running, so that the two gaits of running and normal walking can be distinguished by the characteristics.
For both types of gait, i.e., lean-on crutch claudication and non-lean crutch claudication, it can be seen from fig. 11 and 14 that the frequency distribution range in the limp-on pitch frequency-speed characteristic diagram is small, and the waveform is narrow, because the frequency of leg movement is low and the motion is single, the frequency distribution range is small and is easily distinguished from normal walking and running. As can be seen from fig. 10, 12, 13, 15, the average speed of lameness is lower than during normal walking and running, so the waveform period in the characteristic map is longer. For the distinction between lameness and non-lean turns, it is difficult to make an effective distinction by the lower limb cadence frequency-speed profile of fig. 11 and 14, because the leg movement frequency of both gait is lower and the images reflected in the cadence frequency-speed profile are closer; as can be seen from the speed-time characteristic diagrams of the lower limbs in fig. 10 and 13, the waveform in fig. 10 is more virtual, and the waveform in fig. 13 is purer, because the movement of the crutch in the crutch state and the movement of the leg are mixed together to generate interference on the waveform, so that the waveforms of the speed-time characteristic diagrams of the lower limbs of the two gaits are obviously distinguished, and the situation of crutch and non-crutch in the limp state can be distinguished based on the waveforms.
The foregoing description is only illustrative of the present invention and is not intended to limit the scope of the invention, and all equivalent structures or equivalent processes or direct or indirect application in other related technical fields are included in the scope of the present invention.
Claims (7)
1. A ship environment gait recognition method based on double millimeter wave radar is characterized by comprising the following steps: comprises the steps of,
s1, setting a first millimeter wave radar and a second millimeter wave radar in a personnel detection area of a cabin, enabling the first millimeter wave radar to detect the upper limb characteristics of a detected target in a shipborne environment, enabling the second millimeter wave radar to detect the lower limb characteristics of the detected target in the shipborne environment, and enabling the detection ranges of the first millimeter wave radar and the second millimeter wave radar not to overlap;
s2, performing point cloud preprocessing on detection data of the first millimeter wave radar to obtain a 3D point cloud characteristic diagram of an upper limb, performing short-time Fourier transform on the detection data of the first millimeter wave radar and the detection data of the second millimeter wave radar to respectively obtain a speed-time characteristic diagram of the upper limb and a speed-time characteristic diagram of a lower limb, and performing fast Fourier transform on the speed-time characteristic diagram of the lower limb to obtain a rhythm frequency-speed characteristic diagram of the lower limb;
s3, building a mmGRnet network according to the set gait category; the method comprises the steps of inputting a mmGRnet network after denoising a 3D point cloud characteristic diagram of an upper limb, a speed-time characteristic diagram of the upper limb, a speed-time characteristic diagram of a lower limb and a rhythm frequency-speed characteristic diagram of the lower limb, and outputting the gait type of a measured target after the mmGRnet network is operated; wherein the mmGRnet network comprises an input layer, a convolution layer, an LSTM layer and a full connection layer;
the data form of the input layer is T multiplied by X multiplied by Y multiplied by 4, wherein T is the number of frames, and X, Y is determined by the data form of signals acquired by the first millimeter wave radar and the second millimeter wave radar; the convolution layer comprises 4 groups of parallel CNNs, and each group of CNNs is used for respectively processing a 3D point cloud characteristic map of the upper limb, a speed-time characteristic map of the lower limb and a rhythm frequency-speed characteristic map of the lower limb of each frame; the total number of LSTM layers is 4, and the LSTM layers respectively correspond to each group of CNNs, each LSTM layer comprises T unidirectional LSTM CELL, and the T unidirectional LSTM CELL of the same LSTM layer is used for respectively processing T frames of a certain type of feature map;
in mmGRnet networks, the loss function for a single feature map input is,
wherein L is s A loss function input for the single feature map; t is the number of the input feature image frames; k is sample data, and is obtained through experimental analysis; n is a gait type number; m is the total number of gait types; y is kn Is an actual gait type label; p is p kn A tag for predicting gait type;
the gait recognition loss function is a function of the gait recognition loss,
L＝loss f +ωL s
in the above, loss f Calculating losses for the network when the various feature graphs are fused; omega is the loss weight of the single feature map; l is gait recognition loss function.
2. The dual millimeter wave radar-based ship environment gait recognition method according to claim 1, wherein: s2, point cloud preprocessing comprises point cloud clustering.
3. The dual millimeter wave radar-based ship environment gait recognition method according to claim 2, wherein: the point cloud clustering specifically adopts a DBSCAN algorithm.
4. The dual millimeter wave radar-based ship environment gait recognition method according to claim 1, wherein: and S3, specifically adopting CA-CFAR to reduce noise.
5. The dual millimeter wave radar-based ship environment gait recognition method according to claim 1, wherein: the types of gait that are included in the set gait categories include normal walking, running, leaning crutch lameness, non-leaning crutch lameness.
6. Gait recognition device based on mmGRnet network, its characterized in that: the device is used for realizing the processing of detection data of the first millimeter wave radar and the second millimeter wave radar and the identification of the gait type of the detected target in the method of any one of claims 1 to 5, and comprises a detection data processing module and a gait type identification module;
the detection data processing module performs point cloud preprocessing on detection data of the first millimeter wave radar to obtain a 3D point cloud characteristic diagram of an upper limb, performs short-time Fourier transform on the detection data of the first millimeter wave radar and the detection data of the second millimeter wave radar to respectively obtain a speed-time characteristic diagram of the upper limb and a speed-time characteristic diagram of a lower limb, and performs fast Fourier transform on the speed-time characteristic diagram of the lower limb to obtain a rhythm frequency-speed characteristic diagram of the lower limb; the detection data processing module is used for transmitting the processed detection data to the gait type recognition module after finishing processing the detection data of the first millimeter wave radar and the second millimeter wave radar;
the gait type recognition module is provided with a mmGRnet network established according to the set gait category, wherein the mmGRnet network comprises an input layer, a convolution layer, an LSTM layer and a full connection layer; after receiving the processed detection data from the detection data processing module, the gait type recognition module firstly carries out noise reduction on the processed detection data, then inputs the detection data into the mmGRnet network to carry out gait type recognition, and finally outputs the gait type of the detected target.
7. A ship environment gait recognition system based on double millimeter wave radar is characterized in that: the system is used for realizing the ship environment gait recognition method based on the double millimeter wave radar according to any one of claims 1-5, and comprises a gait detection device and a gait recognition device; the gait detection device comprises a first millimeter wave radar and a second millimeter wave radar, wherein the first millimeter wave radar detects the upper limb characteristics of a detected target in a shipborne environment, the second millimeter wave radar detects the lower limb characteristics of the detected target in the shipborne environment, and the detection ranges of the first millimeter wave radar and the second millimeter wave radar are not overlapped;
the gait recognition device comprises a detection data processing module and a gait type recognition module, wherein the detection data processing module performs point cloud preprocessing on detection data of a first millimeter wave radar to obtain a 3D point cloud characteristic diagram of an upper limb, performs short-time Fourier transform on the detection data of the first millimeter wave radar and a second millimeter wave radar to respectively obtain a speed-time characteristic diagram of the upper limb and a speed-time characteristic diagram of a lower limb, and performs fast Fourier transform on the speed-time characteristic diagram of the lower limb to obtain a rhythm frequency-speed characteristic diagram of the lower limb; the detection data processing module is used for transmitting the processed detection data to the gait type recognition module after finishing processing the detection data of the first millimeter wave radar and the second millimeter wave radar; the gait type recognition module is provided with a mmGRnet network established according to the set gait category, wherein the mmGRnet network comprises an input layer, a convolution layer, an LSTM layer and a full connection layer; after receiving the processed detection data from the detection data processing module, the gait type recognition module firstly carries out noise reduction on the processed detection data, then inputs the detection data into the mmGRnet network to carry out gait type recognition, and finally outputs the gait type of the detected target.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310721908.0A CN116453227B (en) | 2023-06-19 | 2023-06-19 | Gait recognition method based on double millimeter wave radar under ship environment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310721908.0A CN116453227B (en) | 2023-06-19 | 2023-06-19 | Gait recognition method based on double millimeter wave radar under ship environment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116453227A CN116453227A (en) | 2023-07-18 |
| CN116453227B true CN116453227B (en) | 2023-09-19 |
Family
ID=87122386
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310721908.0A Active CN116453227B (en) | 2023-06-19 | 2023-06-19 | Gait recognition method based on double millimeter wave radar under ship environment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116453227B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106291524A (en) * | 2016-08-05 | 2017-01-04 | 南京理工大学 | LFMCW radar detection movement human mesh calibration method based on anthropometric dummy |
| EP3553552A1 (en) * | 2018-04-11 | 2019-10-16 | Aptiv Technologies Limited | Method for the recognition of a moving pedestrian |
| EP3695783A1 (en) * | 2019-02-15 | 2020-08-19 | Origin Wireless, Inc. | Method, apparatus, and system for wireless gait recognition |
| CN114529939A (en) * | 2022-01-17 | 2022-05-24 | 华南理工大学 | Pedestrian identification method based on millimeter wave radar point cloud clustering and deep learning |
| CN114563784A (en) * | 2022-04-27 | 2022-05-31 | 武汉理工大学 | Shipborne environment intrusion detection method and system based on double millimeter wave radar |
| CN115079118A (en) * | 2022-06-08 | 2022-09-20 | 西安电子科技大学 | Millimeter wave radar gait recognition method based on retrieval task |
| CN115616566A (en) * | 2022-10-12 | 2023-01-17 | 三维通信股份有限公司 | Gait recognition method, system and storage medium based on FMCW millimeter wave radar |
-
2023
- 2023-06-19 CN CN202310721908.0A patent/CN116453227B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106291524A (en) * | 2016-08-05 | 2017-01-04 | 南京理工大学 | LFMCW radar detection movement human mesh calibration method based on anthropometric dummy |
| EP3553552A1 (en) * | 2018-04-11 | 2019-10-16 | Aptiv Technologies Limited | Method for the recognition of a moving pedestrian |
| EP3695783A1 (en) * | 2019-02-15 | 2020-08-19 | Origin Wireless, Inc. | Method, apparatus, and system for wireless gait recognition |
| CN114529939A (en) * | 2022-01-17 | 2022-05-24 | 华南理工大学 | Pedestrian identification method based on millimeter wave radar point cloud clustering and deep learning |
| CN114563784A (en) * | 2022-04-27 | 2022-05-31 | 武汉理工大学 | Shipborne environment intrusion detection method and system based on double millimeter wave radar |
| CN115079118A (en) * | 2022-06-08 | 2022-09-20 | 西安电子科技大学 | Millimeter wave radar gait recognition method based on retrieval task |
| CN115616566A (en) * | 2022-10-12 | 2023-01-17 | 三维通信股份有限公司 | Gait recognition method, system and storage medium based on FMCW millimeter wave radar |
Non-Patent Citations (2)
| Title |
|---|
| A simulation model for ship navigation in the "Xiazhimen" waterway based on statistical analysis of AIS data;Xuri Xin.et.;《Ocean Engineering》;第180卷;第279-289页 * |
| 基于毫米波雷达在小样本场景下人体动作识别方法研究;潘波;《中国优秀硕士学位论文全文数据库 信息科技辑》(第2期);第I136-1821页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116453227A (en) | 2023-07-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107818326B (en) | A kind of ship detection method and system based on scene multidimensional characteristic | |
| CN110456320B (en) | Ultra-wideband radar identity recognition method based on free space gait time sequence characteristics | |
| CN105427314A (en) | Bayesian saliency based SAR image target detection method | |
| CN104834915B (en) | A kind of small infrared target detection method under complicated skies background | |
| CN100565557C (en) | System for tracking infrared human body target based on corpuscle dynamic sampling model | |
| CN109919862B (en) | Radar image denoising system and method and computer equipment | |
| CN106650250B (en) | Infrared Multi-Target threat sequercing method based on equivalent evaluation points | |
| CN111123212A (en) | Signal processing method of scene surveillance radar based on complex clutter background | |
| CN112487912B (en) | Arbitrary direction ship detection method based on improved YOLOv3 | |
| CN111323756B (en) | Marine radar target detection method and device based on deep learning | |
| CN111913177A (en) | Method and device for detecting target object and storage medium | |
| CN107103275A (en) | The vehicle detection carried out using radar and vision based on wheel and tracking | |
| CN114942434A (en) | Fall attitude identification method and system based on millimeter wave radar point cloud | |
| CN116259005A (en) | Intelligent monitoring system based on roof photovoltaic fire control | |
| CN113156417B (en) | Anti-unmanned aerial vehicle detection system, method and radar equipment | |
| CN114814832A (en) | Millimeter wave radar-based real-time monitoring system and method for human body falling behavior | |
| CN116148801B (en) | Millimeter wave radar-based target detection method and system | |
| CN104182992A (en) | Method for detecting small targets on the sea on the basis of panoramic vision | |
| CN115685102A (en) | Target tracking-based radar vision automatic calibration method | |
| CN116824335A (en) | YOLOv5 improved algorithm-based fire disaster early warning method and system | |
| CN116453227B (en) | Gait recognition method based on double millimeter wave radar under ship environment | |
| CN111736156B (en) | Upwind area identification method and device based on weather radar | |
| CN116310837B (en) | SAR ship target rotation detection method and system | |
| CN115909086A (en) | SAR target detection and identification method based on multistage enhanced network | |
| CN110363128A (en) | Airport runway foreign matter detection method, system and medium based on biological vision |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |